{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreiftjbzg2jvf66rtewfo2x53qbkuoxektzgrbbjwdjhdtbrmlt3yoi",
    "uri": "at://did:plc:25rdn5elo5izoxrmtis34zuk/app.bsky.feed.post/3mploj6aauoi2"
  },
  "coverImage": {
    "$type": "blob",
    "ref": {
      "$link": "bafkreigjsvf6l5olgqs27j5w6uormq37vepxim2zqqexuqbwlkuk4gc42y"
    },
    "mimeType": "image/webp",
    "size": 582024
  },
  "path": "/smeetgohel/api-error-codes-a-test-suite-pattern-i-stole-from-stripe-3ia4",
  "publishedAt": "2026-07-01T13:36:30.000Z",
  "site": "https://dev.to",
  "tags": [
    "api",
    "backend",
    "testing",
    "webdev",
    "spin up a free trial to try this catalog pattern"
  ],
  "textContent": "_Read Stripe's API reference for an hour and you'll notice every endpoint has a complete enumerated list of error codes with example payloads. Then look at your own API._\n\nThat comparison can be a little uncomfortable.\n\nStripe doesn't treat errors as an afterthought. Their documentation gives error responses nearly as much attention as successful ones. Every endpoint explains not only what can go right, but exactly what can go wrong, including structured error codes, HTTP status codes, descriptions, and example payloads.\n\nMost APIs aren't like that.\n\nThey document `200 OK` responses in detail while reducing failures to a short table:\n\n  * 400 Bad Request\n  * 401 Unauthorized\n  * 404 Not Found\n  * 500 Internal Server Error\n\n\n\nThe business errors—the ones your users actually encounter—are often hidden inside controller code, scattered across wiki pages, or not documented at all.\n\nThe same imbalance usually exists in the test suite.\n\nHundreds of happy-path tests.\n\nVery few negative tests.\n\nA few years ago, I borrowed a simple idea from Stripe and turned it into one of the most valuable testing patterns we've adopted.\n\nInstead of writing negative tests one by one, we created a centralized **error-code catalog** and generated much of the negative test suite directly from it.\n\nThe result wasn't just better **API error code testing**.\n\nIt also improved documentation, reduced maintenance, and made breaking changes much harder to introduce accidentally.\n\nHere's how the pattern works.\n\n#  Why Error Responses Are Part of Your API Contract\n\nMany teams unconsciously treat successful responses as the \"real\" API.\n\nEverything else is considered an exception.\n\nThat mindset creates fragile systems.\n\nImagine a payment API.\n\nA successful request returns:\n\n\n\n    {\n      \"paymentId\": \"PAY-1001\",\n      \"status\": \"Succeeded\"\n    }\n\n\nEasy enough.\n\nNow consider all the legitimate failure cases:\n\n  * Card expired\n  * Insufficient funds\n  * Duplicate transaction\n  * Currency not supported\n  * Merchant suspended\n  * Fraud detected\n  * Payment amount exceeds limit\n\n\n\nThose aren't bugs.\n\nThey're expected business outcomes.\n\nIf your consumers must handle them, then those responses are every bit as much a part of the API contract as the successful response.\n\nOnce you accept that idea, testing strategy changes dramatically.\n\n#  The Error-Code Catalog as a Test Input\n\nThe foundation of this approach is maintaining a single catalog of every business error the API intentionally exposes.\n\nFor example:\n\n\n\n    errors:\n\n      USER_NOT_FOUND:\n        httpStatus: 404\n        message: User not found\n\n      EMAIL_ALREADY_EXISTS:\n        httpStatus: 409\n        message: Email already exists\n\n      INVALID_TOKEN:\n        httpStatus: 401\n        message: Invalid authentication token\n\n      PAYMENT_DECLINED:\n        httpStatus: 402\n        message: Payment declined\n\n      ORDER_ALREADY_SHIPPED:\n        httpStatus: 409\n        message: Order cannot be modified\n\n\nNotice what's happening here.\n\nWe're no longer documenting HTTP status codes alone.\n\nWe're documenting business outcomes.\n\nEvery new error introduced by engineering must first appear inside this catalog.\n\nThat simple rule creates a surprising amount of consistency.\n\n##  Why a Catalog Matters\n\nWithout one:\n\n  * Developers invent new response formats.\n  * Documentation drifts.\n  * QA forgets edge cases.\n  * Frontend developers discover failures during integration.\n\n\n\nWith one:\n\n  * Documentation stays centralized.\n  * Consumers understand every supported failure.\n  * Every error becomes automatically testable.\n\n\n\nThe catalog becomes an executable specification.\n\n#  One Test Per Error Code, Generated from the Catalog\n\nOnce every supported error exists in one place, generating baseline negative tests becomes straightforward.\n\nInstead of manually writing dozens of nearly identical scenarios, a generator simply walks through the catalog.\n\nConceptually:\n\n\n\n    for (const error of errorCatalog) {\n        generateNegativeTest(error);\n    }\n\n\nEach generated test validates:\n\n  * Expected HTTP status\n  * Business error code\n  * Error message\n  * Response schema\n\n\n\nFor example, suppose the catalog contains:\n\n\n\n    EMAIL_ALREADY_EXISTS\n\n\nThe generated scenario becomes:\n\n  1. Create a customer.\n  2. Attempt to create the same customer again.\n  3. Verify:\n\n\n\n\n    {\n      \"code\": \"EMAIL_ALREADY_EXISTS\",\n      \"message\": \"Email already exists\"\n    }\n\n\nNo engineer needed to remember to write that negative test.\n\nAdding a new business error automatically creates a new baseline test.\n\n##  Why This Scales\n\nImagine:\n\n  * 180 endpoints\n  * 95 business error codes\n\n\n\nWithout automation, every additional error increases maintenance.\n\nWith generation:\n\n  * Documentation grows.\n  * Test coverage grows.\n  * Maintenance barely changes.\n\n\n\nEngineers can spend their time writing meaningful business scenarios instead of repetitive validation tests.\n\n#  The Shape Assertion That Prevents Silent Error Drift\n\nChecking only the HTTP status code is one of the biggest mistakes in **error response testing**.\n\nConsider this response:\n\n\n\n    {\n      \"code\": \"USER_NOT_FOUND\",\n      \"message\": \"User not found\",\n      \"requestId\": \"abc123\"\n    }\n\n\nMonths later, someone simplifies the global exception handler.\n\nNow the API returns:\n\n\n\n    {\n      \"error\": \"User not found\"\n    }\n\n\nThe endpoint still returns:\n\n\n\n    404\n\n\nMost tests still pass.\n\nMeanwhile:\n\n  * Mobile applications break.\n  * Frontend parsing fails.\n  * Monitoring dashboards stop correlating request IDs.\n\n\n\nNobody notices until production.\n\n##  Shape Assertions\n\nTo prevent this, every generated negative test also validates the structure of the response.\n\nFor example:\n\n\n\n    expect(response.body).toEqual({\n\n        code: expect.any(String),\n\n        message: expect.any(String),\n\n        requestId: expect.any(String)\n\n    });\n\n\nNotice we're validating more than values.\n\nWe're validating the response contract itself.\n\nThat single assertion catches:\n\n  * Missing fields\n  * Renamed properties\n  * Structural changes\n  * Serialization mistakes\n\n\n\nbefore consumers experience them.\n\n##  Why It Matters\n\nMany API consumers depend on fields such as:\n\n  * Error code\n  * Message\n  * Correlation ID\n  * Documentation URL\n  * Retry hint\n\n\n\nChanging any of those silently becomes a breaking API change.\n\nShape assertions make those changes impossible to miss.\n\n#  Keeping the Catalog in Sync With the Code (Code Generation)\n\nThe obvious concern is maintenance.\n\nNobody wants to update:\n\n  * Source code\n  * Documentation\n  * Tests\n  * Error catalog\n\n\n\nmanually every time a new error appears.\n\nFortunately, most modern applications already define business errors centrally.\n\nExample:\n\n\n\n    export enum ErrorCode {\n\n        USER_NOT_FOUND,\n\n        PAYMENT_DECLINED,\n\n        INVALID_TOKEN,\n\n        EMAIL_ALREADY_EXISTS\n\n    }\n\n\nFrom this single definition, it's possible to generate:\n\n  * Markdown documentation\n  * OpenAPI components\n  * Error catalogs\n  * Client SDK constants\n  * Negative tests\n\n\n\nEverything derives from one source.\n\nThat dramatically reduces maintenance.\n\n##  Additional Benefits\n\nOnce generation is introduced:\n\n###  Documentation Never Falls Behind\n\nThe documentation updates whenever the enum changes.\n\n###  Generated Tests Stay Current\n\nEvery new business error immediately receives baseline coverage.\n\n###  SDKs Stay Consistent\n\nFrontend applications can reference generated constants rather than string literals.\n\n###  Reviews Improve\n\nAdding a new business error becomes visible during pull request review.\n\nInstead of hiding inside controller code, it's now part of the API contract.\n\n#  The Two Error Codes We Deliberately Don't Test\n\nAlthough the catalog covers almost every business failure, there are two categories we intentionally exclude.\n\n##  1. Generic Internal Server Errors\n\nExample:\n\n\n\n    500 Internal Server Error\n\n\nThese represent unexpected failures.\n\nThey're not business behavior.\n\nInstead of attempting to trigger every possible server crash, we simply verify:\n\n  * Sensitive stack traces aren't exposed.\n  * Generic messages are returned.\n  * Request IDs are included.\n  * Logging works correctly.\n\n\n\nTesting every internal failure path produces little value.\n\nTesting the response contract provides much more.\n\n##  2. Infrastructure Failures\n\nExamples include:\n\n  * Database unavailable\n  * Kafka offline\n  * Redis unreachable\n  * DNS failure\n  * Cloud storage outage\n\n\n\nThese aren't business errors.\n\nThey're infrastructure events.\n\nWe test them separately using:\n\n  * Chaos engineering\n  * Fault injection\n  * Resilience testing\n  * Disaster recovery exercises\n\n\n\nKeeping them outside the regular **API negative tests** avoids unnecessary instability in CI pipelines.\n\n#  Unexpected Benefits\n\nAfter adopting this approach, several improvements appeared that we hadn't anticipated.\n\n###  Better Documentation\n\nEngineers could browse every supported business error in one place.\n\n###  Cleaner APIs\n\nEvery endpoint returned a consistent error structure.\n\n###  Faster Reviews\n\nNew business errors became obvious during pull requests.\n\n###  Happier Frontend Teams\n\nConsumers no longer guessed which failures might occur.\n\n###  Stronger Regression Protection\n\nStructural changes to error responses surfaced immediately.\n\n#  Final Thoughts\n\nMost API teams invest enormous effort in testing successful requests while giving comparatively little attention to failures.\n\nStripe's documentation demonstrates a different philosophy.\n\nErrors are part of the public API contract.\n\nThey deserve documentation.\n\nThey deserve consistency.\n\nAnd they deserve automated tests.\n\nBy maintaining an error-code catalog, generating one baseline test per error, validating response shapes, and deriving documentation from code, you can significantly reduce maintenance while increasing confidence that your API behaves consistently—even as it evolves.\n\nThe best part is that this approach scales naturally.\n\nAs new business errors appear, your documentation and test suite grow automatically rather than relying on engineers to remember yet another negative test.\n\nIf you're looking to automate this style of contract-driven testing, you can **spin up a free trial to try this catalog pattern** and explore how generated negative tests, schema validation, and API contracts can work together to keep your error handling consistent over time.",
  "title": "API Error Codes: A Test Suite Pattern I Stole from Stripe"
}