Circuit breakers stop repeated calls to a failing vendor endpoint from cascading into your own timeouts and resource exhaustion.
Choose failure signals that match the vendor contract
Track only the responses that indicate the vendor cannot process work right now. Timeouts, 5xx status codes, and connection resets are reliable signals. 4xx responses usually mean the request itself is invalid, so they should not increment the failure counter.
Set the threshold on consecutive failures rather than a percentage over a long window. A sliding counter that resets after a successful call avoids tripping on transient noise while still catching sustained outages within seconds.
Include the specific HTTP status codes and timeout values in the configuration so operators can adjust them without code changes when the vendor updates its error behavior.
Implement the three-state machine with explicit timers
Closed state passes every request to the vendor and counts failures. After the configured number of consecutive failures, transition to open. Record the exact time of the transition so the open duration can be measured precisely.
Open state rejects calls immediately with a local error that your application can turn into a clear message or fallback. Do not attempt any outbound request during this period; the goal is to let the vendor recover without additional load.
After the open timeout expires, move to half-open. Allow a single probe request. Success closes the circuit. Any failure returns it to open and restarts the open timer. This single-probe rule prevents thundering-herd effects during recovery.
Return useful degraded responses while open
When the breaker is open, surface the failure at the call site rather than letting the request hang. Return a typed error that upper layers can map to cached data, a queue, or an explicit "temporarily unavailable" response to the end user.
Log the breaker state change with the current failure count and the vendor endpoint. Include the same structured fields on every transition so downstream dashboards can alert on open duration without parsing free-form text.
If your application already uses a retry library, disable retries for calls that hit an open breaker. Retries only increase latency when the circuit has already decided the vendor is unreachable.
Test the breaker with controlled failure injection
Add a test mode that forces the breaker into open or half-open without touching the real vendor. Use it in CI to verify that dependent code paths handle the local error and that metrics are emitted correctly.
Run periodic synthetic probes from a separate process that does not share the breaker state. This gives an independent view of vendor health and prevents the application breaker from masking longer-term degradation.