When we say credit decisioning latency is a product feature, we mean it literally. For a credit offer embedded in a checkout flow, the decision has to return before the user notices the wait. That is not a performance optimization target; it is a conversion requirement. A user who submits an application and watches a spinner for three seconds is not in the same mental state as one who sees the decision appear before they have fully processed that they clicked the button. The second experience converts. The first creates doubt.
The sub-200ms P99 target we hold for the Lendforge decision API is the result of that product constraint working backward into the infrastructure design. This post is about how we get there and what tradeoffs we made along the way.
What the Decision Endpoint Actually Does in 200ms
The request to POST /v1/applications triggers several operations that have to complete within the latency budget. Understanding what is happening in that window clarifies why the engineering problem is interesting.
Identity resolution runs first. The applicant's SSN, name, date of birth, and address need to be validated against the bureau data source. For a standard soft-pull pre-qualification, this is an OFAC check and name-match validation. For a full hard-pull application, it includes a tradeline query from at least one bureau. Bureau response latency is the variable we control least: it depends on the bureau's infrastructure, not ours. Typical bureau response times at P50 are in the 50-100ms range, but tails can run to 400-600ms under load. Managing bureau latency is a significant portion of our infrastructure work.
Feature computation runs in parallel with the bureau query wherever possible. Platform behavioral features that were pre-computed from the event ingestion batch are available immediately from cache. Derived features that require real-time computation (trailing 7-day velocity from the most recent event batch, for example) run asynchronously alongside the bureau call and merge into the feature vector when both are ready.
Model inference is the most predictable step. A gradient-boosted tree model with the feature dimensionality we use scores a single applicant in under 2ms on current hardware. The model inference step is not where latency comes from; it is effectively free relative to the other components.
SHAP factor decomposition adds some overhead. Computing per-feature Shapley values for the full feature set is more expensive than the model forward pass. We pre-compute SHAP background datasets at model load time and use the TreeExplainer approximation, which keeps per-sample SHAP computation in the 5-15ms range rather than the seconds that exact computation would take on large feature sets.
Response serialization and network egress are the final steps. JSON serialization of the decision response including the factor array, signed with our response authentication scheme, adds roughly 1ms. Network egress time from our edge nodes to the client adds 10-40ms depending on geographic proximity. We deploy decision nodes in three US regions to keep P99 round-trip times within budget from anywhere in the continental US.
The Bureau Latency Problem
Bureau query latency is the hardest part of the problem because it is the least in our control. Our approach has several layers.
The first is bureau connection pooling with circuit breaking. Maintaining persistent connections to bureau endpoints with connection pool management reduces the per-query overhead of connection setup. When a bureau endpoint shows elevated latency or error rates (the circuit breaker threshold is configurable), we route to an alternate bureau rather than queueing behind a slow endpoint. Most platforms configure at least two bureau connections so this fallback is always available.
The second is pre-qualification caching. When a platform uses the Lendforge pre-qualification flow, we run a soft-pull bureau check at the point of pre-qual. If the full application comes in within a configurable window (default 15 minutes), we use the pre-qual bureau data for the full decision rather than making a second bureau call. This effectively moves the bureau latency to the pre-qualification request, which happens before the user initiates checkout and is not blocking. The full application decision then runs without a live bureau call, which drops P99 latency dramatically for platforms that use the pre-qual flow.
The third is a confidence-based fallback. If both bureau endpoints are degraded and the bureau call is not completing within the latency budget, we can fall back to platform-data-only scoring for applicants with sufficient platform history. The fallback decision has wider confidence intervals and may result in a more conservative credit limit offer, but it avoids showing an error to an applicant whose application was blocked by a bureau outage. This fallback is off by default and requires explicit configuration because it changes the risk characteristics of the decision population.
Feature Caching and the Pre-Ingestion Pattern
Platform behavioral features are the other major variable in decision latency. If you send us a user's full transaction history as part of the application request, we compute features on the fly, which adds computation time proportional to the event volume. If you pre-ingest the event stream through POST /v1/events on an ongoing basis, we maintain a live feature cache that is available instantly when an application arrives.
The feature cache is keyed by your platform's user identifier. When an application arrives for a user whose features are cached, the feature retrieval is a cache hit at roughly 1ms. When the user is new or their events have not been ingested, features compute from the inline context payload at typically 10-30ms for a 90-day event history. For power users with multi-year histories, inline computation can push toward 80-100ms, which matters when you are working with a 200ms total budget.
For most checkout integrations, the recommended pattern is to pre-ingest events through a background job that runs on your side, sending new events to the Lendforge ingest endpoint in near-real-time or at low-latency batched intervals. Your checkout application call then uses pre-computed features and runs in the fast path. This keeps P99 decision latency in the sub-150ms range for the typical applicant.
Where We Accept Latency Tradeoffs
We are not claiming that sub-200ms is achievable for every decision in every configuration. There are tradeoffs we made explicitly.
Full income verification decisions, where the model requires a connection to an account aggregator for real-time bank data, operate outside the 200ms budget. Bank data aggregation APIs have latency profiles in the 2-8 second range. For product configurations that require income verification, the application flow shows a "verifying your information" step that sets a user expectation of a few seconds rather than an instant. The credit decision is not the bottleneck for those applications; the bank data retrieval is.
Manual review decisions (the "referred" state) also operate outside the synchronous latency budget by definition. These are applications where the model confidence is insufficient for an automated decision. We return the referred status immediately (within the standard latency budget), and the actual final decision comes through the application.updated webhook after human review. The latency from the user's perspective is hours, not milliseconds. The synchronous response simply tells them the application is being reviewed.
High-volume batch decisioning, where a platform wants to pre-score large segments of its user base for pre-qualification campaigns, runs in a different mode entirely. Batch jobs are processed with higher parallelism and lower per-job latency targets. Batch processing uses the same model and the same feature computation pipeline; only the request routing and resource allocation differ.
Monitoring Latency in Production
The Lendforge metrics dashboard exposes P50, P95, and P99 decision latency broken down by component: bureau time, feature computation time, model inference time, and total response time. For production integrations, we recommend setting up an alert on P99 total decision latency that fires when it exceeds 300ms for more than 5 consecutive minutes. That threshold gives you warning time before the tail latency starts affecting conversion in checkout flows.
The most common cause of P99 latency spikes we see in production: a platform stopped pre-ingesting events and the feature cache went stale, so applications started hitting the inline computation path at high volume. The second most common: a bureau endpoint degraded without triggering the circuit breaker threshold, so bureau calls were completing slowly rather than failing and routing to the backup. Both are detectable from the component breakdown in the metrics dashboard before they become visible as user experience issues.
Latency is a discipline, not a one-time optimization. The sub-200ms target requires consistent attention to the full component stack, not just the model inference layer where the interesting ML work happens. The less glamorous parts, connection pooling, cache management, circuit breaking, geographic routing, are where the P99 budget is actually won or lost.