Repository navigation
Provider API: severe queueing during evening peak (UTC+8) — minimal requests take 5–60s to first byte, some never complete #999
Description
Summary
For two consecutive evenings (Oct 6–7, 2026), https://api.commandcode.ai/provider/v1 shows extreme and highly variable latency during the evening peak (~11:00–16:00 UTC / 19:00–24:00 UTC+8). Even trivial requests are affected, so this is not long-context related — it looks like server-side queueing plus unstable routing to the Cloudflare edge. The delay is model-independent (DeepSeek V4.1 Flash, DeepSeek V4 Flash and MiMo V2.6 Flash are all affected), which points at gateway queueing rather than any single backend model.
Measured evidence
Minimal requests (client-independent, plain curl against /responses)
A 20-output-token "say hi" request, stream: true, models deepseek/deepseek-v4.1-flash (my primary) and xiaomi/mimo-v2.6-flash:
| When | TTFB | Total | Result |
|---|---|---|---|
| Oct 6 ~23:00 UTC+8 | 6.7–8.0 s | 7.9–22 s | completed (4 runs, both models) |
| Oct 7 ~23:00 UTC+8 | — | >60 s | timeout, zero bytes returned |
| Oct 7 ~23:15 UTC+8 | 5.9 s | 18.5 s | completed |
| Oct 7 ~23:20 UTC+8 | 8.6 s | >45 s | not finished when client gave up |
| Oct 7 ~23:40 UTC+8 | 9.1 s | >30 s | not finished when client gave up |
| Oct 7 ~23:55 UTC+8 (DS 4.1) | 4.8–7.4 s | 5.9–19.3 s | completed (2 runs) |
GET /models (no body, trivial)
- Total time varies between 0.9 s and 20 s across runs, TLS handshake alone fluctuates 0.6 s – 13 s.
- Same behavior direct-connect and through a local proxy — the variable part is on the service side.
Client-side logs (agentic app, ~1,500 requests over 2 days)
- Oct 6: 1,121 model requests — median 14.6 s, p90 50 s, max 410 s; 19
network_errorretries + 1 timeout, 41 requests retried at least once (one retried 3×, total 69.8 s). - Oct 7: 421 model requests — median 9.9 s, p90 31 s, max 516 s (~8.6 minutes for a single request).
- Also saw multiple
502 Stream ended unexpectedlystyle failures (matches Recurring 502 "Stream ended unexpectedly before completion" on large sessions — turns lost after retry exhaustion (macOS, v0.1.47) #991 / 502 Stream ended unexpectedly repeatedly when context is image-heavy (~23 MB request payload) #997).
Route note
ICMP to the resolved Cloudflare edge (104.21.49.202) shows ~66% loss / ~280 ms RTT from CN, but the decisive factor is server-side: even when TLS completes in <1 s, first byte still takes 5–9 s on trivial requests.
Environment
- Endpoint:
https://api.commandcode.ai/provider/v1(OpenAI Responses API) - Models:
deepseek/deepseek-v4.1-flash(primary); also affected:xiaomi/mimo-v2.6-flash,deepseek/deepseek-v4-flash - Region: mainland China; tested both direct and via HTTP proxy — same results
- Time of observation: 19:00–24:00 UTC+8, Oct 6 and Oct 7, 2026
Request
Please look into gateway queueing / connection pool exhaustion on the Provider API during peak hours. Even a status page or a 429 with retry-after would be better than silent 60s stalls.
Expected Behavior
A minimal streaming request (20 output tokens) should return the first byte within ~1–2 s and complete within a few seconds. GET /models should complete in well under 1 s. During Oct 6–7 evenings (UTC+8), first byte took 5–60 s and some requests never completed at all.
Actual Behavior
See the measured table in the description. During evening peak (19:00–24:00 UTC+8), TTFB on trivial requests ranges 5–9 s with totals exceeding 30–60 s (client abandoned), and one request returned zero bytes before a 60 s timeout. The delay is identical across DeepSeek V4.1 Flash, DeepSeek V4 Flash and MiMo V2.6 Flash. Client logs show single requests lasting up to 516 s and p90 ~31–50 s over ~1,500 requests.
Steps to reproduce the issue
- During evening peak (19:00–24:00 UTC+8), from mainland China (direct or via proxy — both affected), run:
- curl -X POST https://api.commandcode.ai/provider/v1/responses -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" -d '{"model":"deepseek/deepseek-v4.1-flash","input":"say hi","stream":true,"max_output_tokens":20}'
- Observe TTFB (
-w "%{time_starttransfer}"): typically 5–9 s; on some attempts the stream does not complete within 30–60 s. - Also run curl -w "%{time_total}" on GET https://api.commandcode.ai/provider/v1/models — total time fluctuates 0.9–20 s across runs.
Command Code Version
0.16.9
Operating System
Windows
Terminal/IDE
Windows Terminal (Git Bash)
Shell
Git Bash (bash)
Session file (optional)
No response
Fix prompt (optional)
No response
Additional context
No response
Activity
masonjian commentedon Oct 8, 2026
I've also experienced intermittent disconnections during normal use with a Command Code GOAT subscription this afternoon (October 8, UTC+8). Is there any known incident or recommended workaround?
xiaolidada commentedon Oct 11, 2026
Confirming the same behaviour from mainland China (2026-10-11 02:04–02:20 CST, i.e. outside the evening peak you describe), endpoint
https://api.commandcode.ai/provider/v1/chat/completions, modeldeepseek/deepseek-v4.1-flash, direct connection (no proxy), same machine as the control below.Workload probe — 12 requests, 8 succeeded / 4 failed
Successful requests ranged 1.65 s → 17.4 s (a 10× spread on identical work).
Bare handshake —
GET /provider/v1/models(no body), 3 runstotal 3.66 s / 16.16 s / 30 s timeout, zero bytes (TCP connect 0.21 s and 0.49 s; TLS handshake 3.37 s and 14.15 s).
Control 1 — same machine, same minute, first-party upstream
https://api.deepseek.com/v1/chat/completions: 8/8 success, connect 43–73 ms, TTFB 64–126 ms, total 0.48–1.16 s.Control 2 — same Cloudflare edge
api.commandcode.ailands on the LAX edge (cf-ray: …-LAX).www.cloudflare.com, which resolves to the same LAX edge from this machine, answered in 0.82 s / 0.88 s / 0.89 s (three runs) during the same minute in which the commandcode zone produced a 40 s no-response connection and TLS handshakes of 0.73 s / 2.16 s. So the loss is not on the China→Cloudflare path.Client-side logs (this machine, last 48 h)
142
Connection error, 123ReadTimeout, 109APITimeoutError, 23 ×504, 9 ×429. An auxiliary vision call routed toapi.commandcode.aihit the 120 s timeout 11 times and fell back to another provider.Could you confirm whether the gateway pool is sized for the UTC+8 evening peak, and whether there is any ETA? Two open reports (#999, #1009) with no acknowledgement makes it hard to plan around this.