The /explain endpoint blocked on a synchronous NIM call (llama-3.3-70b,
30-60s+) while nginx's default 60s proxy-read-timeout fired first,
surfacing every AI request as a 504 Gateway Timeout.
Switch the NIM proxy to streaming (stream:true) and relay its SSE to the
browser via a new _emit() helper; the client now reads resp.body
incrementally and renders tokens as they arrive. With chunks flowing,
nginx's read-timeout resets per chunk so the 60s wall is gone, and the
frozen spinner is replaced by live text.
Ingress: disable proxy-buffering (else nginx buffers the whole response,
defeating streaming) and set proxy-read-timeout to 120s as the
inter-chunk gap budget. Server urlopen timeout 60->110s.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Janis
9bb8f119b5