Janis
9bb8f119b5
The /explain endpoint blocked on a synchronous NIM call (llama-3.3-70b, 30-60s+) while nginx's default 60s proxy-read-timeout fired first, surfacing every AI request as a 504 Gateway Timeout. Switch the NIM proxy to streaming (stream:true) and relay its SSE to the browser via a new _emit() helper; the client now reads resp.body incrementally and renders tokens as they arrive. With chunks flowing, nginx's read-timeout resets per chunk so the 60s wall is gone, and the frozen spinner is replaced by live text. Ingress: disable proxy-buffering (else nginx buffers the whole response, defeating streaming) and set proxy-read-timeout to 120s as the inter-chunk gap budget. Server urlopen timeout 60->110s. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
33 lines
1.1 KiB
YAML
Executable File
33 lines
1.1 KiB
YAML
Executable File
apiVersion: networking.k8s.io/v1
|
|
kind: Ingress
|
|
metadata:
|
|
name: spark-analytics-webview-ingress
|
|
labels:
|
|
app.kubernetes.io/name: spark-analytics
|
|
app.kubernetes.io/part-of: nowchess
|
|
annotations:
|
|
cert-manager.io/cluster-issuer: "letsencrypt-prod"
|
|
nginx.ingress.kubernetes.io/ssl-redirect: "true"
|
|
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
|
|
# AI /explain streams Server-Sent Events — disable response buffering so
|
|
# chunks flush to the client immediately, and allow gaps between chunks.
|
|
nginx.ingress.kubernetes.io/proxy-buffering: "off"
|
|
nginx.ingress.kubernetes.io/proxy-read-timeout: "120"
|
|
spec:
|
|
ingressClassName: nginx
|
|
rules:
|
|
- host: spark-results-st.nowchess.janis-eccarius.de
|
|
http:
|
|
paths:
|
|
- path: /
|
|
pathType: Prefix
|
|
backend:
|
|
service:
|
|
name: spark-analytics-webview
|
|
port:
|
|
number: 8080
|
|
tls:
|
|
- hosts:
|
|
- spark-results-st.nowchess.janis-eccarius.de
|
|
secretName: spark-analytics-webview-staging-cert
|