## Plan: Fix Fast Mode 20-Second Delay in `/api/interpret/click/tun` ### Problem After `{"type": "generating", "source": "fast"}`, the user waits ~20 seconds with no SSE events before any `token` starts streaming. Root causes: 1. `deepseek-v4-pro` has high TTFT (prompt evaluation ~15-30s) 2. Reasoning tokens (`reasoning_content`) silently discarded — only `chunk.content` checked 3. No heartbeat/progress events during LLM wait (unlike review pipeline) ### Changes (all in `api/routes.py`) **File: `api/routes.py`** --- ### Change 1 — `_get_chat_model()`: Switch fast mode to flash model + add timeout Modify the function to: - Accept `fast_mode: bool = True` parameter - When `fast_mode=True`, read `SUMMARY_MODEL` env var (`deepseek-v4-flash`) instead of `DEEPAGENT_MODEL` (`deepseek-v4-pro`) - Add `timeout=180` and `max_retries=2` to `init_chat_model()` call (matching `vent_agent.py` pattern) **Rationale**: `deepseek-v4-flash` is designed for low latency; fast mode just fills a fixed template and doesn't need the heavyweight reasoning of the pro model. This alone could reduce TTFT from ~20s to ~2-5s. --- ### Change 2 — `_fast_tun_sse_generator()`: Add heartbeat + reasoning_content handling Replace lines 413-417 (the simple `async for chunk in model.astream(messages)` loop) with: ```python import asyncio, time first_token = False heartbeat_start = time.time() astream_iter = model.astream(messages) while True: try: if first_token: chunk = await astream_iter.__anext__() else: # 3-second timeout for first token — send heartbeat if slow chunk = await asyncio.wait_for(astream_iter.__anext__(), timeout=3.0) if not first_token: first_token = True except asyncio.TimeoutError: elapsed = int(time.time() - heartbeat_start) yield f"data: {json.dumps({'type': 'progress', 'message': f'模型正在生成报告(已等待 {elapsed} 秒)...'}, ensure_ascii=False)}\n\n" continue except StopAsyncIteration: break # Handle reasoning content (R1-style models) reasoning = chunk.additional_kwargs.get("reasoning_content", "") if hasattr(chunk, "additional_kwargs") else "" if reasoning: yield f"data: {json.dumps({'type': 'reasoning', 'content': reasoning}, ensure_ascii=False)}\n\n" continue # don't save reasoning to full_response content = chunk.content if hasattr(chunk, "content") else "" if content and isinstance(content, str): full_response += content yield f"data: {json.dumps({'type': 'token', 'source': 'fast', 'content': content}, ensure_ascii=False)}\n\n" ``` The `asyncio` and `time` imports are already available in the module. --- ### Change 3 — `_fast_device_sse_generator()`: Same fix Apply the identical heartbeat + reasoning_content pattern (lines 468-472). --- ### Change 4 — Update callers of `_get_chat_model()` Both `_fast_tun_sse_generator` (line 397) and `_fast_device_sse_generator` call `_get_chat_model()` — update to `_get_chat_model(fast_mode=True)` to use the flash model. --- ### Summary | Fix | Impact | Risk | |-----|--------|------| | Flash model for fast mode | TTFT: ~20s → ~2-5s | Output quality may differ (fast mode is template-filling, so low risk) | | Heartbeat during LLM wait | No more dead silence; progress events every 3s | None (pattern already proven in `review_agent.py`) | | Stream reasoning_content | Immediate feedback if model "thinks" | None (new event type `reasoning`; frontend can ignore) | | Add timeout to model init | Prevents hung connections | None (matches `vent_agent.py` pattern) |