docs: add missing directory entries and package descriptions to architecture docs

Fill gaps identified in architecture audit: add artifacts/ and static/ to directory tree, and document 7 auxiliary internal/ packages (textclean, claudeconv, compat, rawsample, devcapture, util, version) in Section 3. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-05 08:55:28 +08:00 · 2026-05-03 02:51:19 +08:00
parent 5f110e6910
commit ad80a57efa
6 changed files with 107 additions and 53 deletions
--- a/README.MD
+++ b/README.MD
@@ -76,13 +76,14 @@ flowchart LR

        subgraph Runtime["运行时核心能力"]
            Compat["PromptCompat\n(API -> 网页纯文本上下文)"]
-            Chat["Chat / Responses Runtime\n(统一工具调用与流式语义)"]
+            Completion["Completion Runtime\n(Session / PoW / Completion)"]
+            Turn["AssistantTurn\n(输出语义归一)"]
            Auth["Auth Resolver\n(API key / bearer / x-goog-api-key)"]
            Pool["Account Pool + Queue\n(并发槽位 + 等待队列)"]
            DSClient["DeepSeek Client\n(Session / Auth / Completion / Files)"]
            Pow["PoW 实现\n(纯 Go)"]
            Tool["Tool Sieve\n(Go/Node 语义对齐)"]
-            History["History Split\n(长历史文件化)"]
+            History["Current Input File\n(DS2API_HISTORY.txt)"]
        end
    end

@@ -94,18 +95,19 @@ flowchart LR

    OA --> Compat
    CA & GA --> Compat
-    Compat --> Chat
-    Compat -.长历史.-> History
-    Vercel -.Go prepare.-> Chat
+    Compat --> Completion
+    Completion -.完整上下文.-> History
+    Completion --> Turn
+    Vercel -.Go prepare.-> Completion
    Vercel -.Node SSE.-> Tool
-    Chat --> Auth
-    Chat -.账号轮询.-> Pool
-    Chat -.工具调用解析.-> Tool
-    Chat -.PoW 计算.-> Pow
+    Completion --> Auth
+    Completion -.账号轮询.-> Pool
+    Completion -.工具调用解析.-> Tool
+    Completion -.PoW 计算.-> Pow
    Auth --> DSClient
    DSClient --> Upstream
    Upstream --> DSClient
-    Chat --> Client
+    Turn --> Client
    Vercel --> Client
 ```

--- a/README.en.md
+++ b/README.en.md
@@ -73,13 +73,14 @@ flowchart LR

        subgraph Runtime["Runtime + Core Capabilities"]
            Compat["PromptCompat\n(API -> web-chat plain text context)"]
-            Chat["Chat / Responses Runtime\n(unified tools + stream semantics)"]
+            Completion["Completion Runtime\n(session / PoW / completion)"]
+            Turn["AssistantTurn\n(output semantic normalization)"]
            Auth["Auth Resolver\n(API key / bearer / x-goog-api-key)"]
            Pool["Account Pool + Queue\n(in-flight slots + wait queue)"]
            DSClient["DeepSeek Client\n(session / auth / completion / files)"]
            Pow["PoW Solver\n(Pure Go)"]
            Tool["Tool Sieve\n(Go/Node semantic parity)"]
-            History["History Split\n(long history as files)"]
+            History["Current Input File\n(DS2API_HISTORY.txt)"]
        end
    end

@@ -91,18 +92,19 @@ flowchart LR

    OA --> Compat
    CA & GA --> Compat
-    Compat --> Chat
-    Compat -.long history.-> History
-    Vercel -.Go prepare.-> Chat
+    Compat --> Completion
+    Completion -.full context.-> History
+    Completion --> Turn
+    Vercel -.Go prepare.-> Completion
    Vercel -.Node SSE.-> Tool
-    Chat --> Auth
-    Chat -.account rotation.-> Pool
-    Chat -.tool-call parsing.-> Tool
-    Chat -.PoW solving.-> Pow
+    Completion --> Auth
+    Completion -.account rotation.-> Pool
+    Completion -.tool-call parsing.-> Tool
+    Completion -.PoW solving.-> Pow
    Auth --> DSClient
    DSClient --> Upstream
    Upstream --> DSClient
-    Chat --> Client
+    Turn --> Client
    Vercel --> Client
 ```

--- a/docs/ARCHITECTURE.en.md
+++ b/docs/ARCHITECTURE.en.md
@@ -15,6 +15,7 @@ ds2api/
 │   └── workflows/                        # GitHub Actions workflows
 ├── api/                                  # Serverless entrypoints (Vercel Go/Node)
 ├── app/                                  # Application-level handler assembly
+├── artifacts/                            # Debug artifacts (raw-stream-sim, stream-debug, etc.)
 ├── cmd/                                  # Executable entrypoints
 │   ├── ds2api/                           # Main service bootstrap
 │   └── ds2api-tests/                     # E2E testsuite CLI bootstrap
@@ -40,13 +41,14 @@ ds2api/
 │   │   ├── admin/                        # Admin API root assembly and resource packages
 │   │   ├── claude/                       # Claude HTTP protocol adapter
 │   │   ├── gemini/                       # Gemini HTTP protocol adapter
-│   │   └── openai/                       # OpenAI HTTP surface
-│   │       ├── chat/                     # Chat Completions execution entrypoint
-│   │       ├── responses/                # Responses API and response store
-│   │       ├── files/                    # Files API and inline-file preprocessing
-│   │       ├── embeddings/               # Embeddings API
-│   │       ├── history/                  # OpenAI context file handling
-│   │       └── shared/                   # OpenAI HTTP errors/models/tool formatting
+│   │   ├── openai/                       # OpenAI HTTP surface
+│   │   │   ├── chat/                     # Chat Completions execution entrypoint
+│   │   │   ├── responses/                # Responses API and response store
+│   │   │   ├── files/                    # Files API and inline-file preprocessing
+│   │   │   ├── embeddings/               # Embeddings API
+│   │   │   ├── history/                  # OpenAI context file handling
+│   │   │   └── shared/                   # OpenAI HTTP errors/models/tool formatting
+│   │   └── requestbody/                  # HTTP body reading and UTF-8/JSON validation helpers
 │   ├── js/                               # Node runtime related logic
 │   │   ├── chat-stream/                  # Node streaming bridge
 │   │   ├── helpers/                      # JS helper modules
@@ -70,6 +72,7 @@ ds2api/
 ├── plans/                                # Stage plans and manual QA records
 ├── pow/                                  # PoW standalone implementation + benchmarks
 ├── scripts/                              # Build/release helper scripts
+├── static/                               # Build artifacts (admin static resources)
 ├── tests/                                # Test assets and scripts
 │   ├── compat/                           # Compatibility fixtures + expected outputs
 │   │   ├── expected/                     # Expected output samples
@@ -78,9 +81,9 @@ ds2api/
 │   │       └── toolcalls/                # Tool-call fixtures
 │   ├── node/                             # Node unit tests
 │   ├── raw_stream_samples/               # Upstream raw SSE samples
-│   │   ├── content-filter-trigger-20260405-jwt3/          # Content-filter terminal sample
 │   │   ├── continue-thinking-snapshot-replay-20260405/    # Continue-thinking sample
-│   │   ├── guangzhou-weather-reasoner-search-20260404/    # Search/reference sample
+│   │   ├── longtext-deepseek-v4-flash-20260429/           # Flash long-text/file-upload sample
+│   │   ├── longtext-deepseek-v4-pro-20260429/             # Pro long-text/file-upload sample
 │   │   ├── markdown-format-example-20260405/              # Markdown sample
 │   │   └── markdown-format-example-20260405-spacefix/     # Space-fix sample
 │   ├── scripts/                          # Test entry scripts
@@ -93,6 +96,8 @@ ds2api/
        ├── features/                     # Feature modules
        │   ├── account/                  # Account management page
        │   ├── apiTester/                # API tester page
+        │   ├── chatHistory/              # Server-side conversation history page
+        │   ├── proxy/                    # Proxy management page
        │   ├── settings/                 # Settings page
        │   └── vercel/                   # Vercel sync page
        ├── layout/                       # Layout components
@@ -126,8 +131,11 @@ flowchart LR
    subgraph RUNTIME[Shared runtime]
        AUTH[internal/auth]
        POOL[internal/account queue + concurrency]
+        CR[internal/completionruntime]
+        TURN[internal/assistantturn]
        STREAM[internal/stream + internal/sse]
        TOOL[internal/toolcall + internal/toolstream]
+        FMT[internal/format/openai + claude]
        DS[internal/deepseek/client]
        POW[pow + internal/deepseek/protocol]
    end
@@ -153,16 +161,24 @@ flowchart LR
    PC --> PROMPT
    PC -.long history.-> HIST
    PC --> AUTH
+    PC --> CR

    NCS -.Go prepare/release.-> CHAT
    NCS --> JS
    JS --> TOOL

    AUTH --> POOL
-    CHAT --> STREAM
-    RESP --> STREAM
+    CHAT --> CR
+    RESP --> CR
+    CA --> CR
+    GA --> CR
+    CR --> DS
+    CR --> STREAM
+    CR --> TURN
+    STREAM --> TURN
    STREAM --> TOOL
-    POOL --> DS
+    TURN --> FMT
+    POOL --> CR
    DS --> POW
    DS --> U[DeepSeek upstream]
 ```
@@ -171,7 +187,8 @@ flowchart LR

 - `internal/server`: router tree + middlewares (health, protocol routes, Admin/WebUI).
 - `internal/httpapi/openai/*`: OpenAI HTTP surface split into chat, responses, files, embeddings, history, and shared packages; chat/responses share the promptcompat, stream, and toolcall semantics.
- `internal/httpapi/{claude,gemini}`: protocol wrappers that normalize into the same prompt compatibility semantics without duplicating upstream execution.
+- `internal/httpapi/{claude,gemini}`: protocol adapters that normalize into the same prompt compatibility semantics; direct paths share DeepSeek session/PoW/completion execution through `completionruntime`, while Vercel/proxy paths can still translate through `translatorcliproxy` into the OpenAI handler.
+- `internal/httpapi/requestbody`: shared HTTP body reading, JSON pre-validation, and UTF-8 error helpers across protocol adapters.
 - `internal/promptcompat`: compatibility core for turning OpenAI/Claude/Gemini requests into DeepSeek web-chat plain-text context.
 - `internal/assistantturn`: Go output-side canonical semantics, converting DeepSeek SSE collection results and stream finalization state into assistant turns and centralizing thinking, tool call, citation, usage, stop/error behavior.
 - `internal/completionruntime`: shared Go completion execution helpers for DeepSeek session/PoW/call startup, non-stream collection, and empty-output retry; streaming paths use it to start upstream requests, continue to use `internal/stream` for real-time consumption, and use `assistantturn` during finalization.
@@ -184,6 +201,13 @@ flowchart LR
 - `internal/chathistory`: server-side conversation history persistence, pagination, detail lookup, and retention policy.
 - `internal/config`: config loading/validation + runtime settings hot-reload.
 - `internal/account`: managed account pool, inflight slots, waiting queue.
+- `internal/textclean`: text cleanup helpers, e.g. stripping `[reference: N]` markers.
+- `internal/claudeconv`: Claude API request to DeepSeek format conversion.
+- `internal/compat`: compatibility regression tests using SSE fixtures to verify output consistency.
+- `internal/rawsample`: upstream raw response capture, read/write, and management.
+- `internal/devcapture`: developer debug capture, storing HTTP request/response for troubleshooting.
+- `internal/util`: cross-package utilities including JSON writing, type conversion, token counting, thinking parsing, etc.
+- `internal/version`: version query and comparison, supporting build-time injection and runtime resolution.

 ## 4. WebUI Runtime Relation

--- a/docs/ARCHITECTURE.md
+++ b/docs/ARCHITECTURE.md
@@ -15,6 +15,7 @@ ds2api/
 │   └── workflows/                        # GitHub Actions 工作流
 ├── api/                                  # Serverless 入口（Vercel Go/Node）
 ├── app/                                  # 应用级 handler 装配层
+├── artifacts/                            # 调试产物（raw-stream-sim, stream-debug 等）
 ├── cmd/                                  # 可执行程序入口
 │   ├── ds2api/                           # 主服务启动入口
 │   └── ds2api-tests/                     # E2E 测试集 CLI 入口
@@ -40,13 +41,14 @@ ds2api/
 │   │   ├── admin/                        # Admin API 根装配与资源子包
 │   │   ├── claude/                       # Claude HTTP 协议适配
 │   │   ├── gemini/                       # Gemini HTTP 协议适配
-│   │   └── openai/                       # OpenAI HTTP surface
-│   │       ├── chat/                     # Chat Completions 执行入口
-│   │       ├── responses/                # Responses API 与 response store
-│   │       ├── files/                    # Files API 与 inline file 预处理
-│   │       ├── embeddings/               # Embeddings API
-│   │       ├── history/                  # OpenAI context file handling
-│   │       └── shared/                   # OpenAI HTTP 公共错误/模型/工具格式
+│   │   ├── openai/                       # OpenAI HTTP surface
+│   │   │   ├── chat/                     # Chat Completions 执行入口
+│   │   │   ├── responses/                # Responses API 与 response store
+│   │   │   ├── files/                    # Files API 与 inline file 预处理
+│   │   │   ├── embeddings/               # Embeddings API
+│   │   │   ├── history/                  # OpenAI context file handling
+│   │   │   └── shared/                   # OpenAI HTTP 公共错误/模型/工具格式
+│   │   └── requestbody/                  # HTTP 请求体读取与 UTF-8/JSON 校验辅助
 │   ├── js/                               # Node Runtime 相关逻辑
 │   │   ├── chat-stream/                  # Node 流式输出桥接
 │   │   ├── helpers/                      # JS 辅助函数
@@ -70,6 +72,7 @@ ds2api/
 ├── plans/                                # 阶段计划与人工验收记录
 ├── pow/                                  # PoW 独立实现与基准
 ├── scripts/                              # 构建/发布/辅助脚本
+├── static/                               # 构建产物（admin 等静态资源）
 ├── tests/                                # 测试资源与脚本
 │   ├── compat/                           # 兼容性夹具与期望输出
 │   │   ├── expected/                     # 预期结果样本
@@ -78,9 +81,9 @@ ds2api/
 │   │       └── toolcalls/                # toolcall 夹具
 │   ├── node/                             # Node 单元测试
 │   ├── raw_stream_samples/               # 上游原始 SSE 样本
-│   │   ├── content-filter-trigger-20260405-jwt3/          # 风控终态样本
 │   │   ├── continue-thinking-snapshot-replay-20260405/    # continue 样本
-│   │   ├── guangzhou-weather-reasoner-search-20260404/    # 搜索+引用样本
+│   │   ├── longtext-deepseek-v4-flash-20260429/           # flash 长文本/文件上传样本
+│   │   ├── longtext-deepseek-v4-pro-20260429/             # pro 长文本/文件上传样本
 │   │   ├── markdown-format-example-20260405/              # Markdown 样本
 │   │   └── markdown-format-example-20260405-spacefix/     # 空格修复样本
 │   ├── scripts/                          # 测试脚本入口
@@ -93,6 +96,8 @@ ds2api/
        ├── features/                     # 功能模块
        │   ├── account/                  # 账号管理页面
        │   ├── apiTester/                # API 测试页面
+        │   ├── chatHistory/              # 服务器端对话记录页面
+        │   ├── proxy/                    # 代理管理页面
        │   ├── settings/                 # 设置页面
        │   └── vercel/                   # Vercel 同步页面
        ├── layout/                       # 布局组件
@@ -126,8 +131,11 @@ flowchart LR
    subgraph RUNTIME[Shared runtime]
        AUTH[internal/auth]
        POOL[internal/account queue + concurrency]
+        CR[internal/completionruntime]
+        TURN[internal/assistantturn]
        STREAM[internal/stream + internal/sse]
        TOOL[internal/toolcall + internal/toolstream]
+        FMT[internal/format/openai + claude]
        DS[internal/deepseek/client]
        POW[pow + internal/deepseek/protocol]
    end
@@ -153,16 +161,24 @@ flowchart LR
    PC --> PROMPT
    PC -.长历史.-> HIST
    PC --> AUTH
+    PC --> CR

    NCS -.Go prepare/release.-> CHAT
    NCS --> JS
    JS --> TOOL

    AUTH --> POOL
-    CHAT --> STREAM
-    RESP --> STREAM
+    CHAT --> CR
+    RESP --> CR
+    CA --> CR
+    GA --> CR
+    CR --> DS
+    CR --> STREAM
+    CR --> TURN
+    STREAM --> TURN
    STREAM --> TOOL
-    POOL --> DS
+    TURN --> FMT
+    POOL --> CR
    DS --> POW
    DS --> U[DeepSeek upstream]
 ```
@@ -171,7 +187,8 @@ flowchart LR

 - `internal/server`：路由树和中间件挂载（健康检查、协议入口、Admin/WebUI）。
 - `internal/httpapi/openai/*`：OpenAI HTTP surface，按 chat、responses、files、embeddings、history、shared 拆分；chat/responses 共享 promptcompat、stream、toolcall 等核心语义。
- `internal/httpapi/{claude,gemini}`：协议输入输出适配，归一到同一套 prompt compatibility 语义，不重复实现上游调用逻辑。
+- `internal/httpapi/{claude,gemini}`：协议输入输出适配，归一到同一套 prompt compatibility 语义；直连路径通过 `completionruntime` 共享 DeepSeek session/PoW/completion 调用，Vercel/代理路径仍可经 `translatorcliproxy` 转到 OpenAI handler。
+- `internal/httpapi/requestbody`：跨协议复用的请求体读取、JSON 解码前置校验与 UTF-8 错误处理辅助。
 - `internal/promptcompat`：OpenAI/Claude/Gemini 请求到 DeepSeek 网页纯文本上下文的兼容内核。
 - `internal/assistantturn`：Go 输出侧统一语义层，把 DeepSeek SSE 收集结果和流式收尾状态归一成 assistant turn，集中处理 thinking、tool call、citation、usage、stop/error 语义。
 - `internal/completionruntime`：Go surface 共享的 completion 执行辅助，负责 DeepSeek session/PoW/call 启动、非流式 collect 和 empty-output retry；流式路径复用它启动上游请求，继续用 `internal/stream` 做实时消费，并在最终收尾阶段接入 `assistantturn`。
@@ -184,6 +201,13 @@ flowchart LR
 - `internal/chathistory`：服务器端对话记录持久化、分页、单条详情和保留策略。
 - `internal/config`：配置加载、校验、运行时 settings 热更新。
 - `internal/account`：托管账号池、并发槽位、等待队列。
+- `internal/textclean`：文本清洗，移除 `[reference: N]` 标记等噪声。
+- `internal/claudeconv`：Claude API 请求到 DeepSeek 格式的协议转换。
+- `internal/compat`：兼容性回归测试套件，用 SSE 夹具验证输出一致性。
+- `internal/rawsample`：上游原始响应的采集、读写与管理。
+- `internal/devcapture`：开发调试抓包，存储 HTTP 请求/响应用于问题排查。
+- `internal/util`：跨包通用工具，含 JSON 写入、类型转换、token 计数、thinking 解析等。
+- `internal/version`：版本号查询与比较，支持构建注入和运行时解析。

 ## 4. WebUI 与运行时关系

--- a/docs/DeepSeekSSE行为结构说明-2026-04-05.md
+++ b/docs/DeepSeekSSE行为结构说明-2026-04-05.md
@@ -1,7 +1,7 @@
 # DeepSeek SSE 行为结构说明（第三方逆向版）

 > 说明：本文基于当前仓库 `tests/raw_stream_samples/` 下全部 `upstream.stream.sse` 原始流样本整理而成，属于第三方逆向观察文档，不是官方协议。
-> 当前 corpus 由 4 份原始流组成，覆盖搜索+引用、风控终态、Markdown 输出和空格敏感输出等行为。
+> 当前 corpus 由 5 份原始流组成，覆盖长文本生成、文件上传上下文、continue 接续、Markdown 输出和空格敏感输出等行为。
 > 补充：文末还会注明少量“当前实现已确认、但 corpus 尚未完整覆盖”的行为，例如长思考场景下的自动续写状态。

 文档导航：[文档总索引](./README.md) / [测试指南](./TESTING.md) / [样本目录说明](../tests/raw_stream_samples/README.md)
@@ -12,8 +12,9 @@

 | 样本 | 观察重点 |
 | --- | --- |
-| [guangzhou-weather-reasoner-search-20260404](../tests/raw_stream_samples/guangzhou-weather-reasoner-search-20260404/upstream.stream.sse) | 搜索+思考流程，包含 `reference:N` 引用标记与工具片段 |
-| [content-filter-trigger-20260405-jwt3](../tests/raw_stream_samples/content-filter-trigger-20260405-jwt3/upstream.stream.sse) | `CONTENT_FILTER` 终态分支，包含拒答模板与 `ban_regenerate` |
+| [longtext-deepseek-v4-flash-20260429](../tests/raw_stream_samples/longtext-deepseek-v4-flash-20260429/upstream.stream.sse) | DeepSeek V4 flash 长文本流，包含 current input file 上传后的 completion 样本 |
+| [longtext-deepseek-v4-pro-20260429](../tests/raw_stream_samples/longtext-deepseek-v4-pro-20260429/upstream.stream.sse) | DeepSeek V4 pro 长文本流，包含文件上传上下文和较长 reasoning/content 输出 |
+| [continue-thinking-snapshot-replay-20260405](../tests/raw_stream_samples/continue-thinking-snapshot-replay-20260405/upstream.stream.sse) | 多轮 `completion + continue` 原始流，用于验证接续思考去重 |
 | [markdown-format-example-20260405](../tests/raw_stream_samples/markdown-format-example-20260405/upstream.stream.sse) | Markdown 输出的早期样本，用于观察 token 级输出形态 |
 | [markdown-format-example-20260405-spacefix](../tests/raw_stream_samples/markdown-format-example-20260405-spacefix/upstream.stream.sse) | Markdown 输出修正样本，用于验证空格 chunk 必须保留 |

@@ -194,7 +195,7 @@ close

 ## 8. 终态行为

-当前 corpus 里有两条很重要的终态分支。
+当前 corpus 直接覆盖正常完成和 continue 接续；当前实现还兼容 `CONTENT_FILTER` 风控终态，相关分支由协议观察与兼容性 fixture 继续守护。

 ### 8.1 正常完成

@@ -208,7 +209,7 @@ close

 ### 8.2 风控终态

-`content-filter-trigger-20260405-jwt3` 展示了另一种终态路径：
+`CONTENT_FILTER` 不在当前 raw stream corpus 的目录样本中，但代码和兼容性测试仍按下面这种终态路径处理：

 1. 先继续输出一段正常正文。
 2. 出现提示类 fragment，例如 `TIP`。
--- a/docs/prompt-compatibility.md
+++ b/docs/prompt-compatibility.md
@@ -45,7 +45,8 @@ DS2API 当前的核心思路，不是把客户端传来的 `messages`、`tools`
  -> promptcompat 统一消息标准化
  -> tool prompt 注入
  -> DeepSeek 风格 prompt 拼装
-  -> 文件收集 / inline 上传 / current input file（OpenAI 链路）
+  -> 文件收集 / inline 上传（OpenAI 文件链路）
+  -> current input file（completion runtime 全局入口）
  -> completion payload
  -> 下游网页对话接口
  -> assistantturn 输出语义归一（Go 非流式 + 流式收尾）
@@ -282,7 +283,7 @@ OpenAI 的文件上传现在不再是“只传文件本体”的通用路径，
 - 全局 completion runtime 应用点：
  [internal/completionruntime/nonstream.go](../internal/completionruntime/nonstream.go)

-当前输入转文件启用并触发时，上传文件的真实文件名是 `DS2API_HISTORY.txt`，文件内容是完整 `messages` 上下文；它仍会先用 OpenAI 消息标准化和 DeepSeek 角色标记序列化，再按轮次编号成 `DS2API_HISTORY.txt` 风格的 transcript（不再注入文件边界标签）：
+当前输入转文件启用并触发时，上传文件的真实文件名是 `DS2API_HISTORY.txt`，文件内容是完整 `messages` 上下文；它会使用 OpenAI-compatible 的消息/transcript 序列化规则和 DeepSeek 角色标记，再按轮次编号成 `DS2API_HISTORY.txt` 风格的 transcript（不再注入文件边界标签）：

 ```text
 [uploaded filename]: DS2API_HISTORY.txt