Skip to content

AI 辅助流式响应 #10668

Open
Open
@QIN2DIM

Description

@QIN2DIM

In what scenarios do you need this feature?

我自己搭了一个中间件用于中转请求流量,提供符合 Open API 规范的接口。在正确配置 BASE_URL 指向部署端点后发现思源笔记内置的 AI Chat 不支持流式传输,也即,middleware 返回 Streaming SSE,思源笔记直接报错了,只有 middleware “立即返回” JSONResponse 才可用(此时UI界面显示进度条)。

Describe the optimal solution

如何实现流式 SSE 响应, provider LLM 弹出的 tokens 经过 middleware 到思源笔记时,文本可以直接输出在笔记页,而不是有个进度条,等 middleware 将所有数据接受完再打包成 JSONResponse 一次性返回。

Describe the candidate solution

No response

Other information

No response

Activity

TCOTC

TCOTC commented on Mar 20, 2024

@TCOTC
Contributor

希望支持 流式输出 以及 中断输出

JHPZ

JHPZ commented on Apr 1, 2024

@JHPZ

我也希望。

wish5115

wish5115 commented on Apr 26, 2025

@wish5115

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @wish5115@JHPZ@QIN2DIM@TCOTC

        Issue actions

          AI 辅助流式响应 · Issue #10668 · siyuan-note/siyuan