Skip to content

Add kernel API /api/archive/zip and /api/archive/unzip #9028

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
HowcanoeWang opened this issue Aug 22, 2023 · 23 comments
Closed

Add kernel API /api/archive/zip and /api/archive/unzip #9028

HowcanoeWang opened this issue Aug 22, 2023 · 23 comments
Assignees
Milestone

Comments

@HowcanoeWang
Copy link

HowcanoeWang commented Aug 22, 2023

插件系统有可能把node的第三方库也打包进去吗

开发插件需要解压缩zip文件的功能,node:zlib没法处理zip文件,其他的第三方库如unzipper、adm-zip之类的,全都是node库,webpack尝试过无法打包成功

@88250
Copy link
Member

88250 commented Aug 22, 2023

通过内核 API 提供如何?

@HowcanoeWang
Copy link
Author

刚刚在折腾群群主的帮助下,用一个纯前端的库实现了,自己手撸了文件遍历解压,还是挺麻烦的。

有的内核API能提供这个功能的话,就更好不过了

@88250 88250 transferred this issue from siyuan-note/bazaar Aug 23, 2023
@88250 88250 changed the title 插件系统有可能把node的第三方库也打包进去吗 Add kernel API /api/archive/zip and /api/archive/unzip Aug 23, 2023
@88250
Copy link
Member

88250 commented Aug 23, 2023

既然你已经实现了,内核的实现就先缓缓……

@HowcanoeWang
Copy link
Author

HowcanoeWang commented Aug 23, 2023

既然你已经实现了,内核的实现就先缓缓……

同样的解压代码在MacOS上跑,会报OOM(out of memory?)错误,还是期待一下官方的API实现,先在windows端把主要功能实现了再看看全平台的问题

@88250
Copy link
Member

88250 commented Aug 25, 2023

麻烦给出参数和返回值,我参考一下,谢谢。

@HowcanoeWang
Copy link
Author

自己随手撸的渣代码,感觉没啥参考价值

引入部分:https://github.com/HowcanoeWang/sython/blob/7151b311cc255c5f5ddeacd88ddcb7661b92b9f5/src/constants.ts#L3-L16

功能实现部分:https://github.com/HowcanoeWang/sython/blob/7151b311cc255c5f5ddeacd88ddcb7661b92b9f5/src/fileTool.ts#L70C2-L161

真就不如直接打包一个unzipper到思源内核里,比较省心

以及提供一个可以用来测试解压性能的zip包:https://gitee.com/HowcanoeWang/python3-embeddable/releases/download/v1.0.0/python3-windows-3.10.4-amd64.zip

@88250 88250 self-assigned this Aug 26, 2023
@88250 88250 added this to the 2.10.2 milestone Aug 26, 2023
@88250
Copy link
Member

88250 commented Aug 26, 2023

image

image

@HowcanoeWang
Copy link
Author

image

image

优雅,太优雅了🐮

@88250
Copy link
Member

88250 commented Aug 27, 2023

稍后发布一个 dev 版,请帮忙测试看看。

@88250 88250 closed this as completed Aug 27, 2023
@HowcanoeWang
Copy link
Author

HowcanoeWang commented Aug 27, 2023

尝试用这个方法来调用,遇到了400 (Bad Request)错误,不确定是我写的问题还是api的问题

export async function unzipFile(zipFilePath: string, extractToPath: string) {

    let data = new FormData();
    data.append("path", extractToPath);
    data.append("zipPath", zipFilePath);

    console.log(zipFilePath, extractToPath);

    let res = await fetch("/api/archive/unzip", {
        method: "POST",
        body: data,
    });
    return await res.json();
};

其中,console.log 出来的zipFilePathextractToPath值分别如下:

C:\Users\hwang\Documents\SiYuanDev\data\storage\envs\python.zip  
C:\Users\hwang\Documents\SiYuanDev\data\storage\envs\base\

报错的log:

plugin:sython:34     POST http://127.0.0.1:54991/api/archive/unzip 400 (Bad Request)
eval @ plugin:sython:34
eval @ plugin:sython:34
c @ plugin:sython:34
s @ plugin:sython:34
eval @ plugin:sython:39
eval @ plugin:sython:35
v @ plugin:sython:35
onLayoutReady @ plugin:sython:39
Nd @ main.7fe2383762a74004568f.js:3552
(匿名) @ main.7fe2383762a74004568f.js:3552
mv @ main.7fe2383762a74004568f.js:3552
(匿名) @ main.7fe2383762a74004568f.js:5572
(匿名) @ main.7fe2383762a74004568f.js:3172
Promise.then(异步)
b @ main.7fe2383762a74004568f.js:3172
Rv @ main.7fe2383762a74004568f.js:5572
(匿名) @ main.7fe2383762a74004568f.js:5598
(匿名) @ main.7fe2383762a74004568f.js:3172
Promise.then(异步)
b @ main.7fe2383762a74004568f.js:3172
(匿名) @ main.7fe2383762a74004568f.js:5598
(匿名) @ main.7fe2383762a74004568f.js:3172
Promise.then(异步)
Dh @ main.7fe2383762a74004568f.js:3172
(匿名) @ main.7fe2383762a74004568f.js:5598
(匿名) @ main.7fe2383762a74004568f.js:9
(匿名) @ main.7fe2383762a74004568f.js:3172
Promise.then(异步)
b @ main.7fe2383762a74004568f.js:3172
Xt @ main.7fe2383762a74004568f.js:9
(匿名) @ main.7fe2383762a74004568f.js:5598
await in (匿名)(异步)
(匿名) @ main.7fe2383762a74004568f.js:3172
Promise.then(异步)
b @ main.7fe2383762a74004568f.js:3172
Fv @ main.7fe2383762a74004568f.js:5598
(匿名) @ main.7fe2383762a74004568f.js:5598
(匿名) @ main.7fe2383762a74004568f.js:5598
(匿名) @ main.7fe2383762a74004568f.js:5598
processTicksAndRejections @ node:internal/process/task_queues:82

@88250
Copy link
Member

88250 commented Aug 27, 2023 via email

@HowcanoeWang
Copy link
Author

改成这样的代码就成功了

        body: JSON.stringify({
            "path": extractToPath,
            "zipPath": zipFilePath
        })

下午去实验室测试一下macos和linux上面的效果

@HowcanoeWang
Copy link
Author

HowcanoeWang commented Aug 27, 2023

额,macos上更新了dev之后,就开始报错获取内核服务端口失败,给了防火墙权限还是没有用。重启也不行,回退到旧版也不行了

image

#7001


重装+重启,然后拼手速点弹出来的允许网络访问,正常了=_=

@88250
Copy link
Member

88250 commented Aug 27, 2023 via email

@HowcanoeWang
Copy link
Author

Win / macOS / Linux 均测试通过,解压速度很快,也能正常使用

@88250
Copy link
Member

88250 commented Aug 27, 2023 via email

@Zuoqiu-Yingyi
Copy link
Contributor

image

image

@88250 这里的路径使用绝对路径有绕过访问控制的安全风险, 建议使用相对于工作空间目录的相对路径作为路径

@88250
Copy link
Member

88250 commented Aug 28, 2023

@Zuoqiu-Yingyi 考虑到有的场景可能需要全局路径所以就这样吧。

@Zuoqiu-Yingyi
Copy link
Contributor

@Zuoqiu-Yingyi 考虑到有的场景可能需要全局路径所以就这样吧。

有那哪些场景需要全局路径❓

@HowcanoeWang
Copy link
Author

@Zuoqiu-Yingyi 考虑到有的场景可能需要全局路径所以就这样吧。

有那哪些场景需要全局路径❓

比如导出插件,导出整个笔记为zip然后指定桌面为保存目录?

@Zuoqiu-Yingyi
Copy link
Contributor

比如导出插件,导出整个笔记为zip然后指定桌面为保存目录?

桌面端有 Electron 的 globalThis.require("@electron/remote").dialog.showSaveDialog
web 端没啥意义

@Zuoqiu-Yingyi
Copy link
Contributor

@88250 该 API 可以通过覆盖原文件的方式篡改 可执行程序/shell脚本 实现远程代码执行, 最好还是限制一下其访问目录的范围

@88250
Copy link
Member

88250 commented Nov 22, 2023

@Zuoqiu-Yingyi 方便的话你 PR 吧,谢谢。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants