Colibri 在无 GPU 笔记本上运行 744B 参数模型
Colibri just ran a 744 billion parameter AI model on a laptop with no GPU by streaming active parts from RAM and disk. This smart AI setup beats paid models offline with just one setup file. #AI #AITools #MachineLearning #OpenSourceAI #TechInnovation
开发者用 Colibri 在无 GPU 的笔记本上运行了 744B 参数模型，靠把约 10GB 活跃部分留在内存、其余从硬盘按需流式读取实现。运行方式是克隆仓库后执行一个 setup 文件，模型下载占 370GB 磁盘，之后输入 collie 即可完全离线对话，输入 collie serve 可转为 OpenAI 接口。
原文给出克隆仓库、运行一个 setup 文件、输入 collie 即可离线对话的完整步骤，适合录屏演示笔记本无 GPU 跑 744B 模型。
粉丝 6034
- 浏览
- 6.2万
- 点赞
- 1782
- 评论
- 32
- 分享
- 358
- 收藏
- 2076
数字来自平台，为最近一次抓取时的数据。
以下是视频口播的自动转写，可能有错字。
Someone just ran a 744 billion parameter ai model on a laptop with no graphics card. A developer in italy pulled it off with gllm 5.2, the open model that beats the ones that you pay for.
Nvidia's entire pitch is that you need their gpus for something like this, and he just proved you don't. The project is called calibri.
It works because only about 10 gigs of the model is actually thinking at any moment. So calibri keeps that part in your ram and streams the rest off your hard drive as it needs it.
It's the same way netflix streams movies instead of making you download it. And running it is easier than it sounds. You clone the repo and run one setup file.
The whole thing builds itself from there. The only heavy part is the model download, which takes 370 gigs on your drive.
After that, you just type collie in chat, and you're talking to it completely offline. If you build apps, type collie serve instead, and it turns into an OpenAI endpoint.
You change one line in your code, and your token bill goes to zero, because nothing ever leaves your machine. The github link with a full install guide is in the comments.
Follow for useful ai updates explained clearly.
来源：TikTok · 搜「ai tools」 · tiktok.com