开源工具 airllm 让 2.8 万亿参数模型在 4GB 显存显卡上运行
This free tool runs huge AI models on a cheap graphics card Link in bio for the full guide 📩 #ai #localai #opensource #techtok
一个开源项目提供在廉价显卡上运行超大模型的方法，2.8 万亿参数的 kimi k3 可在不到 4GB 显存上运行。其做法是一次加载一层模型执行后再加载下一层，同样的技巧也能让通常需要约 140GB 的 70B 参数模型跑在 4GB 显卡上，但每层都要从磁盘读取，速度较慢且需要大量空闲空间。
原文给出开源工具名和一条命令装好的入口，适合录屏实测在 4GB 显存卡上跑大模型的分层加载过程与速度表现。
粉丝 4.8万
- 浏览
- 3.3万
- 点赞
- 1617
- 评论
- 60
- 分享
- 209
- 收藏
- 1067
数字来自平台，为最近一次抓取时的数据。
以下是视频口播的自动转写，可能有错字。
Someone open sourced a way to run the biggest open model on your cheap graphics card. So kimi k3 a model with 2.8 trillion parameters runs on less than 4 gigabytes of graphics memory.
Basically, it loads one layer of the model at a time runs it, then loads the next and get this. The same trick runs a 70 billion parameter model, which normally needs about 140 gigabytes on a 4 gigabyte card, but it's slow because every layer comes off your disk.
So you need lots of free space. So you paste one prompt into cloud and it installs everything and runs your first model for you.
It's called are llm and it's already passed 35,000 stars for the link. Comment giant.
来源：TikTok · 搜「ai tools」 · tiktok.com