大力出奇迹,几本书是不够,要很多很多。
参考,推理能力超过 gpt-3.5 的 Llama3
- https://ai.meta.com/blog/meta-llama-3/
- https://ai.meta.com/blog/meta-llama-3-1/
> Meta reports on Llama 3.1's page on Huggingface, using 39.3 million hours of H100 80GB instances to train all 3.1 models (8, 70, 400 B).
大概是,两万四千张 H100 训练 74 天
> Llama 3 is pretrained on over 15T tokens that were all collected from publicly available sources.
大概等同于 60TB 数据,在它的技术报告里,提到这是更大的数据集上做清洗去重的精华
---
所以先有这么多的资源才能训练得到高质量机器人 |