In April 2024, Meta released Llama 3, the latest version of its AI-based large language models based on a dataset at least 7 times larger than Llama 2.
Initially available in 8B and 70B parameter sizes, the Llama 3 already outperforms the Llama 2, Google’s open source Gemma and Anthrophic’s Claude Sonnet at launch. Sonnet has since had an upgrade, making it one of the most powerful AI models.
But now a leak suggests that the long-awaited release of the most powerful Llama 3 models, which are trained on more than 400 billion parameters, may also be around the corner. This is just one of many new models that Meta is working on using its hundreds of thousands of Nvidia H100 GPUs.
Efficient yet powerful
📝 WhatsApp beta for Android 2.24.14.7: what’s new? WhatsApp is working on a Meta AI Llama model selection feature and it will be available in a future update!https://t.co/fInfKYk8Oo pic.twitter.com /eVqWfJ1wGAJune 26, 2024
In early tests, the instruction-tuned Llama 3 400B scored 86.1 on the MMLU benchmark, which already puts it on par with the GPT-4’s performance with less than half the parameters.
There’s a lot of technical information to unpack here, so let’s talk about why this really matters.
Simply put, large language models with more parameters always tend to perform better in benchmarks and real-world tasks. But the fact that Llama 3 400B can almost match GPT-4’s MMLU score with under 50% of the parameters suggests that Meta has made enough progress in model architecture and training to give OpenAI a serious run for its money .
By achieving the same performance with fewer parameters, Llama 3 400B is likely to be much more efficient than OpenAI’s ChatGPT 4 in terms of computational resources, power consumption, and cost.
Advantage of open source
Another important reason why people are so excited about Llama 3 is that it is released under an open license for research and commercial use. Although it is not yet clear whether the 400B will be released under the same open license.
If released as an open model, then these state-of-the-art language capabilities will now be available to researchers and developers for free through multiple cloud platforms and ecosystems, accelerating innovation and enabling newer applications of the technology.
With the new 400B model packing enough power to rival ChatGPT 4, this puts a lot of power in the hands of the researcher. This would enable faster development of advanced language AI applications without relying on expensive proprietary APIs.
What we know so far
Meta AI hinted at the release of the 400B model from its initial Llama 3 press release on April 18th. “Our largest models are over 400B,” it wrote at the time, adding that “in the coming months, you’ll be releasing multiple models with new capabilities, including multimodality, multi-language capability, a much longer context window, and more strong overall capabilities.
Since then, the internet has been abuzz with theories and ideas about a possible release date for the 400B models. While the folks at Meta have confirmed that development on the Llama 3 400B is now complete, no official release date has been announced yet.
However, WhatsApp Beta users on Android 2.24.14.7 noticed a new option to try the Llama 3-405B model for Meta AI. While this option is currently only rolled out to beta users and with significant usage volume restrictions, it’s enough to get people excited for a full release, possibly in late July or August of 2024.
More from Tom’s Guide
Back to the MacBook Air
