“If you look back at Meta’s history, we’ve been a huge proponent of open source,” he says, pointing to the example of PyTorch, a popular tool for developers working with machine learning. “One of the major motivations for building a community around this was that we saw there was demand beyond researchers to work on these models and improve them.” Al-Dahle says work is already underway on the development of Llama 3, but he would not specify how it will be different.
Though Llama 2 lends credibility to Meta as a leader in open source AI, not all aspects of the release can be characterized as open. The training data used to create the model is described in release materials only as “publicly available online sources,” and the company won’t offer further details about what went into the model’s creation.
Meta’s license for Llama 2 also requires companies with more than 700 million monthly active users to establish a separate license agreement with Meta. It is not clear why, but the clause creates a barrier to other tech giants building on the system. The model also comes with an acceptable use policy, which prohibits generating malicious code, promoting violence, or enabling criminal activity, abuse, or harassment. Meta did not respond to a question about what actions it might take if Llama 2 was used in breach of that policy.
Jon Turow, an investor at Madrona Ventures in Seattle, says Meta’s pivot from trying to restrict distribution of the first Llama model to open-sourcing the second could enable a new wave of creativity using large language models. “Developers and entrepreneurs are very resourceful, and they are going to find out what they can squeeze out of Llama 2,” he says.
Turow likens Meta’s choice to release Llama 2 this month to Google introducing the Android mobile operating system in 2007 to rival Apple’s iOS. By giving away a cheap but powerful alternative, Meta can become a counterbalance to proprietary systems like the kind developed by OpenAI, sparking innovation that could feed back ideas that help improve Meta products and services.
Llama 2 is the first openly released model on par with ChatGPT, says Nathan Lambert, an AI researcher at Hugging Face, a startup that releases open source machine-learning software, including generative models. He doesn’t consider the project truly open source, because of Meta’s limited disclosures about its development, but he is astonished by the number of Llama 2 variations he sees in his social media feed. One example is the latest version of WizardLM, an AI system, similar to ChatGPT, designed to follow complex instructions. Eight out of 10 models trending currently on Hugging Face, a number of which are made to generate conversational text, are variations of Llama 2.
“I think there’s a case to be made that Llama 2 is the biggest event of the year in AI,” Lambert says. He says proprietary models have the advantage today, but he believes that later versions of Llama will catch up and, before long, will be able to perform most tasks that people turn to ChatGPT for today.
