Programming is one area in which recent advances in AI have already had a considerable impact.
In May 2021, GitHub, a subsidiary of Microsoft, launched Copilot, a plug-in for coding programs that auto-completes sections of code based on the first line or a comment typed by the user. Copilot uses a version of Open AI’s GPT, the large language model behind ChatGPT. That model is trained further using code that GitHub stores for developers, as well as, reportedly, by contractors who are paid to annotate their own code.
GitHub faces a lawsuit for using some open source code in its training data, and Masad says Meta is likely to have limited the training data to avoid such complications. Copilot costs $10 per month for individuals and $19 per month, per user, for businesses.
Copilot has apparently been a hit with developers. According to figures released by GitHub in June, it is used by more than over a million developers and more than 200,000 businesses. The company’s own studies also suggest that Copilot accelerates the rate at which coders can perform tasks, leading to a 30 percent increase in productivity.
Meta is releasing two versions of Code Llama, one geared toward producing Python code and another optimized for turning natural language commands into code. It is also making three sizes of models available. The smallest can run on a single GPU.
Meta says that Code Llama is trained on code that is in the public domain. In two common coding benchmarks, HumanEval and Mostly Basic Python Problems, it performs much better than existing open source coding models and is “on par with ChatGPT,” the company says.
Amjad Masad, CEO of Replit, an online coding platform that offers several generative AI tools, does not expect Code Llama to supplant Copilot because it’s more limited training data is likely to make it more limited. But he says the release could allow developers to experiment with agents that perform useful tasks, like browsing the web for information or using an API to book a flight or order a meal. “I think that’s a really exciting area,” Masad says. “Interactions where you can type natural language instructions, and the model can crunch data can do interesting things in the world.”
The release of Code Llama may also provide benefits to Meta. The company might not have ChatGPT or an AI-powered search engine, but establishing itself as the provider of free AI to many developers, companies, and researchers could give it a foothold in the race to harness generative AI. Meta chose to adopt an open approach after seeing someone leak an early version of Llama to the web in May.
Neither Llama 2 nor Code Llama are not released under regular open source software licenses that would allow unfettered commercial usage. Under Meta’s license, for instance, users are restricted from using the models in apps or services with more than 700 million monthly users.
A research paper posted online this month notes that releasing AI tools can have significant indirect benefits to the companies behind them by locking researchers into their tools, for instance, and providing them with new ideas that they can use at scale.
