StarCoder 2: A new open-source code generator with competitive performance

Code generators powered by artificial intelligence are becoming increasingly popular among developers. These tools can help complete, summarize, and retrieve code snippets based on natural language queries. However, not all code generators are equally accessible, reliable, or ethical. Some are expensive, some have restrictive licenses, and some may pose legal or security risks.

To address these challenges, a team of researchers and engineers from Hugging Face, ServiceNow, and Nvidia have developed StarCoder 2, a new open-source code generator that aims to provide a better alternative for developers. It is a family of models that can run on most consumer GPUs and can be fine-tuned and deployed locally.

It also claims to offer improved performance, transparency, and ethics compared to other code generators.

What is StarCoder 2 and how does it work?

StarCoder 2 is the successor of StarCoder, an open-source code generator that was created by Hugging Face and ServiceNow in 2023. It builds on the original by using more data, more parameters, and more collaborators. It comes in three variants:

• A 3-billion-parameter (3B) model trained by ServiceNow
• A 7-billion-parameter (7B) model trained by Hugging Face
• A 15-billion-parameter (15B) model trained by Nvidia

Parameters are the parts of a model that are learned from data and determine its skill on a task. In this case, the task is generating code. It was trained on 67.5 terabytes of code data from Software Heritage, a nonprofit organization that archives code from various sources. This is four times more data than the original StarCoder, which used 6.4 terabytes.

StarCoder 2 can perform various code-related tasks, such as:

• Completing unfinished lines of code
• Summarizing code snippets
• Retrieving code snippets based on natural language queries
• Generating code from natural language specifications

It can also be fine-tuned on custom data to create specific applications, such as chatbots or personal coding assistants. This can be done in a few hours using a GPU like the Nvidia A100. Moreover, it can learn from a developer’s source code or codebase, making it more adaptive and personalized.

What are the benefits of StarCoder 2?

According to the creators of StarCoder 2, the main benefits of their code generator are:

• Performance: StarCoder can generate code faster and more accurately than other code generators, such as Code Llama, a popular open-access model. Hugging Face says that StarCoder 2 15B can match Code Llama 33B on some code completion tasks at twice the speed.
• Transparency: StarCoder is fully open-source and reproducible. Developers can access the models, the source code, the training data, and the training recipe from the project’s GitHub page. They can also fork, reproduce, or audit the models as they please. This allows developers to have more insight and control over how the code generator works and what data it uses.
• Ethics: StarCoder is licensed under the BigCode Open RAIL-M 1.0, which is a license that promotes responsible use of AI by imposing some restrictions on both model licensees and downstream users.
• For example, StarCoder 2 cannot be used for medical advice-giving apps or for malicious purposes. The license also aims to comply with current laws and regulations, such as the EU AI Act. Furthermore, StarCoder 2 was trained only on data that was licensed by the Software Heritage, and code owners were given the option to opt out of the training set.
• This reduces the risk of violating intellectual property rights or exposing sensitive information.

What are the limitations of StarCoder 2?

Despite its advantages, StarCoder 2 is not perfect. It has some limitations and challenges, such as:

• License: StarCoder 2’s license, while less restrictive than some others, is not truly open. It still imposes some conditions that may not suit every developer or use case. Some critics also argue that the license is too vague or too complex to follow and that it may conflict with other regulations or licenses.
• Security: StarCoder 2, like other code generators, may introduce security vulnerabilities in the code it produces. A Stanford study found that engineers who use code-generating systems are more likely to create insecure apps. A Sonatype survey also found that most developers are worried about the lack of visibility and quality control over the code from code generators.
• Therefore, developers should always review and test the code generated by StarCoder 2 before using it in production.
• Bias: StarCoder 2, like other AI models, may reflect biases from the data it was trained on. For example, it may generate code that contains stereotypes about gender or race. It may also perform worse on languages other than English or on less common programming languages, such as Fortran or Haskell.

Why did Hugging Face, ServiceNow and Nvidia create StarCoder 2?

Hugging Face, ServiceNow, and Nvidia are all companies that have a stake in the AI and code generation market. By creating StarCoder 2, they hope to:

• Foster goodwill: By offering an open source and ethical code generator, they hope to gain trust and reputation among developers and the AI community. They also hope to contribute to the advancement of AI research and innovation.
• Build paid services: By providing a free and accessible code generator, they hope to attract more customers and users to their paid products and services. For example, ServiceNow has used StarCoder to create Now LLM, a code generation product for its workflow platform.
• Hugging Face and Nvidia also offer hosted versions of StarCoder 2 on their platforms, along with other features and support.

StarCoder 2 is a new open-source code generator that promises to deliver competitive performance, transparency, and ethics for developers. It is a family of models that can run on most GPUs and can be fine-tuned and deployed locally. It can also perform various code-related tasks based on natural language queries.

However, it also has some limitations and challenges, such as its license, security, and bias. StarCoder 2 was created by Hugging Face, ServiceNow, and Nvidia, who hope to foster goodwill and build paid services on top of the open-source release.