JetBrains, the company behind a range of popular app development tools, has released its first “open” AI model for coding.
On Wednesday, JetBrains made Mellum, a code-generating model the company released for its various software development suites last year, openly available on the AI dev platform Hugging Face. Mellum, trained on more than 4 trillion tokens, weighs in at 4 billion parameters and is designed specifically for code completion (i.e., completing code snippets based on the surrounding context).
Parameters roughly correspond to a model’s problem-solving skills, while tokens are the raw bits of data that a model processes. A million tokens is equivalent to ~30,000 lines of code.
“Designed for integration into professional developer tooling (e.g., intelligent code suggestions in integrated developer environments), AI-powered coding assistants, and research on code understanding and generation, Mellum is also well-suited for educational applications and fine-tuning experiments,” explains JetBrains in a technical report.
JetBrains says that it trained Mellum, which is Apache 2.0 licensed, on a collection of datasets, including permissively licensed code from GitHub and English-language Wikipedia articles. Training took around 20 days on a cluster of 256 H200 Nvidia GPUs.
Mellum takes some work to get up and running. The base model can’t be used out of the box; it has to be fine-tuned first. While JetBrians has provided a few Mellum models fine-tuned for Python, the company cautions that they’re meant for “estimation about potential capabilities” — not deploying into a production environment.
AI-generated code is no doubt changing how software is built, but it’s also introducing new security challenges. More than 50% of organizations encounter security issues with AI-produced code sometimes or frequently, according to a late 2023 survey by developer security platform Snyk.
