On March 17, Elon Musk’s AI company, xAI, released the large language model (LLM) Grok-1 as open source. The billionaire declared last week that the AI chatbot will be open-sourced on his social media platform X, which was formerly known as Twitter. It is currently accessible to developers and researchers. Interestingly, the xAI developers said that only the pre-trained LLM has been made available to the general public. This implies that although Grok lacks training data, you can still build upon it using the weights and network architecture.
“We are releasing the base model weights and network architecture of Grok-1, our large language model,” xAI wrote in a blog post announcing the open release. Grok-1 is a Mixture-of-Experts model with 314 billion parameters that was trained from scratch using xAI. The AI company also mentioned that the Apache 2.0 license is being used to provide the LLM as open source software. GitHub is where interested parties can obtain the AI model.
VentureBeat reported that the Apache 2.0 license permits both commercial use and modification and redistribution. This implies that programmers can enhance the LLM, customize it for particular uses, and market it. The model cannot be trademarked, though, and developers will need to credit any modifications they made to the original code.
The version of Grok-1 that has been made available is from October 2023, before the model was trained using data from X, and this gives the Grok chatbot a distinct personality, even though a commercial license is being offered. This implies that obtaining the data will be the responsibility of any researchers or developers wishing to use the model.
The publicly available Grok-1 model, according to xAI, is a Mixture-of-Experts (MoE) model with 314 billion parameters. Compared to other LLMs in the public domain, like the Mistral 8x7B or Meta LLaMa 2, the parameter size is substantially larger. The AI’s context window grows as the parameter size increases. It can now respond with greater contextual accuracy and thoroughness as a result.
Grok, a chatbot created to rival the industry leaders, was introduced on November 3, 2023. Limited users who bought the X Premium+ subscription had access to it. Additionally, Musk announced at the time that Grok will receive a stand-alone app.