This model, featuring 12 billion parameters and roughly 24GB in size, builds on Mistral's existing text-based model, Nemo 12B. Parameters correspond to the model's ability to solve complex problems. Generally, the more parameters a model has, the better its performance. Pixtral 12B is tailored for tasks such as image captioning, object identification, and answering queries related to images.

Pixtral 12B expands on Nemo 12B's capabilities by allowing it to process an arbitrary number of images in any size, using either URLs or base64-encoded images, a binary-to-text encoding method. This approach is akin to other multimodal models, like Anthropic’s Claude series and OpenAI’s GPT-4. In theory, Pixtral 12B should excel at various tasks, including generating image captions and counting objects within photos.

The model is freely available under the Apache 2.0 license, permitting unrestricted use, modification, and commercialisation. Developers can access it through GitHub and Hugging Face, although fully functional web demos have not been released yet.

Mistral’s Head of Developer Relations has reported that Pixtral 12B would soon be incorporated into the startup’s chatbot, Le Chat, and its API platform, La Platforme.

Multimodal models like Pixtral 12B represent the next wave in generative AI, building on the progress made by models such as OpenAI's GPT-4 and Anthropic’s Claude. However, there are concerns about the data sources used for training these models.

Mistral released this model after securing $645 million in funding, reaching a valuation of $6 billion. With backing from Microsoft, the startup is positioning itself as Europe’s answer to OpenAI.

Mistral AI was founded by a team of former researchers from companies such as Meta and DeepMind, and has quickly emerged as a significant contender in the AI landscape. With a focus on developing cutting-edge AI models, Mistral aims to bridge the gap between European and American advancements in this rapidly evolving field. The startup has garnered considerable attention and financial backing, raising $645 million in its initial funding round and achieving a valuation of $6 billion. Notably, Microsoft is among its investors, signalling industry confidence in Mistral's potential to compete with established AI giants like OpenAI.