Every enterprise these days wants to leverage artificial intelligence (AI), and they want to be able to do so with a variety of content — not just text, but code, video, audio, images (and undoubtedly more to come).

This makes multimodality ever more critical in the enterprise AI market: That is, the ability for AI models to digest — and, more importantly, provide useful, relevant answers to — a multitude of different kinds of inputs.

AI providers are rushing to meet this demand and set themselves apart as the latest and greatest, and Google is no exception. The tech giant today upped the ante with the announcement of its new multimodal Gemini Pro for enterprise use.

[Related: Google Cloud’s genAI and security integration improves threat detection and response]

The news comes just a week after the company unveiled its Gemini flagship generative AI (genAI) model that it calls its largest and “most capable” — but which quickly came under criticism for its image analysis capabilities (or lack thereof).

“Gemini, the first native multimodal model, is a game-changer and sets new benchmarks that other models will be measured against,” Gartner VP analyst Chirag Dekate told SDxCentral. “With Gemini, Google is realizing its potential as an AI-first company.”

The Gemini Pro API is available for enterprises through Google Cloud’s Vertex AI platform, and for developers in Google AI studio. It incorporates a dedicated Gemini Pro Vision multimodal input that accepts both text and imagery inputs for text outputs.

Building and deploying models with Gemini

Google’s Gemini comes in three sizes: Ultra, Pro and Nano. Ultra is currently in private preview for select customers, Nano is available in Android, and Pro can be found in Google’s Bard chatbot.

Gemini Pro is being offered as both a free version for users to try out “within limits” (translation: it’s free for up to 60 queries per minute, and anything users input will be used to help further train the model) and a pay-as-you-go version. “The free-access model enables all consumers to evaluate and test the model’s capabilities,” said Dekate.

Pay-as-you-go starts at 60 queries a minute and features chat, function calling, embeddings, semantic retrieval and custom knowledge grounding. It supports 38 languages and Python, Android (Kotlin), Node.js, Swift and JavaScript.

Enterprises looking to scale can access Gemini Pro APIs through Google’s Vertex AI and Google AI Studio. This will enable them to build powerful, multimodal AI models with built-in safety and security guardrails.

Developers have the ability to select from a curated list of more than 130 models from Google as well as open-source and third parties, according to Google.

They can then customize model behavior with specific enterprise data using tuning tools that adjust model weights when necessary. Supported tuning techniques include prompt design, adapter-based low-rank adaptation (LoRA) and distillation. Organizations can use grounding against their structured and unstructured data and Google Search technology. Vertex AI also enables reinforcement learning from human feedback (RLHF).

Google has also released a new automated, on-demand evaluation model, Automatic Side by Side (Auto SxS), which compares models. This is faster and more cost-effective than manual evaluation, according to Google, and allows for quick model deployment and maintenance.

Google commented in a blog post that “we’ll be further fine-tuning it in the weeks and months ahead as we listen and learn from your feedback.”

Navigating the ‘hype-intensive’ market

Going forward, we’ll likely see several providers offering free access to innovation, Dekate noted.

As enterprises navigate this “hype-intensive” market, he suggested they actively explore the benefits of native multimodal AI over alternative technologies offering “several disparate unimodal techniques to emulate multimodality.”

“Bottom line, not all free models are the same, and enterprises should be selective in investing time and resources in techniques that deliver the most value and enable the latest innovations in their enterprise context,” said Dekate. “Anything less would be a distraction.”

Gemini across Google’s cloud portfolio

Gemini features “ultra-scale” AI infrastructure and is “natively multimodal,” meaning it can use text, code, audio, video and images “in a way that none of its peers can,” said Dekate, noting that Gartner anticipates Google to infuse Gemini across its cloud portfolio.

“The results are accurate and enable hyper-immersive experiences that other models struggle to match,” he said. “It is no surprise that Gemini sets the new benchmark in most measures.”

Gemini is trained on Google’s custom AI Hypercomputer — comprising TPUs and custom interconnects including Jupiter and OCS — allowing Google to deliver the model at lower cost and energy footprint compared to conventional GPU-based tools, said Dekate.

While other providers rely on “simplistic prompt and text response-type interfaces,” Gemini is integrated with AlphaCode2, allowing the system to create interactive experiences with complex responses.

“The combination of these capabilities is bound to unlock enterprise use cases that are beyond the legacy text prompts,” said Dekate.

Other Google AI updates

As part of today’s announcement, Google also noted that it has upgraded its Imagen 2 text-to-image diffusion tool. According to Google’s blog post, “Imagen 2 delivers significantly improved image quality and a host of features, including the ability to generate a wide variety of creative and realistic logos — including emblems, lettermarks and abstract logos — for business, brands and products, and can deliver improved results in areas where text-to-image tools often struggle, including rendering text in multiple languages.”

The cloud giant also introduced MedLM, a family of foundation models fine-tuned for healthcare. MedLM can be accessed by U.S. Google Cloud customers via Vertex AI.

Finally, the company announced the general availability of Duet AI for Developers, which helps developers build applications using AI-powered coding and chat assistance, and Duet AI in Security Operations, which the company claims it is the “first major cloud provider to make generative AI generally available to defenders in a unified SecOps platform.”

Google also promised to make both its MedLM and Duet products available soon in Gemini.