Google Cloud Next: 10 Huge Nvidia, Arm, AI And Workspace Launches

Google launched its first-ever custom ARM-based CPU, a new TPU, joint innovation with Nvidia and several new AI features for Workspace. Here are 10 new products launched today at Google Cloud Next 2024.

Thousands of Google partners, developers and customers swarmed to Google Cloud Next 2024 this week in Las Vegas to learn about new CPU and TPU products, Workspace AI features and Google’s bullish artificial intelligence roadmap.

At Google Cloud Next Tuesday, the tech giant unveiled ten new AI-optimized infrastructure, Workspace and Vertex AI offerings. This includes Google’s first-ever custom ARM-based CPU chip for the data center, Google’s new TPU v5p, and new AI features in Google Workspace for meeting and messaging.

“The world is changing, but at Google, our north star is the same: to make AI helpful for everyone, to improve the lives of as many people as possible,” said Google Cloud CEO Thomas Kurian today at Google Cloud Next 2024.

Google Cloud also unveiled today when Nvidia’s newest Grace Blackwell platform will be coming to Google Cloud, as well as a new jointly developed A3 mega instance that leverages Nvidia’s H100 GPUs.

Google Workspace Injected With More AI

Google Workspace, which includes Gmail, Docs, Sheets, Meet and more, has more than 3 billion users worldwide along with 10 million paying customers.

Kurian said over the past eight months, Google has introduced “over a thousand product advances” across Google Cloud and Workspace.

“We have expanded our planet-scale infrastructure to 40 regions and announced new subsea cable investments to connect the world to our cloud with predictable low latency,” he said. “We have introduced new, state-of-the-art models — including our Gemini models — and brought them to developers and enterprises.”

From Google’s new CPU and chip offerings to a slew of new AI features injected inside Vertex AI and Workspace, here are the 10 product launches from Google today during Google Cloud Next 2024 that you should know about.

Google’s First Arm-Based CPU: Axion

Google unveiled its first-ever custom ARM-based CPU chip designed for the data center with Google Axion.

Google Axion is a general-purpose compute option that delivers up to 50 percent better performance and 60 percent better energy efficiency than comparable current-generation x86-based instances, Google said.

“While not every workload is an AI workload, every workload you run in the cloud needs optimization — from web servers to containerized microservices,” said Kurian (pictured) referring to Axion. “Each application has unique technical needs, which is why we’re pleased to introduce new, general-purpose compute options that help customers maximize performance, enable interoperability between applications, and meet sustainability goals, all while lowering cost.”

Google also announced two new machine series for its general purpose VM portfolio in N4 and C4, along with the general availability of Hyperdisk Advanced Storage Pools.

Google TPU v5p Goes GA; New Google Kubernetes Engine Support

At Google Cloud Next 2024, Google announced the general availability of TPU v5p. Each TPU v5p pod composes together nearly 9,000 chips over Google’s highest-bandwidth inter-chip interconnect (ICI).

Google says its new TPU can train large language models nearly three times faster than the previous generation TPU v4.

“TPU v5p is our most powerful, scalable, and flexible AI accelerator for training and inference, with 4X the compute power per pod compared to our previous generation,” said Google Cloud’s CEO.

Additionally, Google announced new availability of Google Kubernetes Engine (GKE) support for TPU v5p.

“Over the past year, the use of GPUs and TPUs on GKE has grown more than 900 percent,” Kurian said.

Nvidia’s New Grace Blackwell Chips Coming To Google Cloud In Two Variations

One of the biggest anticipated Nvidia AI platforms in years is coming to Google Cloud in early 2025.

Nvidia’s newest Grace Blackwell platform will be coming to Google Cloud in early 2025 in two variations: B200 and GB200 NVL72.

The B200 is designed for mainstream training and serving, while the GB200 NVL72 is designed for next-frontier model training and serving.

The Nvidia Grace Blackwell platform is looking to enable businesses of all shapes and sizes to build and run real-time generative AI on trillion-parameter LLMs. The GPU architecture features technologies for accelerated computing that Nvidia says will unlock breakthroughs in data processing, engineering simulation, electronic design automation, quantum computing and GenAI.

Gemini And Google Workspace: Google Vids App

Google Gemini AI has been injected into Google Workspace with the creation of AI-powered video creation app: Google Vids.

Google Vids is an all-in-one video, writing, production, and editing assistant. It can generate a storyboard users can easily edit, and after choosing a style, it pieces together a first draft with suggested scenes from stock videos, images, and background music.

Vids will sit alongside Google’s other productivity tools like Docs, Sheets, and Slides. It includes an easy-to-use interface and the ability to collaborate and share projects securely from a user’s own browser.

Google Vids is being released to Workspace Labs in June.

New AI Features In Workspace For Meetings and Messaging

Google injected new AI features into Workspace including in Meet, with a new ‘Take notes for me’ option that transcribes and takes notes during meetings.

Other features for Meet include ‘Translate for me’ which will automatically detect and translate captions in Meet with support for 69 languages. Another feature is automatic translation of messages and on-demand conversation summaries in Google Chat.

Google says these new collaboration features will cost $10 per user, per month.

Additionally, Workspace administration can now automatically classify and protect sensitive files and data using privacy-preserving AI models and Data Loss Prevention controls trained for their organization.

These new AI security add-ons to Workspace are available for $10 per user, per month.

New A3 Mega GPU Instance Developed With Nvidia

Google and Nvidia developed a new A3 mega instance that leverages Nvidia’s H100 GPUs.

“This new GPU-based instance is generally available and delivers twice the bandwidth per GPU compared to A3 instances, to support the most demanding workloads,” said Kurian.

Google also unveiled Confidential A3, which enables customers to better protect the confidentiality and integrity of sensitive data and AI workloads during training and inferencing.

“The potential for gen AI to drive rapid transformation for every business, government and user is only as powerful as the infrastructure that underpins it,” Kurian added.

New AI In Google Distributed Cloud

Google says it’s bringing AI closer to where the data is being generated and consumed—from the edge to air-gapped environments to Google Sovereign Clouds and cross-cloud.

“We are enabling AI anywhere through Google Distributed Cloud (GDC), allowing you to choose the environment, configuration, and controls that best suit your organization's specific needs,” Kurian said.

Several new capabilities in GDC include bringing Nvidia GPUs to GDC for both connected and air-gapped configurations. Each of these will support new GPU-based instances to run AI models efficiently.

Additionally, Google Kubernetes Engine (GKE) is now available in GDC, as well as AlloyDB Omni for Vector Search to allow search and information retrieval on GDC for customers private and sensitive data with low latency.

Lastly, Google is validating a variety of open AI models—including Gemma and Llama 2—on GDC to run in air-gapped and connected edge environments.

New MLOps Capabilities For Vertex AI

Google launched new machine learning operations (MLOps) capabilities for Vertex AI including prompt management tools that lets user collaborate on built-in prompts with notes and status, track changes over time, and compare the quality of responses from different prompts.

“We’ve expanded Vertex AI’s MLOps capabilities to meet the needs of those building with large models, letting customers work on all AI projects with a common set of features, like model registry, feature store, pipelines to manage model iteration and deployment, and more,” said Google Cloud’s Amin Vahdat, vice president and general manager for ML, Systems and Cloud AI.

New features include automatic side-by-side (Auto SxS) which provides explanations of why one response outperforms another and certainty scores, which helps users understand the accuracy of the evaluation.

Another new feature is Google’s Rapid Evaluation feature that helps customers quickly evaluate models on smaller data sets when iterating on prompt design.

Vahdat added that with the new capabilities in Vertex AI, customers can “continue to benefit from their existing MLOps investments, while meeting the needs of their Gen AI workloads.”

Vertex AI Agent Builder

Google launched its new Vertex AI Agent Builder that brings together foundation models, Google Search, and other developer tools, to make it easy for customers to build and deploy agents.

The new offering provides the convenience of a no-code agent builder console alongside powerful grounding, orchestration and augmentation capabilities. With Vertex AI Agent Builder, users can now quickly create a range of generative AI agents, grounded with Google Search and their own organization’s data.

New AI Models And Capabilities For Vertex AI

At Google Cloud Next, the company expanded access to a variety of models to give more customer choice in AI model selection.

Anthropic’s new Claude 3 family of AI models is now generally available for customers on Google’s Vertex AI.

A new fine-tuned version of Google Gemma designed for code generation and code assistance, CodeGemma, is also now available on Vertex AI. Gemma is a family of lightweight open models built by the same technology used to create Google’s Gemini models.

Also new to Vertex AI is Google’s most advanced text-to-image technology yet: Imagen 2.0. The new Imagen offering has a variety of image generation features to help businesses create images that match their specific brand requirements. A new text-to-live image capability allows marketing and creative teams to generate animated images equipped with safety filters and digital watermarks.