Storing, Feeding And Managing Data For Cloud, AI: The Biggest Storage News At AWS re:Invent 2024

Multiple storage vendors, along with Amazon itself, took advantage of this month’s AWS re:Invent 2024 conference to introduce and demonstrate new storage technologies aimed at helping businesses better store, protect, and manage and make it available for use by the latest AI and cloud workloads.

Storage For A Cloud And AI World

AWS re:Invent 2024, Amazon Web Services’ annual conference, highlighted a wide range of new technologies for improving business cloud and AI environments.

Storage technologies were a prominent part of the AWS re:Invent experience, reflecting the importance of storing, protecting, and managing data and ensuring that data is ready for use with the latest performance-demanding AI and cloud applications.

CRN has brought together the latest offerings from seven storage vendors who used AWS re:Invent 2024 to introduce new hardware and software aimed at backing up data, providing visibility into stored data, providing seamless access wherever it is needed, improving performance, and easily extending data to AI environments.

Amazon itself was busy showing off new technologies aimed at making stored data more useful for S3 and EC2 workloads.

AWS re:Invent 2024 was a great place to learn about the latest storage technologies for cloud and AI environments. Read on for a look at nine of those technologies.

Amazon S3 Tables

Amazon S3 Tables provides storage that is optimized for tabular data such as daily purchase transactions, streaming sensor data, and ad impressions in Apache Iceberg format. It aims to offer easy queries using popular query engines like Amazon Athena, Amazon EMR, and Apache Spark.

When compared to self-managed table storage, Amazon estimates that users can expect up to 300 percent faster query performance and up to 10 times more transactions per second. Users can also expect improved operational efficiency as part of a fully managed service.

Amazon EC2 18g Instances

Amazon also announced the general availability of Amazon EC2 I8g instances, a new storage-optimized instance type aimed at providing high real-time storage performance among storage-optimized EC2 instances with the third generation of AWS Nitro SSDs and AWS Graviton4 processors.

It features the AWS Graviton4 processor for running a broad range of workloads on EC2 instances using a 64-bit ARM instruction set architecture. EC2 I8g instances are the first instance type to use third-generation AWS Nitro SSDs. These instances offer up to 22.5 TBs of local NVME SSD storage with up to 65 percent better real-time storage performance per TB and 60 percent lower latency variability compared to the previous generation I4g instances. I8g instances offer up to 96 vCPUs, 768 GiB of memory, and 22.5 TB of storage to deliver more compute and storage choices compared with I4g instances.

Eon Platform

The Eon platform from New York-based Eon helps enterprises take advantage of their cloud backups by providing full visibility and seamless access to data. Eon autonomously and continuously scans, maps, and classifies cloud resources and provides backup recommendations based on business and compliance needs while ensuring appropriate backup policies are in use.

Unlike competing technologies based on non-searchable and vendor-locked snapshots, Eon’s backup storage is fully manageable and portable while providing global search capabilities so customers can find and restore individual files and run SQL queries without any resource provisioning.

Pure Storage FlashArray for AWS Outposts

Pure Storage, headquartered in Santa Clara, Calif., used AWS re:Invent to introduce its Pure Storage FlashArray for AWS Outposts, a hybrid cloud solution that integrates Pure Storage’s enterprise-grade FlashArray with AWS services to help modernize on-premises data centers.

The combination seamlessly connects AWS compute with Pure Storage’s high-performance FlashArray storage to provide a simplified user experience through the AWS Management Console. It brings high-performance all-flash storage, advanced data reduction, and built-in security and compliance features with AWS compute to help businesses innovate their on-premises hybrid cloud infrastructures.

Nutanix Enterprise AI

San Jose, Calif.-based Nutanix used AWS re:Invent to show how it has extended its AI infrastructure platform with Nutanix Enterprise AI, a new-cloud native offering. Nutanix Enterprise AI can be deployed on any Kubernetes platform, at the edge, in core data centers, and on public cloud services like AWS EKS, Azure AKS, and Google GKE.

The new offering leverages Nvidia NIM inference microservices for optimized performance of foundation models to help organizations securely deploy, run, and scale inference endpoints for large language models (LLMs) to support the deployment of GenAI applications.

Commvault Clumio Backtrack

Commvault, based in Tinton Falls, N.J., in September acquired data protection technology provider Clumio and used AWS re:Invent to introduce Clumio Backtrack, a new technology that lets business users recover billions of Amazon S3 objects to a specific version and point and time in just seconds.

The technology, which Clumio said takes Amazon S3 Versioning to a new level, can help cloud operations, IT, and security teams recover data from errors, accidents, or cyberattacks. Backtrack can also be used to recover datasets of practically any size, scaling from individual objects to massive Amazon S3 buckets.

N2WS Backup and Recovery 4.4

West Palm Beach, Fla.-based N2WS showed off new functionality in N2WS Backup and Recovery, the company’s cloud-native backup and disaster recovery platform for enterprises and MSPs, at AWS re:Invent. These improvements aim to address the growing threats of ransomware and other malicious attacks while cutting operational costs, streamlining cross-cloud and multi-cloud data management, and helping maximize the potential of cloud investments.

Now available to Amazon AWS and Microsoft Azure customers using AWS Backup & Recovery, the updates include per-VM pricing for Azure backups, advanced snapshot technology for enhanced speed and flexibility across repositories, partial retry capabilities for policy execution failures, improved recovery functionality, and smarter S3 compliance locking.

MinIO AIStor

MinIO used AWS re:Invent to introduce MinIO AIStor, a new enterprise object storage technology aimed at exascale data infrastructure challenges presented by AI workloads. While Redwood City, Calif.-based MinIO has always offered scale and performance for these workloads, AIStor adds new functionality to help guarantee operational, technical, and economic success.

These capabilities include a new S3 API, known as promptObject, enabling users to “talk” to unstructured objects in the same way one would engage an LLM; a private Hugging Face repository for storing AI models and datasets called AIHub; support for S3 over RDMA; and a new global console to launch and manage AI applications in a single interface.

Rubrik Annapurna for Amazon Bedrock

Rubrik Annapurna for Amazon Bedrock was Palo Alto, Calif.-based Rubrik’s highlight at AWS re:Invent 2024. The new technology lets customers have ready-to-go access to all enterprise data for GenAI initiatives. Delivered as a single API service, customers can tap into all secure data embeddings from Rubrik Security Cloud, including data across on-premises, cloud, and SaaS environments.

By integrating its secure data retrieval capabilities with Amazon Bedrock, Rubrik Annapurna helps customers access a broad selection of high-performing foundation models. The result is a single flexible and configurable datastore for use across an organization’s entire stack of AI applications.