AWS re:Invent 2024 - Storage
Back to all Re:Invent 2024
Table of Contents
Compute
 |
This session provides a comprehensive overview of building a data foundation to fuel generative AI applications. It covers different approaches to reusing or training foundation models, data considerations like discovery, preparation, and governance, as well as storage infrastructure options like Amazon S3 and Amazon FSx for Lustre to support these workloads.
|
 |
This talk provides an overview of how AWS Storage Gateway can be used to build hybrid data workflows that leverage Amazon S3 and other AWS services. The presentation covers key use cases, features, and a live demonstration of an event-driven architecture that uses Storage Gateway to move on-premises data to the cloud and trigger downstream processing and analytics.
|
Databases
 |
S3 Tables is a new fully-managed Apache Iceberg table storage service in Amazon S3, providing high-performance, simplified security, and automated cost optimization for customers' tabular data workloads. The service integrates seamlessly with AWS analytics services, enabling easy querying, secure access control, and real-time analytics capabilities.
|
 |
This talk discusses the journey of Forever Ecommerce, a fictional company, in building and optimizing a data lake on Amazon S3. The presentation covers key aspects such as data architecture, security, governance, and performance optimization to create a scalable and efficient data platform.
|
Developer Experience
 |
The video discusses how AWS DataSync can help customers accelerate and automate secure data transfers at scale, showcasing Resilience's use case in the biomanufacturing industry. It highlights DataSync's capabilities, such as high-speed data movement, encryption, and parallel processing, which enable customers to efficiently manage large-scale data transfers.
|
 |
The session introduces Storage Browser for Amazon S3, an open-source React UI component that allows customers to connect end-users to their data in S3 through their own applications, providing features like federated access, customizable UI, and optimized SDK for improved performance and user experience.
|
Storage
 |
This presentation provides a deep dive into Amazon Elastic File System (EFS), a serverless, fully elastic file storage service that simplifies file storage and enables easy application modernization. The presenter showcases EFS's seamless integrations, performance enhancements, cost optimization features, and robust security capabilities, demonstrating how EFS can help customers focus on solving business problems rather than managing storage infrastructure.
|
 |
The video discusses the latest updates and announcements from AWS re:Invent 2024 related to file storage services, including Amazon EFS and Amazon FSx. It covers improvements in managing data sprawl, performance scalability, and resilience to better serve the evolving needs of customers.
|
 |
The video covers the various mechanisms Amazon S3 uses to achieve 11 9s of durability, including integrity checks, data redundancy, and periodic durability auditing. It then explores additional data protection features like versioning, multi-factor authentication, and object locking to help customers protect their critical data against accidental deletions, overwrites, and lifecycle expirations.
|
 |
This session covers the key features and use cases of Amazon EBS snapshots, including encryption, snapshot sharing, snapshot lock for immutability, and integration with Amazon Data Lifecycle Manager for automated backup and recovery. The session also showcases how SAP leverages EBS snapshots to enable business continuity and ransomware protection for their RISE with SAP offering.
|
 |
The video discusses best practices for securing and controlling access to data stored in Amazon S3, covering basic S3 security features, IAM-based access control approaches, and advanced techniques like S3 Access Grants and AWS Lake Formation for managing access to structured and unstructured data at scale.
|
 |
This session discusses data protection and resilience strategies using AWS storage services. The presenters cover topics such as high availability, disaster recovery, backup and replication, and demonstrate how to implement these strategies using AWS services like S3, EBS, and Elastic Disaster Recovery.
|
 |
This presentation discusses various strategies and features introduced by AWS to optimize the performance of Amazon S3 storage. The key points covered include leveraging new storage classes, utilizing the AWS Common Runtime, optimizing file sizes and partitioning data, and employing prefetching and caching techniques to improve overall workload performance.
|
 |
Amazon FSx provides fully managed network-attached storage (NAS) solutions in the cloud, offering customers the ability to migrate their on-premises file storage to the cloud while maintaining the same functionality and ease of use they are familiar with. The presentation highlights how FSx can help address the challenges of managing and scaling on-premises file storage, while providing performance, cost optimization, and agility features to enable customers to focus on their core business needs.
|
 |
The session discusses how to modernize managed file transfer with SFTP using AWS Transfer Family. It covers the key features and capabilities of the service, including support for multiple protocols, authentication options, event-driven workflows, and integration with AWS data services.
|
 |
This session covers best practices for cost optimization of AWS storage services, including Amazon EBS, Amazon FSx, Amazon EFS, and Amazon S3. The presenters provide real-world examples and actionable strategies to help customers rightsize their storage, leverage native tools for monitoring and automation, and continuously optimize their storage costs.
|
 |
This talk provides an overview of Amazon S3 Glacier storage classes and best practices for managing cold data, including using S3 Lifecycle policies and Intelligent-Tiering to optimize storage costs. The presenters also discuss strategies for efficiently restoring data from Glacier storage, highlighting the benefits of using S3 Batch Operations to streamline the restoration process.
|
 |
AWS Transfer Family introduces no-code web apps to simplify file sharing and access control within organizations. The web apps leverage AWS Identity Center and S3 Access Grants to provide a secure, branded, and user-friendly interface for both technical and non-technical users to interact with their organizational data stored in Amazon S3.
|
 |
The session provides a deep dive into how Amazon S3 leverages scale to its and customers' advantage. It discusses techniques like shuffle sharding, power of two random choices, and erasure coding that enable S3 to deliver consistent performance, resilience, and development velocity at scale.
|
 |
This presentation covers the latest updates and enhancements to Amazon S3, including the launch of S3 Tables for structured data, S3 Metadata for unstructured data, and various performance, security, and durability improvements. The presenters provide a comprehensive overview of the new features and how they can benefit customers in managing their data storage and processing needs on AWS.
|
 |
The session discusses the launch of AWS S3 Metadata, a new feature that automatically extracts and manages rich metadata for objects stored in S3, enabling customers to easily discover and extract value from their growing unstructured data. The presentation covers the key capabilities of S3 Metadata, including automatic generation of system and custom metadata, integration with S3 Tables for optimized querying, and examples of how customers can leverage this feature to address common data discovery challenges.
|
 |
This session covers how to maximize performance for high-intensity workloads using Amazon EBS, with a focus on the io2 Block Express volume type. It provides real-world case studies from industries like healthcare and SAP HANA, demonstrating the benefits of io2 Block Express in terms of high IOPS, throughput, and consistent low latency.
|
 |
Amazon FSx Intelligent-Tiering, a new storage class for Amazon FSx for OpenZFS, provides fully elastic, intelligently tiered storage that reduces storage costs for large NAS data sets by up to 85% compared to the existing SSD storage class. This new storage class combines the functionality of network-attached storage with the benefits of cloud-native storage, including automatic data tiering and lower storage costs, making it easier for customers to migrate their large on-premises data sets to the cloud.
|
 |
This session explores how Canva, Bria AI, and Anthropic leverage AWS services like Amazon S3 to manage massive amounts of data and build scalable AI solutions. The speakers discuss strategies for data curation, performance optimization, and cost management to enable their AI innovations.
|
 |
The session discusses the importance of building resilience against ransomware using AWS Backup, covering the cyber threat landscape, backup strategies, and recovery planning. The key focus is on minimizing the mean time to normal operations and ensuring business continuity in the event of a ransomware attack.
|
 |
This talk provides an overview of the latest features and capabilities of Amazon Elastic Block Store (EBS), focusing on how customers can design resilient architectures by understanding potential risks, minimizing the scope of impact, and automating recovery. The speakers discuss new volume types, monitoring and observability tools, and disaster recovery capabilities to help customers build highly available and performant applications on AWS.
|
 |
This session provides an overview of AWS Backup, highlighting its capabilities in centralized data protection, disaster recovery, and ransomware protection. The presentation features insights from AWS product managers and customer case studies, showcasing how Freddie Mac and Skyscanner have leveraged AWS Backup to address their data protection needs.
|
 |
The video discusses AWS storage services and how customers can build a data foundation in the cloud. It covers various use cases, including migrating enterprise applications, building data lakes, using storage for AI and ML workloads, and implementing data protection strategies.
|
 |
The presentation discusses high-performance storage solutions for AI/ML, analytics, and HPC workloads, focusing on Amazon S3 for data lakes and Amazon FSx for Lustre for file-based workloads. It also covers strategies for leveraging the strengths of both object and file storage to optimize performance and cost for various use cases.
|
 |
This presentation covers the various storage options available on AWS, including Amazon EBS, FSx for Windows File Server, FSx for OpenZFS, and FSx for NetApp ONTAP, highlighting their unique features and capabilities for supporting database workloads. The speakers provide detailed insights into the benefits of these storage solutions, such as high availability, data protection, and performance optimization, and offer guidance on how to choose the most suitable option based on specific requirements.
|