Tutorial / Cram Notes
Amazon EFS provides a simple, scalable, elastic file storage for use with AWS Cloud services and on-premises resources. It is built to scale on demand to petabytes without disrupting applications, growing and shrinking automatically as files are added and removed.
Use Cases:
- Shared File Storage: For applications that require a shared file system accessible by multiple EC2 instances.
- Big Data and Analytics: Suitable for analytics workloads that require shared access to data lakes.
- Content Management and Web Serving: Streamlines the sharing of assets and content across different servers.
Benefits:
- Simplicity and Scalability: Automatically scales and requires no pre-provisioning of storage.
- Built-in Redundancy and Availability: EFS is designed for high availability and durability across multiple Availability Zones.
Example:
Mounting an EFS file system to an EC2 instance:
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 file-system-id.efs.region.amazonaws.com:/ /mnt/efs
Amazon S3 (Simple Storage Service)
Amazon S3 offers object storage with a simple web service interface to store and retrieve any amount of data from anywhere on the web. It is designed to deliver 99.999999999% (11 9’s) of durability and scales past trillions of objects worldwide.
Use Cases:
- Backup and Restore: Ideal for backup and recovery solutions.
- Data Lakes: Can be used to create data lakes for analytics purposes.
- Static Websites: Can host static websites and content with a global reach.
Benefits:
- High Durability and Availability: Provides robust, secure, and scalable storage.
- Wide Range of Storage Classes: Offers storage classes optimized for various use cases like frequently accessed data to seldom-accessed data with lower storage costs.
Example:
Uploading a file to Amazon S3 using AWS CLI:
aws s3 cp my-file.txt s3://my-bucket/
Amazon Elastic Block Store (Amazon EBS)
Amazon EBS is a block storage service designed for use with Amazon EC2 for both throughput and transaction-intensive workloads at any scale. A broad range of workloads, such as databases, enterprise applications, containerized applications, and big data analytics engines, can benefit from durable block storage.
Use Cases:
- Databases: Supports relational and NoSQL databases like MySQL, PostgreSQL, and MongoDB.
- Enterprise Applications: Ideal for ERP, CRM, and other business-critical applications.
- Boot Volumes: Can be used as the primary storage for boot volumes of EC2 instances.
Benefits:
- High Performance: Provides consistently high performance for both IOPS-intensive and throughput-intensive workloads.
- Snapshots and Cloning: Allows taking point-in-time snapshots and volume cloning for data backup and recovery.
Example:
Creating an EBS volume and attaching it to an EC2 instance:
aws ec2 create-volume –availability-zone us-west-2a –size 10 –volume-type gp2
aws ec2 attach-volume –volume-id vol-xxxxxxxx –instance-id i-xxxxxxxx –device /dev/sdf
Comparison Table
| Feature | Amazon EFS | Amazon S3 | Amazon EBS |
|---|---|---|---|
| Type | File Storage | Object Storage | Block Storage |
| Access | Multiple EC2, on-premises | HTTP/S, RESTful API | Single EC2 instance |
| Scaling | Automatic | Automatic | Manual Provisioning |
| Durability | High across multiple AZs | Very High | High, snapshot to S3 for increased durability |
| Availability | High across multiple AZs | Very High | High |
| Performance | Lower throughput than EBS/S3 | High-throughput, low latency | High IOPS, low latency |
| Use Cases | Shared access, analytics | Web hosting, Datalakes, Backup | Databases, Boot volumes |
By understanding the strengths and weaknesses of Amazon EFS, S3, and EBS, DevOps engineers can make informed decisions when architecting systems in AWS. These storage solutions facilitate a wide range of applications and workloads, each with its own set of features, price points, and performance characteristics. It is also important to keep an eye on the evolving services and features within AWS to continually optimize the storage architecture as the application needs and AWS offerings change.
Practice Test with Explanation
True or False: Amazon Elastic File System (Amazon EFS) is optimized for high-performance workloads and supports file-level locking and consistency.
- True
- False
Answer: True
Explanation: Amazon EFS is designed to provide a scalable, elastic, cloud-native file system with file-level locking and strong consistency, making it suitable for high-performance workloads.
Amazon S3 supports which of the following data consistency models?
- A. Read-after-write consistency for PUTS of new objects
- B. Eventual consistency for overwrite PUTS and DELETES
- C. Strong consistency for all read and write operations
- D. Read-after-write consistency for updates to existing objects
Answer: A, B
Explanation: Amazon S3 offers read-after-write consistency for PUTS of new objects and eventual consistency for overwrite PUTS and DELETES. As of December 2020, Amazon S3 provides strong consistency for all read and write operations, but this question relates to patterns which could include past behavior.
Which AWS storage service is a block-level storage and is suitable for use with EC2 instances?
- A. Amazon S3
- B. Amazon EBS
- C. Amazon EFS
- D. Amazon Glacier
Answer: B
Explanation: Amazon EBS provides block-level storage volumes for use with EC2 instances.
True or False: Amazon EFS can be directly mounted on an on-premises server via Direct Connect or VPN.
- True
- False
Answer: True
Explanation: EFS can be mounted on on-premises servers using AWS Direct Connect or a VPN, making it possible to create a seamless experience between cloud and on-premises environments.
Which of the following are characteristics of Amazon S3? (Select TWO)
- A. Object-level storage
- B. Can provision IOPS
- C. Designed for infrequently accessed data
- D. Supports NFS and SMB protocols
- E. Automatically scales to high request rates
Answer: A, E
Explanation: Amazon S3 is an object-level storage service that automatically scales to support very high request rates. Provisioning IOPS is a characteristic of Amazon EBS, not S3, and support for NFS and SMB protocols is a feature of Amazon EFS, not S Amazon S3 is designed for a range of access patterns, from frequently accessed to infrequently.
True or False: Amazon EBS volumes persist independently from the life of the EC2 instance they are attached to.
- True
- False
Answer: True
Explanation: Amazon EBS volumes exist independently of the EC2 instance lifespan, and you have the option to retain EBS volumes when the EC2 instance is terminated.
Amazon S3 Intelligent-Tiering is most beneficial for:
- A. Data with unknown or changing access patterns
- B. Data that is frequently accessed
- C. Data that is infrequently accessed but requires millisecond access times when needed
- D. Data that is archived for long-term storage and seldom accessed
Answer: A
Explanation: Amazon S3 Intelligent-Tiering is designed for data with unknown or changing access patterns, as it automatically moves data between different storage tiers based on access patterns to optimize costs.
True or False: Amazon EFS is not suitable for use cases that require a single EC2 instance to scale beyond the limits of one EC2 instance’s file system throughput.
- True
- False
Answer: False
Explanation: Amazon EFS is highly suitable for scale-out workloads because it can grow as needed and supports a large number of concurrent NFS connections, making it a good choice even when the workload exceeds the throughput capacity of a single EC2 instance.
Which of the following storage options is best for a relational database that requires consistent low-latency and high IOPS?
- A. Amazon EFS
- B. Amazon S3
- C. Amazon EBS provisioned IOPS SSD (io1/io2)
- D. Amazon Glacier
Answer: C
Explanation: Amazon EBS provisioned IOPS SSD (io1/io2) is the best option for workloads that require consistent, low-latency performance such as relational databases.
True or False: Data stored in Amazon S3 is automatically encrypted at rest using AWS managed keys by default.
- True
- False
Answer: False
Explanation: Amazon S3 data is not encrypted at rest by default; however, you can enable default encryption on a bucket so that all new objects are encrypted when being stored.
Amazon EBS allows you to create point-in-time snapshots of volumes, which are stored in:
- A. Amazon EFS
- B. Amazon EC2
- C. Amazon S3
- D. Amazon Glacier
Answer: C
Explanation: Amazon EBS allows you to create snapshots of volumes, which are stored incrementally in Amazon S3, with only the blocks that have changed after your last snapshot saving space.
True or False: Amazon S3 and Amazon EBS offer the same durability for stored data.
- True
- False
Answer: False
Explanation: Amazon S3 is designed to offer 999999999% (11 9’s) of durability, while Amazon EBS promises a yearly failure rate of 1%-2%, which is less durable than S3’s design.
Interview Questions
Can you explain the key differences between Amazon EFS, Amazon S3, and Amazon EBS in terms of their use cases and performance characteristics?
Amazon EFS is a file storage service for use with Amazon EC2 instances and other AWS services that provides a simple, scalable file storage for use with Linux-based applications. It’s suitable for applications that need a file system and can be accessed by multiple instances simultaneously. Amazon S3 is an object storage service that offers industry-leading scalability, data availability, security, and performance for use cases like data lakes, websites, mobile applications, backup and restore, archival, and big data analytics. Amazon EBS provides block-level storage volumes for use with EC2 instances, ideal for databases, file systems, or any applications that require fine granular updates and access to raw, unformatted block-level storage.
How does Amazon S3 achieve high durability and what redundancy options does it offer?
Amazon S3 achieves high durability by automatically replicating data across multiple facilities and multiple devices within each facility in an AWS Region. It offers redundancy options like Standard, which replicates data across multiple devices in a minimum of three Availability Zones, Standard-Infrequent Access, and One Zone-Infrequent Access for less frequently accessed data, and Amazon S3 Glacier for long-term archival storage.
In what scenarios would you choose to use provisioned IOPS with Amazon EBS?
Provisioned IOPS are chosen in scenarios where consistent and predictable high performance is needed, such as relational or NoSQL databases, which require fast and reliable IO operations. This is suitable for I/O-intensive workloads where performance requirements exceed the capabilities of standard EBS volumes.
How would you secure data at rest on Amazon S3?
To secure data at rest in Amazon S3, you can use S3 managed encryption keys (SSE-S3), customer managed encryption keys with S3 (SSE-C), or AWS Key Management Service managed keys (SSE-KMS). Additionally, setting up access control with bucket policies, IAM policies, and Access Control Lists (ACLs) can help further secure data.
Can you describe the process of scaling storage for an application that uses Amazon EFS?
Amazon EFS is designed to be highly scalable. It automatically scales without the need for pre-provisioning or intervention. The storage capacity grows and shrinks as files are added and removed, and you only pay for what you use. This allows applications to scale storage seamlessly as the workload demands change.
What factors should you consider when choosing between General Purpose (SSD) and Provisioned IOPS (SSD) for Amazon EBS?
When choosing between General Purpose (SSD) and Provisioned IOPS (SSD) for Amazon EBS, consider factors like I/O characteristics and performance requirements of the application. General Purpose (SSD) volumes offer a cost-effective solution that covers a broad range of workloads. Provisioned IOPS (SSD) should be chosen for workloads requiring consistent performance with more than 16,000 IOPS or more than 250 MiB/s of throughput per volume.
How would you automate the backup process for Amazon EBS volumes?
You can automate the backup process for Amazon EBS volumes by using AWS Backup or by creating a lifecycle policy in Amazon Data Lifecycle Manager to take snapshots of volumes at regular intervals. Automation can also be achieved by writing custom scripts using AWS Command Line Interface (CLI) or by using AWS SDK to take snapshots on a schedule.
What are storage classes available in Amazon S3, and how do you choose the right one?
Amazon S3 offers several storage classes including S3 Standard, S3 Intelligent-Tiering, S3 Standard-Infrequent Access (S3 Standard-IA), S3 One Zone-Infrequent Access, and Amazon S3 Glacier. The choice depends on factors such as access frequency, how long you will store the data, retrieval time requirements, and cost considerations.
Great overview of application storage patterns in AWS! Helped clear up a few confusions.
Very informative post! Thanks for sharing.
Can someone explain the difference between Amazon EBS and Amazon EFS?
What are the storage costs for Amazon S3 versus Amazon EFS?
Thanks for the detailed explanation, it was very helpful!
How reliable is Amazon EFS for running a production workload?
Excellent article, really appreciated!
Just a quick question, how does S3 bucket versioning work?