AWS Simple Storage Service – S3

S3 is an object based storage solution for AWS. You can upload an unlimited amount of objects in any format like pictures, text files, videos, or binary files. But S3 is not suitable to install an operating system or a database on it. Each file can have up to 5 TB in size, without exceptions! The organisation is in buckets, where each bucket can have a collection of files and has a unique name. To access objects inside a bucket via URL, you have to follow the following structure:

https://<bucket-name>.s3.<region>.amazonaws.com/<key-name>

Each object consists of the following parts:

• Key: object name
• Value: data itself
• Version ID: if versioning is enabled
• Metadata: like content-type, last-modified, etc.
  

Objects larger than 100 MB in size should be uploaded with multipart upload. Objects larger than 5 GB must be uploaded with multipart.

Cross-Origin Resource Sharing (CORS) defines a way for client web apps that are loaded in one domain to interact with resources in a different domain. You need to enable it to allow access to your S3 resources while avoiding the need to use a proxy. You can activate CORS in the permissions tab.

Storage Classes

• S3 Standard: Suitable for most workloads, websites, content distribution, mobile, gaming apps and big data analytics.
• S3 Intelligent – Tiering: Unknown or unpredictable access patterns. Minimum storage duration are 30 days.
• S3 Standard – Infrequent Access: Long-term, infrequently accessed critical data, backups, data store for disaster recovery files, etc. Minimum storage duration are 30 days.
• S3 One Zone – Infrequent Access: Same as before but just in one AZ.
• S3 Glacier: Long-term data archiving that occasionally needs to be accessed within a few hours or minutes. Minimum storage duration are 90 days.
• S3 Glacier Deep Archive: Rarely accessed data archiving with a default retrieval time of 12 hours, e.g., financial records for regulatory purposes. Minimum storage duration are 180 days.

Creating a S3 Bucket

Login into your AWS Console and navigate to the S3 service. Then hit the button “Create bucket”! You will face the overview page which will be described in detail here:

• Bucket names have to be globally unique and can not contain spaces or uppercases!
• Buckets are also region specific, so be aware to create them in the right one.

• ACLs enable you to manage access to buckets and objects. By default ACLs are disabled so you own every object and can control the access to it. Access control is then based on policies such as IAM policies, S3 bucket policies, VPC endpoint policies and Organisations service control policies.
• When ACLs are enabled, another account who uploads an object to your bucket, owns that object, so he has access to it and can grant other users access.

• By default S3 doesn’t allow public public access to objects. Users can modify bucket polices, access point policies or object permissions to allow public access.

• Versioning keeps multiple variants of an object in the same bucket. You can use versioning to preserve, retrieve and restore every version of every object stored in your S3 bucket. With versioning you can easily recover from both unintended user actions and application failures.

• You can set the default encryption behaviour for an S3 bucket so that all new objects are encrypted when they are stored in the bucket. Objects are encrypted using server-side encryption with wither S3 managed keys (SSE-S3) or KMS keys stored in Key Management Service.

• With Object Lock you can store objects using a write-once-read-many model, which prevents objects from deletion or overwriting. This can be set to a fixed amount of time, retention-period or indefinitely, legal hold. Object Lock works only in versioned buckets.

Protecting Data

AS with any data, it is best practice to have a backup and to put safeguards in place against malicious or accidental deletion. S3 Object Lock can be added on top of S3 Versioning to prevent data from being deleted or overwritten for a fixed amount of time or indefinitely. For creating additional copies of your data in another AWS Region, S3 Replication is the way to go. And finally you can bring visibility of your current data protection levels and the usage of these features all together into a single dashboard with S3 Storage Lens.

• To accomplish that, go ahead and create a new S3 bucket. Name it and enable versioning, encryption and object lock!
• Then upload a basic text file.
• Afterwards change the text file and upload it again.
• Inside the bucket, toggle the “Show versions” button to list your both file versions.

• With versioning enabled, deletion of a file will still preserve the previous version of an object. Keeping versions of objects will increase the costs, so lifecycle rules a good way to manage objects. You can also configure lifecycle rules to move old versions to lower cost S3 storage types like Glacier.

CloudFront

Easy and cost effective way to distribute content with low latency and high data transfer speeds for webpages and other web content.

This gets accomplished by caching the web content at an edge location, a user will then request the content at his closest edge location. If it is already stored, he just receives it from there, otherwise the edge location will forward the request to the actual hosting location and saves it for future requests.

Objects are cached for a period of time which is their Time To Live. Default is 1 day but it can be changed.

Transfer Acceleration

For transferring files from one AZ to another one. It uses edge locations to upload files at the closest location and the AWS internal network directly moves it to the target AZ.