AWS S3 Bucket Creation Date Discrepancy in master and other regions

AWS S3 buckets are among the most commonly used AWS resource. At the same time, S3 buckets are most prone to misconfigurations. All major famous leaks have generally happened with S3 bucket misconfigurations. For this reason, the cloud teams continuously keep track of S3 bucket configurations.

The AWS resource inventory is a common way of keeping track of the AWS resources. The OSS projects such as CloudQuery, Steampipe, and Scout Suite are used by many cloud teams for this purpose.

The AWS resources are either global or region-wise. Services such as IAM are completely global, while resources such as EC2 are regional. The S3 bucket is one weird creature with a global namespace, even spanning beyond the usual AWS account bubble. At the same time, S3 buckets are associated with regions. S3 buckets are created in a region. This is typically done to reduce latency, minimize costs, or address regulatory requirements.

We collect the AWS resources information with AWS service APIs such as Describe/Get/List. S3 buckets are listed with the list-buckets API. Since S3 buckets are region-specific, we would call this API region-wise. And this is when we noticed one interesting discrepancy in S3 bucket creation time information.

The same bucket amplify-testfigma-staging-164952-deployment shows a different creation time.

More digging into the issue took us to the AWS documentation for this API.

CreationDate -> (timestamp)

Date the bucket was created. This date can change when making changes to your bucket, such as editing its bucket policy.

After some more digging, we stumbled upon the AWS CLI issue reported in GitHub and Mark Sayson’s blog post. An AWS support engineer wrote:

This is expected behavior due to the design of the service. The GET Service call in S3 (s3api list-buckets or s3 ls with no further arguments in the CLI) works differently when being run against different regions. All bucket creations are mastered in us-east-1, then replicated on a global scale - the resulting difference is that there are no "replication" events to the us-east-1 region. The Date Createdfield displayed in the web console is according to the actual creation date registered in us-east-1, while the AWS CLI and SDKs will display the creation date depending on the specified region (or the default region set in your configuration).

When using an endpoint other than us-east-1, the CreationDate you receive is actually the last modified time according to the bucket's last replication time in this region. This date can change when making changes to your bucket, such as editing its bucket policy.

The S3 global namespace is confined to the AWS partitions. This is also due to the fact that each partition has its own designated master region. Mark Sayson confirmed in his blog post his findings.

• “us-east-1” for the Classic “aws” partition
• “cn-north-1” for the AWS China “aws-cn” partition
• “us-gov-west-1” for the AWS GovCloud “aws-us-gov” partition

Now this is what we call fun of working on the AWS resource inventory service and discovering interesting tidbits.