/ cloud

AWS Security Overview - S3 Data Exposure

Overview

With the recent news about the, Pentagon exposing some of its data on Amazon server I figured I would write a quick post on S3 and recreate a similar misconfiguration.

NOTE: I have NO knowledge of this data exposure.  I do not know if the below is what caused the data exposure.  

Again, I do not know if this is what lead to the data exposure.  This is one of multiple ways it could have happened.

UpGuard Article

UpGuard, the organization that released the exposure, goes on to say in the article, "On September 6th, 2017, UpGuard Director of Cyber Risk Research Chris Vickery discovered three Amazon Web Services S3 cloud storage buckets configured to allow any AWS global authenticated user to browse and download the contents".

AuthenticatedUsers

Based on the comment above it's possible the S3 buckets were configured to allow; the AuthenticatedUsers group via ACLs.

AuthenticatedUsers is an Amazon S3, predefined group, which you can assign access via a S3 ACL by specifying the URI (http://acs.amazonaws.com/groups/global/AuthenticatedUsers) instead of a canonical user ID.

The Authenticated Users group represents all authenticated users within AWS. This group account allows ANY AWS account to access your resources, not just the authenticated IAM users within your AWS account.

There is a clear warning in the S3 Developer documentation. It says, "When you grant access to the Authenticated Users group any AWS authenticated user in the world can access your resource."

Maybe they didn't read the documentation clearly? It is also possible the admins saw the words, "AWS authenticated user" and assume it's just their AWS authenticated users and not every AWS Authenticated user? Or maybe it was something else...

It's not something you can screwup bia a typo, it's not a sophisticated attack, no one circumvented AWS security protections (based on the information from the article). It appears to have been a misconfiguration on the part of the cloud admins (or someone in a similar role).

In either case, this stuff happens. That's life. I decided to try and recreate a similar misconfiguration as I wasn't aware of this group and level of access prior to the article.

I am not a AWS Cloud Architect or Cloud SysAdmin so take the below with a grain of salt.

My Test Bucket

I created a test bucket called, fasdfgsgdfagdaggb-test using all of the default options.

When you create a bucket or an object, there is a default access control list (ACL) created that grants the resource owner full control over the resource.

ACLs enable you to manage access to both buckets and objects. They define which AWS accounts or groups are granted access and the type of access they have.

After creating the bucket I wanted to see the ACLs for the new bucket so we had a before and after.

To see what bucket ACLs are assigned to a bucket, use the following command:

aws s3api get-bucket-acl --bucket fasdfgsgdfagdaggb-test

You can see here the Grantee, po, has FULL_CONTROL over the bucket. As mentioned above, when you create a bucket, AWS grants the resource owner full control over the resource.

An ACL can have up to 100 grants.

{
    "Owner": {
        "DisplayName": "po",
        "ID": "0fb<snip>dad"
    },
    "Grants": [
        {
            "Grantee": {
                "Type": "CanonicalUser",
                "DisplayName": "po",
                "ID": "0fb<snip>dad"
            },
            "Permission": "FULL_CONTROL"
        }
    ]
}

AuthenticatedUsers Example

If we try and list the contents of that bucket using Account B (s3test) we are unable to do so.

aws s3 ls s3://fasdfgsgdfagdaggb-test --profile s3test

You can see here when we attempt to ListObjects it's denied.

An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied

Now we are going to assign an ACL and grant AuthenticatedUsers read access to our, fasdfgsgdfagdaggb-test bucket.

aws s3api put-bucket-acl --bucket fasdfgsgdfagdaggb-test --grant-read uri=http://acs.amazonaws.com/groups/global/AuthenticatedUsers

Now we want to get the bucket ACL again and compare it with the before get-bucket-acl.

You can see we now have two grants. Our new one, Type: Group and the URI pointing to, AuthenticatedUsers.

aws s3api get-bucket-acl --bucket fasdfgsgdfagdaggb-test

A Grantee can be an AWS account or one of the predefined Amazon S3 groups. The three predefined groups are:

  • Authenticated Users group
  • All Users group
  • Log Delivery group
{
    "Owner": {
        "DisplayName": "po",
        "ID": "0fb<snip>dad"
    },
    "Grants": [
        {
            "Grantee": {
                "Type": "Group",
                "URI": "http://acs.amazonaws.com/groups/global/AuthenticatedUsers"
            },
            "Permission": "READ"
        },
        {
            "Grantee": {
                "Type": "CanonicalUser",
                "DisplayName": "po",
                "ID": "0fb<snip>dad"
            },
            "Permission": "FULL_CONTROL"
        }
    ]
}

You can also verify this within the AWS console. After we added those permissions, we now can see that a Group, Any AWS user was created and given List Objects of, Yes.

1

AuthenticatedUsers - Read Only

To test the AWS AuthenticatedUsers access I setup a profile using my second AWS Account (Account B). The user for the second AWS account is, s3test.

You can see here that s3test is now able to list the items within this S3 bucket from AWS Account A, where as before this ACL was granted they could not.

aws s3 ls s3://fasdfgsgdfagdaggb-test --profile s3test

                           PRE NotEncrypted/
                           PRE SSE-S3Encryption/

We can also do a recursive list and see everything in this sample S3 bucket.

aws s3 ls s3://fasdfgsgdfagdaggb-test --recursive --profile s3test

2017-11-18 19:01:24          0 NotEncrypted/
2017-11-18 19:02:20         12 NotEncrypted/Test.txt
2017-11-18 19:01:37          0 SSE-S3Encryption/
2017-11-18 19:02:05         12 SSE-S3Encryption/Test.txt

Now let's try to download them.

As you can see, the GetObject operation is denied here. So we are unable to download these files even though we can list them.

The reason for this is we haven't set a bucket policy and/or an object ACL giving us permissions to do so.

aws s3 cp s3://fasdfgsgdfagdaggb-test . --recursive --profile s3test

download failed: s3://fasdfgsgdfagdaggb-test/SSE-S3Encryption/Test.txt to SSE-S3Encryption\Test.txt An error occurred (AccessDenied) when calling the GetObject operation: Access Denied
download failed: s3://fasdfgsgdfagdaggb-test/NotEncrypted/Test.txt to NotEncrypted\Test.txt An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

Bucket Policies and Object Access

I'm not sure what their bucket policy was set for obviously so I will just add a simple object acl to show you how it's possible to download the files.

Initially I set the put-object-acl only for my avatar.png file inside of People.

aws s3api put-object-acl --bucket fasdfgsgdfagdaggb-test --key People/avatar.png --acl authenticated-read

Then I attempt to copy all of the files again.

aws s3 cp s3://fasdfgsgdfagdaggb-test . --recursive --profile s3test

You can see here that two of them were denied because I had not setup object ACLs on them, and then the avatar.png downloaded successfully.

download failed: s3://<snip>iam-ug.pdf An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

download failed: s3://<snip>iam-api.pdf An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

download: s3://fasdfgsgdfagdaggb-test/People/avatar.png to People\avatar.png

At the same time, if you were to navigate to this via the web you would get denied.

2

Summary

I would suspect the misconfiguration was similar in nature to the above; however, I really have no idea.

Even if it isn't exactly the same, it highlights a similar situation of where poor Bucket and Object ACLs can lead you down a dark path.

If you're using ACLs to manage access to S3 you should move away from it unless you have a snowflake case why you need it. It's recommended by AWS that, bucket policies and/or IAM policies be used for S3 access control.

The article mentions that some of the web scraping occurred back in 2009, but the most recent was 2017. Back in 2009 (I wasn't an AWS user back then), from what I gather (looking at older screen shots and what not), it was easier to misconfigure permissions. It appears at one time you could configure ACLs permissions in the GUI where now you cannot. Maybe this is what lead to the misconfigurations? Who knows. If so, maybe it's time go back back and revisit your permissions on your S3 buckets.

I would also suggest not using company identifiable information in your bucket names to the extent possible. There are scripts that will brute force company names in an attempt to find company named s3 buckets.

Object access logging could have helped to identify any brute force successful downloading of objects within a particular bucket.

I have been writing a series on AWS Security here

If you find this post useful and educational you can donate via PayPal.Me here: $1, $2, $3, $4, $5 or Custom.

Enjoy!

References

S3 Developer Guide
S3 API Documentation