Amazon EC2 - Attack & Defense
Learn about EC2 and how to compromise an EC2 Instance.
medium
60 min
346
To access material, start machines and answer questions login.
Amazon Elastic Compute Cloud () is ’s core computing service. It provides virtual machines, called instances, running an operating system of your choice. Every Instance resides in a private network called a Virtual Private Cloud () (covered in the next room).
meets all of the attributes of cloud computing as defined by . Instances are created via the and take several seconds to a few minutes to fully provision. They run on multi-tenant hardware and, in most cases, can spin up and spin down at a moment’s notice. They support per-second billing providing high-resolution measured service. With , you only pay for computing resources when you need them. instances can be deployed into 26 different geographical regions (opens in new tab) (as of Jul 2022) across 19 countries and six continents.
Instances deploy as an Instance Type. The Instance Type defines the Memory, Architecture (Intel, AMD, ARM), and Network Capacity in combinations optimized for different use-cases. As of July 2022, more than 500 different instance types are available. (opens in new tab)
Amazon supports multiple billing models. On-Demand instances are the most common; you pay for what you use and don’t pay for what you don’t use. When workloads are more predictable, customers can purchase Reserved Instances in 1 - 3 year terms to save money. If you have a workload that can easily recover from an interruption, you can deploy your workload as Spot Instances and pay below-market rates for ’s unused capacity.
supports , Windows, and even MacOS as operating systems.
If you haven't already done so, click the orange Cloud Details button at the top-right of the page, generate the Cloud Environment, and use the Credentials provided to access your TryHackMe Cloud ( console).
Select the Cloud details button at the top of the room:
Where needed generate the environment required for the room. The "Generate Environment" button will appear if the room contains an environment that needs to be generated.
For any issues with the environment, select the "Reset Environment" button. Review this article for more information.
To view the credentials required for the environment, select the credentials tab. You can use these credentials to access the environment in various ways. More information can be found here:
Let us walk through creating a new Instance via the Console.
- Visit the Instances Page (opens in new tab)
- Click on the Orange "Launch Instances" box in the upper right. You'll be taken to a screen to create your Instance.
- For Name and tags, enter "My First Instance"
- For Application and Images (Amazon Machine Image), first, select "64-bit (Arm)", then make sure "Amazon " is highlighted.
- Under Instance type, make sure to select "t4g.nano".
- Since this is the first time you're launching an instance in this account, you will need to create a new key pair. Click the blue "Create new key pair" link and complete the dialog. You can name your key pair anything. Select pem or ppk, depending on your preference.
- Make sure the key pair name matches the key pair you just created
- Under Network settings, ensure "Allow traffic from" has "My IP" selected. Do not allow a security group to be open from "Anywhere 0.0.0.0/0"
- Expand out Advanced Details
- For the instance profile, make sure "Ec2RoomInstanceProfile" is selected.
- You can leave the remaining Advanced Details as default.
- Review the Summary on the right
- Now click "Launch Instance." will begin creating your Instance.
This task will discuss the number of ways you can connect to your new Instance for administrative purposes.
and
The original method for accessing an Instance was over the network via either for or for Windows. This method would leverage the key pair you created in the previous task. You would need to have a network path to the machine for this to work. If the machine follows best practices, it is in a private subnet without a public IP Address. Either a publicly-accessible jump host or some form of would be required to connect this way.
With Windows instances, the Key is required for you to decrypt the Administrator Password, which is then used for authentication using Remote Desktop.
Connect
While there are many ways you can connect to an Instance, we will be using a newer service from called Instance Connect (opens in new tab) for all the remaining labs. Instance Connect provides a browser-based interface to instances. It leverages temporary public/private key pairs deployed by an agent leveraging Permissions.
The Instance named "TryHackMe Demo Instance" is set up to work with Connect. To connect to this machine, go to the Console (opens in new tab), and select the Instance.
Click "Connect" in the upper right.
You will want to use the defaults in most cases, but you can also directly log in as root. Once you hit "Connect", The Web Console will present you with a web-based Console like the following:
Generally, the URL for Instance Connect is of the form:
https://console..amazon.com//v2/connect/$USERNAME/$INSTANCE_ID
SSM
SSM used to stand for Simple Systems Manager, but they dropped "Simple" as they added more and more capabilities to this service. Now it's called Systems Manager but still abbreviated as SSM.
One of the system management capabilities in SSM is the ability to install a package or run a command on a or Windows server. SSM Requires an Agent to be installed on the host , and the Instance must have the correct permissions to communicate with the SSM Service.
In addition to running a command or installing a package, SSM has a third capability called Systems Manager Session Manager (opens in new tab). What a mouthful that name is!
Serial Console
A new feature to is the Serial Console. An account administrator must enable the Serial Console for the region before anyone can use it. Additionally, when using the Serial Console, you will receive a Login prompt. Unless the Instance has a user that has been configured with a password in /etc/shadow, you cannot log in to a machine via the Serial Console.
Summary of the major ways to connect to an Instance
| Method | Requires Direct Network Access | Requires Agent | Requires Permissions |
| Direct / | YES | NO | NO |
| Instance Connect | YES | YES (Installed on amazon linux2) | NO |
| SSM Run Command | NO | YES | YES |
| SSM Session Manager | NO | YES | YES |
| Serial Console | NO | NO, but users must have a password set | NO |
Connect to your EC2 Instance via Instance Connect. Run the id command, and answer what user you're logged in as:
Now connect via SSM Session Manager. Run the id command. What user are you connected as?
Using either the Instance Connect or SSM Session Manager session, run "sudo passwd root" and set a password for the root user. Can you now log in with the EC2 Serial Console? (Yea or Nay)
As we discussed in the Room, Instances have their own way to get Credentials in order to interact with other resources: the Instance MetaDataService (IMDS). As mentioned in the previous task, one of the services an instance can interact with is SSM, but an Instance can access pretty much any resource if granted permissions.
The act of getting Credentials from within the instances leverages the Instance Metadata Service or IMDS. The IMDS listens on the special IP address 169.254.169.254, part of the Hypervisor.
IMDS was vulnerable to several server-side request forgery and attacks and was a primary culprit in the Capital One breach. In response, released IMDSv2 in late 2019. IMDSv2 leverages a session-based approach to interactions with the Metadata service and requires both an PUT and an GET, making an attack like Capital One harder.
These next few commands will show you how to leverage the Instance Metadata Service for both v1 and v2. Login to the TryHackMe Demo Instance.
In order to retrieve credentials from the /security-credentials/ endpoint, you must specify the role name. conveniently provides that via another call to /security-credentials/. Run these two commands to get the role name, store it in the environment variable role_name, and echo it out to you so you can answer the next question:
user@machine$ role_name=$( curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/ )
echo "Role Name is $role_name"
Please scroll down and answer question 1 of this task.
Now that you have the role name, you can ask for the session credentials for the role. That's a simple curl command:
user@machine$ curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/${role_name}
{
"Code" : "Success",
"LastUpdated" : "2022-03-13T12:59:08Z",
"Type" : "ANSWER TO QUESTION 2",
"AccessKeyId" : "ASIAZOHYLBSNABQBBXWK",
"SecretAccessKey" : "40CharacterSecretRedacted",
"Token" : "IQoJb3JpZ2luX2VjEM3//////////REDACTEDw==",
"Expiration" : "2022-03-13T19:15:53Z"
}
Please answer question 2 of this task.
Now we should enable IMDSv2. Enabling IMDSv2 can be done via the instance you're connected to. The following three commands will:
- Get the Instance Id
- Echo the Instance ID value back to you
- Update the Instance Metadata options to require a Token to make the instance metadata call
user@machine$ instance_id=$( curl -s http://169.254.169.254/latest/meta-data/instance-id )
user@machine$ echo "My Instance ID is $instance_id"
user@machine$ aws ec2 modify-instance-metadata-options --instance-id $instance_id --http-tokens required --region us-east-1
{
"InstanceId": "i-04ffb7fc50f08378c",
"InstanceMetadataOptions": {
"State": "pending",
"HttpEndpoint": "enabled",
"HttpTokens": "required",
"HttpPutResponseHopLimit": 1
}
}
The final command is an authenticated command to the Service that will enable IMDSv2 by setting the -tokens required field to true.
Here is how you can get credentials via the Token method. First, you need to get the token:
user@machine$ TOKEN=`curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
user@machine$ echo $TOKEN
In the above command you've requested a token that will last 21,600 seconds or 6 hours.
Please use the output of the above command to answer question 3 of this task.
Now you can get credentials.
user@machine$ role_name=$( curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/iam/security-credentials/ )
user@machine$ echo "Role Name is $role_name"
user@machine$ curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/iam/security-credentials/${role_name}
-H "X-aws-ec2-metadata-token: $TOKEN" flag to curl adds a custom header to your call. This makes it much more difficult to leverage standard exploits to access and exfiltrate credentials via the IMDS.
What was the "Type" of the Credentials returned?
What are the first four characters of the IMDSv2 token returned?
What HTTP Error Code do you get if you attempt to use the metadata service without the token?
In addition to specifying the Instance Type, two configurable aspects of are Networking and Storage.
Networking
Networking revolves around the Elastic Network Interface (ENI). All Instances have at least one ENI. However, an Instance can have multiple ENIs to act as a network device between two subnets, for additional network capacity, or as part of a failover capability.
Every ENI has at least one Security Group attached to it. The security group defines what IP source address, ports, and protocols are permitted to communicate with the ENI. We will cover Security Groups in more detail when we cover VPCs in the next room.
As part of 's Network Security design, Instances can only send or receive traffic from or destined to the Instance. For an Instance to act as a NAT, , or router, you must disable this default source/destination check.
Start the CloudShell from the Management Console or this link (opens in new tab) and let's look at one of the ENIs in your account:
[cloudshell-user@ip-10-1-29-54 ~]$ aws ec2 describe-network-interfaces | jq '.NetworkInterfaces[0]'
{
"Association": {
"IpOwnerId": "XXX",
"PublicDnsName": "ec2-35-170-203-245.compute-1.amazonaws.com",
"PublicIp": "35.170.203.245"
},
"Attachment": {
"AttachTime": "2022-04-17T10:09:42+00:00",
"AttachmentId": "eni-attach-0b34028068c1e73c7",
"DeleteOnTermination": true,
"DeviceIndex": 0,
"NetworkCardIndex": 0,
"InstanceId": "i-050a1b2abfb4b9250",
"InstanceOwnerId": "179982773718",
"Status": "attached"
},
"AvailabilityZone": "us-east-1a",
"Description": "",
"Groups": [
{
"GroupName": "launch-wizard-1",
"GroupId": "sg-0b367684ea92c5622"
}
],
"InterfaceType": "interface",
"Ipv6Addresses": [],
"MacAddress": "12:0d:b9:6c:53:3b",
"NetworkInterfaceId": "eni-077e778d991378e0f",
"OwnerId": "179982773718",
"PrivateDnsName": "ip-172-31-91-60.ec2.internal",
"PrivateIpAddress": "172.31.91.60",
"PrivateIpAddresses": [
{
"Association": {
"IpOwnerId": "XXX",
"PublicDnsName": "ec2-35-170-203-245.compute-1.amazonaws.com",
"PublicIp": "35.170.203.245"
},
"Primary": true,
"PrivateDnsName": "ip-172-31-91-60.ec2.internal",
"PrivateIpAddress": "172.31.91.60"
}
],
"RequesterManaged": false,
"SourceDestCheck": true,
"Status": "in-use",
"SubnetId": "subnet-0cc7231ef015e3cd2",
"TagSet": [],
"VpcId": "vpc-0dbca4684636f42a3"
}
This output provides a wealth of helpful information if you're trying to understand the network topology of a target's cloud infrastructure. You get the public IP, Private IP, and Instance ID, but you also get the name and ID of the Security Group, Subnet, and (which we will discuss more in the Room).
Please answer question one with the value of the PrivateIpAddresses.Association.IpOwnerId value provided in the command output.
Storage
Amazon uses Amazon Elastic Block Store (EBS) as the hard disk storage for Instances. These disks are called EBS Volumes and support multiple types based on the workload. EBS Supports SSD, HDD (i.e., spinning platters), and provisioned IOPS (for high performance and mission-critical workloads) in capacities ranging from 1GB to 64TB.
EBS Volumes can be snapshotted for backup and sharing purposes. These EBS Snapshots are stored in Amazon S3 and thus cost much less than an EBS Volume.
Certain Instance Types also have ephemeral storage. These SSD drives reside inside the physical machine running the Instance. Unlike EBS volumes which are kept separate from the compute workload for durability, ephemeral storage is erased when an instance is stopped and restarted. Ephemeral storage does not support snapshots.
EBS Volumes and Snapshots can be encrypted using different types of KMS Encryption (which we will cover in a future room). EBS Snapshots can also be shared across Accounts, providing a method of sharing organizationally approved golden images and accidentally sharing data, source code, or secrets with the world.
One way to gain access to valuable data is by leveraging EBS Snapshots. In this exercise, we will create a volume from a public snapshot, mount that volume onto our Instance, then read the data from that snapshot.
First, let's look at the snapshot we've created for this exercise:
[cloudshell-user@ip-10-1-29-54 ~]$ aws ec2 describe-snapshots --snapshot-ids snap-051a9c27b12a797a4
{
"Snapshots": [
{
"Description": "",
"Encrypted": false,
"OwnerId": "019181489476",
"Progress": "100%",
"SnapshotId": "snap-051a9c27b12a797a4",
"StartTime": "2022-04-17T16:19:28.473000+00:00",
"State": "completed",
"VolumeId": "vol-ffffffff",
"VolumeSize": 1,
"StorageTier": "standard"
}
]
}
This tells us a bit of what we'll need to do. We can see the volume isn't encrypted. That will make our job a bit easier. Based on VolumeSize, we know it's a 1GB volume. We also know the account that created this volume is 019181489476.
The first thing we need to do is create a volume from this snapshot. Important note - Snapshots exist in an Region, but a Volume must be in the same Availability Zone (AZ) as the machine you will mount it from. We can get the AZ of your Instance from the metadata service.
[ec2-user@ip-172-31-89-198 ~]$ TOKEN=`curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
[ec2-user@ip-172-31-89-198 ~]$ curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/availability-zone
us-east-1c
In my case, it's us-east-1c.
Now we create a volume from the snapshot. We'll specify the gp3 volume type (because it's cheaper). Note: make sure you select the AZ from your Instance.
[ec2-user@ip-172-31-89-198 ~]$ aws ec2 create-volume --snapshot-id snap-051a9c27b12a797a4 --volume-type gp3 --region us-east-1 --availability-zone REPLACE-ME-WITH-YOUR-AZ
{
"AvailabilityZone": "us-east-1c",
"MultiAttachEnabled": false,
"Tags": [],
"Encrypted": false,
"VolumeType": "gp3",
"VolumeId": "vol-0ee52dc1a5b2b7bc5",
"State": "creating",
"Iops": 3000,
"SnapshotId": "snap-051a9c27b12a797a4",
"CreateTime": "2022-03-19T20:21:04.000Z",
"Size": 1
}
Note the volume ID in the above output. We need that to attach the volume to our Instance.
[ec2-user@ip-172-31-89-198 ~]$ instance_id=$( curl -H "X-aws-ec2-metadata-token: $TOKEN" -s http://169.254.169.254/latest/meta-data/instance-id )
[ec2-user@ip-172-31-89-198 ~]$ aws ec2 attach-volume --region us-east-1 --device /dev/sdh --instance-id $instance_id --volume-id vol-REPLACE-ME-WITH-YOUR-VolumeId
{
"AttachTime": "2022-03-19T20:25:46.448Z",
"InstanceId": "i-0f08e464c321caad2",
"VolumeId": "vol-0ee52dc1a5b2b7bc5",
"State": "attaching",
"Device": "/dev/sdh"
}
The first command grabs the instance-id from the IMDS for use in the second command. Once the second command is executed, it won't take long for the attachment to happen. If you run fdisk -l, you should see a 1GB volume.
[ec2-user@ip-172-31-89-198 ~]$ sudo fdisk -l
Disk /dev/nvme0n1: 8 GiB, 8589934592 bytes, 16777216 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: CB18A911-A22A-48F1-A95C-932F60C80EA4
Device Start End Sectors Size Type
/dev/nvme0n1p1 22528 16777182 16754655 8G Linux filesystem
/dev/nvme0n1p128 2048 22527 20480 10M EFI System
Partition table entries are not in disk order.
Disk /dev/nvme1n1: 1 GiB, 1073741824 bytes, 2097152 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Now we need to create a mount point, mount the disk, view the contents of the disk, and discover the flag:
[ec2-user@ip-172-31-89-198 ~]$ sudo mkdir /snapshot-recovery
[ec2-user@ip-172-31-89-198 ~]$ sudo mount /dev/nvme1n1 /snapshot-recovery
[ec2-user@ip-172-31-89-198 ~]$ ls /snapshot-recovery
[ec2-user@ip-172-31-89-198 ~]$ cat /snapshot-recovery/flag.txt
What CTO is quoted in the flag you found on the volume?
As you saw when you created your first instance, Instances are launched from an or Amazon Machine Image. Most times, you launch an instance from an -provided . However, you can custom "bake" your own AMIs - adding your customizations to a base . These custom AMIs are often used to launch instances in an AutoScaling group - a way of quickly scaling your application up and down according to customer demand.
The final source of AMIs is the MarketPlace - here, you can find AMIs from commercial vendors. The marketplace can contain AMIs that are Open Source (no license costs), BYOL (bring your own license), or billed by via Marketplace charges.
You can see the AMIs in your account with this command (Give it a try in your CloudShell (opens in new tab)):
aws ec2 describe-images --owners 019181489476If you omit the --owners flag, you will see all the public images available in that region. This includes marketplace images, images from amazon, and images shared by other AWS accounts. Warning: there are a lot of public images and this command will most like timeout. The TryHackme AWS Development account is 019181489476 and is how we share images out to the TryHackMe sandbox accounts.
One of the ways an EC2 Instance can be configured at launch time is via UserData. UserData is part of the cloud-Init (opens in new tab) package that several public and private cloud providers use. If the UserData begins with "#!/bin/bash", the UserData is considered a script and will be executed by cloud-init.
UserData allows cloud administrators to configure their machines at creation with environment-specific settings ( is the machine part of Dev, Staging, or Production). Two important things to know about UserData:
- Because it is used for configuration, it often contains secrets or other configuration information useful for a penetration tester or attacker.
- It is executed on boot when it is changed. This means if you have permission to modify the UserData script, you can execute arbitrary commands on the instance when it's rebooted.
There are two ways to get the user-data. If you're on the machine (via Instance Connect or the SSM Session Manager console), you can do a curl for it:
curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/user-dataor look in /var/lib/cloud/instance/scripts/part-001 - That requires using su or sudo to read the file as root.
Connect to the TryHackMe Demo Instance via EC2 Connect and run the following
[ec2-user@ip-172-31-89-198 ~]$ TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
[ec2-user@ip-172-31-89-198 ~]$ instance_id=$( curl -H "X-aws-ec2-metadata-token: $TOKEN" -s http://169.254.169.254/latest/meta-data/instance-id )
[ec2-user@ip-172-31-89-198 ~]$ aws ec2 describe-instance-attribute --attribute userData --instance-id $instance_id --region us-east-1 --query UserData --output text | base64 -d
--query UserData), and then pipes that through a base64 decode to make the shell script readable. What is the password that is written to the file in /etc?
Most web applications in the cloud are configured to "auto scale". The application will spin up and down based on demand. This demand-based deployment is one of the primary ways companies save money in the public cloud. Rather than purchase hardware for peak demand, they leverage the provider's excess capacity during peak usage.
There are three primary components involved in Auto Scaling. They are:
- Elastic Load Balancers (opens in new tab) (ELBs) that broker requests across N-number of instances. These load balancers are often the trigger to spin up or spin down capacity.
- Launch Configurations are the instructions for creating a new instance. The Launch Config contains the , instance type, and other critical configurations.
- The AutoScaling Group (ASG) is the resource responsible for responding to capacity alerts and creating or terminating instances.
From a security perspective, there are a few things to note about Auto Scaling:
- When ASGs scale down, they scale down the oldest instance in the fleet. This means that the machine with the most exposure to untrusted traffic is the first to be terminated.
- Only the Load Balancer needs to be publicly exposed. The ELBs take the external traffic and then pass it along to the fleet using private addressing. Therefore the fleet doesn't need to have public addresses.
- Depending on the type of Load Balancer, it can terminate encrypted connections using the free Certificate Management (ACM) system. The second generation of ELBs, known as Application Load Balancers or ALBs, can have an Web Application (WAF) attached. This WAF can detect or block common layer-7 (application layer) attacks like those defined by (opens in new tab).
Ready to learn Cyber Security?
TryHackMe provides free online cyber security training to secure jobs & upskill through a fun, interactive learning environment.
Already have an account? Log in