AWS Lambda

According to AWS, Lambda is "a serverless, event-driven compute service that lets you run code for virtually any type of application or backend service without provisioning or managing servers."

While that's a mouthful of marketing jargon, what does that mean?

"Serverless" is the concept that you, as the end-user, do not need to build or deploy any infrastructure to run a Lambda function. AWS manages all that for you. AWS claims you can "Simply write and upload code as a .zip file." In reality, each Lambda is a function that runs in its own container with multiple isolation layers between AWS customers. But all of that is abstracted away from the customer by AWS.

Lambda supports a number of languages (opens in new tab), called runtime environments. The most popular are NodeJS, Python, Java, and Go. 

"Event-driven (opens in new tab)" is what happens after you upload your code. All Lambda functions are triggered by an event. That event can be a manual invocation of the function or tied to other activities in the account, such as a console login or a user writing an object to an S3 bucket. This event-driven nature allows you to create complex applications based on multiple AWS services. 

Because Lambda functions are event-driven, they have a few limitations. The most significant limitation is that a Lambda function can run for no longer than 15 minutes before it times out. 

AWS Lambda is one of the next frontiers of cloud-native applications, so it's important to know how they work, how to create them securely, and how they can be attacked. Additionally, many cloud security tools leverage Lambda to manage security events, so as a cloud security practitioner, you will need to be able to build with Lambda.

Learning Objectives

• Understand the benefits and limitations of AWS Lambda service and Lambda functions
• Learn about the various components and settings for a Lambda function
• Understand the multiple ways a developer or operator can misconfigure a Lambda function to make it less secure.

Select the Cloud details button at the top of the room:

Where needed generate the environment required for the room. The "Generate Environment" button will appear if the room contains an environment that needs to be generated. 

For any issues with the environment, select the "Reset Environment" button. Review this article for more information.

To view the credentials required for the environment, select the credentials tab. You can use these credentials to access the environment in various ways. More information can be found here:

AWS instantiates a new execution environment (opens in new tab) when a Lambda function is first invoked. This instantiation is called a cold-start and is significant for applications where response time is critical. If multiple requests come in at once, AWS will scale up the number of execution environments. Only one invocation will run in an execution environment at any one time; however, after the invocation is completed, the execution environment will be reused for subsequent requests (for a non-deterministic period, but typically between 45 minutes to an hour).

After the Lambda execution environment is instantiated, the function handler is invoked. This is where the Lambda service calls a specific function or method in the Lambda code. The inputs of the invocations are called the "event". Also passed to the handler is the "context," which contains details such as the Lambda function name, logging information, the function timeout, and time remaining before the timeout occurs.

import boto3
from botocore.exceptions import ClientError
import json
import os
import logging

# This code executes when the execution environment is created and sets up how the python logger will work
logger = logging.getLogger()
logger.setLevel(getattr(logging, os.getenv('LOG_LEVEL', default='INFO')))
logging.getLogger('botocore').setLevel(logging.WARNING)
logging.getLogger('boto3').setLevel(logging.WARNING)

# This code executes when the function is invoked
def lambda_handler(event, context):
 logger.debug("Received event: " + json.dumps(event, sort_keys=True))
 for key in os.environ.keys():
   logger.info(f"{key}={os.environ[key]}") os.environ[key]
 return(event)

The code outside the lambda_handler() is executed when Lambda creates the execution environment. In this case it's setting up the default python logger module. Each time the Lambda function is invoked, the function named lambda_handler() is called with the event provided by the caller and the context defined by the AWS Lambda service. This function then logs the event and each of the environment variables.

When a Lambda function execution completes, the object returned is passed back to the caller.

Lambda functions can be invoked synchronously or asynchronously. In a synchronous invocation, the caller waits for the function to finish executing before returning. With an asynchronous invocation, the caller sends the event but doesn't wait for the execution to complete. It's incumbent on the application designer to handle error conditions and the resulting output for these "fire and forget" invocations.

You can develop Lambda functions in the Lambda Console (opens in new tab), or via infrastructure as code like CloudFormation (opens in new tab) or Terraform (opens in new tab). When you're developing in the console, you can create test events for your functions.

You can have multiple test events and give them unique names. You can even share the test events with other developers in the same AWS account.

AWS Lambda functions have many moving parts, and it's good to know what most of them are when defending, attacking, or just building with Lambda. You can find most of these settings and components in the Lambda Console (opens in new tab), and we've created a sample function for you to play with.

Code

Lambda code is typically stored as a zip file in S3, but for smaller functions, you can often pass the zip file as part of the API call. Every code package must have a file name specified by the handler, and in that file, a function or method specified by the handler. The exact specifications for the handler are run-time dependent. 

Test events

You can configure Test Events for writing and troubleshooting code in the AWS Console. 

Memory and Timeout

You can specify the amount of memory allocated to a function: 128MB to 10GB. The Lambda service allocates CPU capacity to the execution environment based on the memory size. Additionally, you can specify the timeout of a function, up to 900 seconds. 

AWS Lambda billing uses two dimensions. The number of executions and the gigabyte-seconds of execution. A Lambda function that uses 3GB of ram and runs for 3 seconds bills at 9GB-Seconds. A 512MB Function that runs for 4 seconds will bill at 2GB-Seconds.

Runtimes

Every Lambda function has a selected runtime (opens in new tab). NodeJS and Python are the most popular runtimes, but Java, Ruby, .Net, and Go are also available. Customers can even create custom runtimes (opens in new tab). You can find the complete list of runtimes at AWS's website (opens in new tab).

Role

The Execution Role (opens in new tab) is like an IAM Instance profile. It's a set of predefined permissions the function has permission to act on resources in the AWS account. 

In AWS's least-privilege fashion, you must explicitly define the ability to write logs to CloudWatch logs in the Lambda execution role. 

Environment Variables

AWS Lambda allows configuring environment variables as part of the Lambda runtime. This allows developers to use the same code zip file across multiple environments (dev, test, prod). The same lambda:GetFunction API call that gets the code will also grant you a copy of the environment variables and values. 

Because environment variables could contain sensitive information, AWS encrypts them using Key Management Service (KMS).

Invocation Policy

In addition to the execution role, a Lambda function can have a resource-based policy that defines who or what can call the lambda:InvokeFunction call on the function. When we discussed resource policies in the IAM module, you invoked a function in the AWS account 019181489476. That was possible because the TryHackMe-time function granted InvokeFunction to all the accounts in the TryHackMe organization. 

It is possible to have a public Lambda function that anyone could execute if they knew the account id, region, and function name (the unique components of the ARN). 

Logging

The Lambda service will log any data written by the function to STDOUT (output) or STDERR (error messages) to CloudWatch Logs (opens in new tab). CloudWatch Logs is AWS's monitoring and observability service and consists of a service that can accept metrics and logs. CloudWatch also allows you to create Alarms and Dashboards about the metrics and logs.

Each function will write its logs to a log group called /aws/lambda/<functionname>. You cannot change the name of the log group, nor can you configure multiple Lambda functions to write to the same log group.

VPC

By default, AWS Lambda functions are assigned a public IP address from a dedicated pool of AWS-owned IP addresses. However, it is possible to have a function invoked inside your VPC with a private IP (RFC1918) address. This is typically required if the Lambda function must communicate with internal resources like an RDS database or a resource in an on-prem data center. 

When a function is created inside a VPC, a Hyperplane ENI (opens in new tab) is created in each specified subnet that all Lambda execution environments share. Like all ENIs, these Lambda ENIs have security groups attached to them. While Lambda execution environments usually don't listen for incoming connections, you can use the egress (outbound) rules of the Security Group to determine what resources the Lambda function can access. 

Concurrency and Throttling

Since Lambda functions are event-driven, an extremely large number of events can potentially trigger a large number of executions. In order to capacity plan and avoid runaway Lambda bills, AWS implements concurrency limits (opens in new tab) on function executions. An AWS account will support 1000 simultaneous Lambda executions in each region by default. If your application needs to scale beyond 1000 concurrent connections, you can request a limit increase with AWS Support. When the concurrency limit is reached, executions are throttled. Events are not lost but instead queued up for later execution. 

Lambda developers can also set a throttling limit on a per-function basis to not exceed the scaling capacity of resources the function relies on.

EFS

Every Lambda execution environment has 500MB of scratch space in the /tmp directory. That is the only part of the execution environment filesystem that is writable by the function code. When the execution environment is terminated, the contents of /tmp are lost. For most use-cases, that is fine. Lambda functions are intended to be stateless. 

A recent addition to Lambda is the ability to mount an Elastic FileSystem into the function. Amazon Elastic File System (opens in new tab) (EFS) is AWS's managed Network File System (NFS) service. It allows EC2 Instances, Containers, and Lambda functions to share a common filesystem. 

Lambda Layers

Often, a developer needs to share a common set of code or libraries across a number of Lambda functions. Rather than embedding the same set of code and libraries in multiple zip files across potentially hundreds or thousands of functions, AWS Lambda has the concept of a Lambda Layer. The layer is expanded and is available to the runtime when the execution environment is created. Lambda layers have resource policies so that they can be shared across accounts. Files in the Lambda layer will appear in the Lambda filesystem under the /opt directory.

Lambda Function URLs

AWS recently introduced Lambda function URLs. These are dedicated HTTPS endpoints to your function, allowing it to be invoked over the internet. The URLs are in the form of:

https://<url-id>.lambda-url.<region>.on.aws where the url-id is a unique identifier. AWS states: 

Lambda generates the <url-id> portion of the endpoint based on a number of factors, including your AWS account ID. Because this process is deterministic, it may be possible for anyone to retrieve your account ID from the <url-id>.

Function URLs can be invoked with or without IAM Authentication. If IAM Authentication is required, you must sign the request (opens in new tab) as part of the HTTP request. If no IAM Authentication is required, any simple CURL command from anywhere will invoke the function.

There are several ways Lambda can be misconfigured or attacked. The primary risk is via insecure code. Most Lambda functions execute based on an event that is user generated. If the code doesn't validate user-supplied inputs properly, a Lambda function could be compromised. 

An additional risk is when the code writes sensitive data to STDOUT. This data would appear in the CloudWatch logs for the function and be visible to anyone with read-access to the account.

Another way that Lambda functions can be misconfigured is through overly permissive execution roles—granting a function more permissions than required increases the blast radius of the damage that an attacker can do with any code vulnerability discovered.

Role Permissions are available via the environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN. If the code allowed an external attacker to access these three environment variables, they would be able to use the powers of the function to engage in potential privilege escalation or data exfiltration. 

While user-defined environment variables are KMS encrypted, they're typically easy to access from within the function or via the API or AWS Console. If environment variables are used to pass secrets to the function, those secrets are easy to exfiltrate from the account.

How the Lambda function is invoked can be a security misconfiguration if the resource policy is misconfigured. While resource policies are usually used with the lambda:invokeFunction IAM Action, if you used lambda:* instead, the principal would have the ability to upload new code. If the resource policy allowed all AWS users (i.e., Principal: *), then you would have a major security risk.

It's important to realize how Lambda concurrency works as an attacker and defender. As an attacker, you can use Lambda concurrency limits to engage in Denial of Service attacks and potentially throttle Lambda functions used to respond to security events. As a defender or author of a function responding to the security events, understanding throttling and implementing some reserved concurrency can mitigate the potential damage from a Lambda denial of service event.

Modifying a Lambda layer is a great way to inject malicious code into a large number of functions.

Finally, when you call the lambda:GetFunction API, part of the response is a pre-signed URL to download the Code zip file from the Lambda service's S3 bucket. This is a useful discovery tactic, because developers often hardcode things in their applications where they shouldn't. 

In the next room, you'll get a chance to use many of these techniques to exfiltrate sensitive data using misconfigured Lambda functions.