Problem statement

You are primarily billed based on how long your Lambda functions run and how often it is invoked. This is why AWS Lambda is great for scheduled jobs, short-duration tasks, and event-based processes. But does this mean you shouldn’t use them for high-volume traffic applications? Well, the short answer is… it depends.

Whether or not to use AWS Lambda for high-volume traffic workloads ultimately depends on the requirements of your application and the cost trade-off you’re willing to make. Regardless, if you ever find yourself wanting to use Lambda in a high-activity application like stream processing, it’s good to know that there are methods available to offset the cost of running functions.

Event Filtering to the rescue 

AWS Lambda’s event filtering enables you to select specific events from an event source to trigger your Lambda function. With event filtering, you can filter out events that are irrelevant to your application, reducing the number of unnecessary invocations and minimizing the operating cost of your Lambda functions. Event filtering is an optional feature of event source mapping — an AWS Lambda resource that reads from an event source (e.g., DynamoDB streams, SQS queues, Kinesis Data Streams) and invokes a function.

Scenario

Let’s say you need to write a Lambda function that reacts to voltage drops and temperature changes in a site in order to detect faulty components. To do this, multiple sensors send readings to a Kinesis Data stream. In turn, the Lambda function must consume and process these readings to determine if any issues are present.

Your initial thought might be to include a filtering logic within the function code, like the one below:

def lambda_handler(event, context):
    temperature = event["temperature"]
    voltage = event["voltage"]
    if temperature > 100 or voltage < 5:
        #do something
    else:
         #do nothing

While it may seem reasonable, it is inefficient from a cost standpoint. This is due to the fact that the Lambda function will be triggered needlessly even if sensor data doesn’t require any action. For example, a batch size of 1 and batching window of 0 can result in 60 invocations per minute, given that sensor data is sent every second. If none of the readings indicate a failure, you’d still end up paying for those 60 invocations. Imagine if this goes on for a week or even months!

Rather than doing conditional checks for each event at the function level, you can filter the events before they are passed down to your function. Going back to the scenario, since you’re only interested in specific voltage and temperature values, you can specify a filter pattern in the filter-criteria option of your event source mapping.

A single event source can have up to five unique filters (you may refer to this link for the full list of filter syntax). If an event matches any of the filters, Lambda routes it to your function. Otherwise, the event is discarded.

You can configure event filtering on either a new event source or an existing one. This can be done in the AWS Lambda Console or programmatically using the CreateEventSourceMapping or UpdateEventSourceMapping APIs. 

In part 2 of this article, I’ll walk you through a simple demo of using Event filtering in AWS Lambda.

To learn more about AWS Lambda, check out this FREE AWS Lambda Foundations course. And if you’re preparing for an AWS Certification, our high-quality practice exam samplers can help you get ready for test day. Try them out for FREE to get a feel for the exam format and level of difficulty.