Lambda Fan-Out: Parallel Processing with SNS and SQS (2026)

Lambda functions are often used for event-driven architectures, but sometimes a single event needs to trigger multiple Lambda functions. This is where a fan-out pattern comes in. The most surprising thing about this pattern is that it’s not just about parallelism; it’s about * decoupling* and * resilience* as much as speed.

Imagine an e-commerce order. When an order is placed, several things need to happen: inventory must be updated, a shipping label generated, an email confirmation sent, and analytics data recorded. If these were all sequential calls from a single Lambda, a failure in any one step would halt the entire process, and the order might not be fully processed. Moreover, if one of these downstream services is slow, it holds up the entire request.

Here’s how a fan-out pattern using SNS and SQS addresses this:

1. The Trigger: An event (like an OrderCreated message) lands in an Amazon Simple Notification Service (SNS) topic. This is the central hub.

2. The Fan-Out: The SNS topic has multiple subscriptions, each pointing to a different downstream service. In our example, we’d have subscriptions for:

   • An SQS queue for inventory updates.
   • An SQS queue for shipping label generation.
   • An SQS queue for email confirmations.
   • An SQS queue for analytics.

3. The Parallel Processing: Each SQS queue is configured to trigger a separate AWS Lambda function. When a message arrives in an SQS queue, its corresponding Lambda function is invoked. Because each Lambda is triggered by its own queue, they can all run concurrently. If the email service is slow, it doesn’t affect the inventory update or shipping label generation.

Let’s look at the configuration.

First, the SNS topic. You’d create one, let’s call it OrderProcessingTopic.

aws sns create-topic --name OrderProcessingTopic

Next, you’d create SQS queues for each downstream process.

aws sqs create-queue --queue-name InventoryUpdateQueue
aws sqs create-queue --queue-name ShippingLabelQueue
aws sqs create-queue --queue-name EmailConfirmationQueue
aws sqs create-queue --queue-name AnalyticsQueue

Now, subscribe each SQS queue to the SNS topic. This is the crucial fan-out step. For each queue, you need its ARN. You can get this with aws sqs get-queue-attributes --queue-url <QUEUE_URL> --attribute-names QueueArn.

For example, subscribing InventoryUpdateQueue:

aws sns subscribe \
    --topic-arn arn:aws:sns:us-east-1:123456789012:OrderProcessingTopic \
    --protocol sqs \
    --notification-endpoint arn:aws:sqs:us-east-1:123456789012:InventoryUpdateQueue

You’d repeat this for all your SQS queues.

Finally, you configure each Lambda function to be triggered by its respective SQS queue. In the Lambda console, when setting up a trigger, you select SQS and choose the queue. You’ll want to configure the batch size for each Lambda. For example, for the InventoryUpdateLambda:

• Trigger Type: SQS
• SQS Queue: InventoryUpdateQueue
• Batch Size: 10 (This means Lambda will try to process up to 10 messages from the queue in a single invocation.)
• Batch Window: 0 seconds (Process immediately, don’t wait to fill the batch if not needed.)
• Retry Attempts: 3 (For transient errors.)

The Lambda code itself would then receive an event object containing an array of messages from SQS. It processes each message individually.

import json

def lambda_handler(event, context):
    for record in event['Records']:
        message_body = json.loads(record['body'])
        order_data = message_body.get('order') # Assuming your SNS message has an 'order' key

        print(f"Processing order: {order_data.get('orderId')}")

        # --- Your specific processing logic here ---
        if 'inventory' in record['messageAttributes']:
            update_inventory(order_data)
        elif 'shipping' in record['messageAttributes']:
            generate_shipping_label(order_data)
        # ... and so on for other types of messages
        # -----------------------------------------

    return {
        'statusCode': 200,
        'body': json.dumps('Successfully processed batch!')
    }

def update_inventory(order_data):
    print(f"Updating inventory for order {order_data.get('orderId')}")
    # Actual inventory update logic

def generate_shipping_label(order_data):
    print(f"Generating shipping label for order {order_data.get('orderId')}")
    # Actual shipping label logic

The key here is that SNS acts as the dispatcher, and SQS queues act as buffers and independent triggers. If an InventoryUpdateLambda fails to process a message (e.g., a temporary database outage), SQS will retry delivering that message to Lambda. The other Lambdas (for shipping, email, analytics) are completely unaffected and continue processing their own messages. This makes the overall system much more robust.

You can even send different types of messages to the same SNS topic, and use message attributes to filter which SQS queues receive them. For example, you could send a message with MessageAttributes: {"EventType": {"DataType": "String", "StringValue": "InventoryUpdate"}} to the OrderProcessingTopic. Then, you can configure SQS subscriptions to filter messages based on these attributes, ensuring that only relevant messages reach each queue.

The one thing most people don’t realize is how much control you have over error handling and retries at the SQS-to-Lambda integration level. You can configure Dead-Letter Queues (DLQs) for each SQS queue. If Lambda repeatedly fails to process messages from a queue (after its retry attempts), SQS can be configured to move those failed messages to a DLQ. This prevents message loss and allows you to inspect problematic messages later for debugging without blocking new, valid messages.

The next logical step is to consider how to handle ordered processing if certain steps absolutely must happen before others, even in a fan-out scenario.