Lambda functions often need sensitive configuration data, like API keys or database credentials. The conventional wisdom is to inject these as environment variables, but that means your secrets are sitting in plain text in your AWS console, which is a security risk.
Here’s a function that fetches a secret from AWS Secrets Manager and uses it:
import boto3
import json
def lambda_handler(event, context):
secret_name = "my/api/secret"
region_name = "us-east-1"
session = boto3.session.Session()
client = session.client(
service_name='secretsmanager',
region_name=region_name
)
try:
get_secret_value_response = client.get_secret_value(
SecretId=secret_name
)
except Exception as e:
# Handle exceptions, e.g., secret not found, permissions error
print(f"Error fetching secret: {e}")
return {
'statusCode': 500,
'body': json.dumps('Failed to retrieve secret.')
}
# Secrets Manager stores secrets in a JSON string or plain text.
# If it's a JSON string, parse it.
if 'SecretString' in get_secret_value_response:
secret = json.loads(get_secret_value_response['SecretString'])
api_key = secret['api_key']
print(f"Successfully retrieved API key.")
# Use the api_key for your application logic
return {
'statusCode': 200,
'body': json.dumps(f'API Key retrieved and used (length: {len(api_key)}).')
}
else:
# Handle binary secrets if applicable
print("Secret is not a string.")
return {
'statusCode': 500,
'body': json.dumps('Secret is not in expected format.')
}
This seems straightforward: call boto3.client('secretsmanager').get_secret_value(). The surprising part is that while this is a secure way to store secrets, the first time your Lambda function runs, it has to make a network call to AWS Secrets Manager. This adds latency. If you have many Lambda functions, each doing this on their first invocation (the "cold start"), your overall application can feel sluggish. The trick is to cache the secret within the Lambda execution environment.
Let’s see how that looks. Instead of fetching the secret every single time the lambda_handler is invoked, we can fetch it once when the Lambda execution environment is initialized.
import boto3
import json
import os
# Global scope: This code runs when the Lambda execution environment is initialized
secret_name = "my/api/secret"
region_name = os.environ.get("AWS_REGION", "us-east-1") # Use environment variable for region
# Attempt to fetch the secret during initialization
try:
session = boto3.session.Session()
client = session.client(
service_name='secretsmanager',
region_name=region_name
)
get_secret_value_response = client.get_secret_value(
SecretId=secret_name
)
# Assume secret is JSON and parse it
if 'SecretString' in get_secret_value_response:
secrets = json.loads(get_secret_value_response['SecretString'])
API_KEY = secrets['api_key'] # Store the actual secret in a global variable
print("Secret fetched and cached during initialization.")
else:
API_KEY = None
print("Secret is not a string, cannot cache API_KEY.")
except Exception as e:
API_KEY = None
print(f"Error fetching secret during initialization: {e}")
def lambda_handler(event, context):
# Check if the secret was successfully cached
if API_KEY:
print(f"Using cached API key (length: {len(API_KEY)}).")
# Use the API_KEY for your application logic
return {
'statusCode': 200,
'body': json.dumps('API Key retrieved from cache and used.')
}
else:
print("API key not available in cache. Fetching now (will be slow).")
# Fallback: fetch if not cached (this will be slow on cold starts)
try:
session = boto3.session.Session()
client = session.client(
service_name='secretsmanager',
region_name=region_name
)
get_secret_value_response = client.get_secret_value(
SecretId=secret_name
)
if 'SecretString' in get_secret_value_response:
secrets = json.loads(get_secret_value_response['SecretString'])
api_key_runtime = secrets['api_key']
print(f"API key fetched at runtime (length: {len(api_key_runtime)}).")
return {
'statusCode': 200,
'body': json.dumps('API Key retrieved at runtime.')
}
else:
return {
'statusCode': 500,
'body': json.dumps('Secret is not in expected format.')
}
except Exception as e:
print(f"Error fetching secret at runtime: {e}")
return {
'statusCode': 500,
'body': json.dumps('Failed to retrieve secret.')
}
The key here is defining API_KEY in the global scope, outside the lambda_handler function. When Lambda initializes the execution environment for your function, it runs this global code once. If the secret is fetched successfully, it’s stored in the API_KEY variable. Subsequent invocations of lambda_handler within that warm execution environment will find API_KEY already populated and use it directly, bypassing the network call to Secrets Manager entirely. This dramatically reduces latency for warm invocations.
To make this work, your Lambda function’s IAM role needs permissions to secretsmanager:GetSecretValue for the specific secret. The secret itself can be stored as plain text or a JSON string. If it’s JSON, you’ll typically parse it to extract individual key-value pairs. The region_name should ideally be configured via an environment variable, AWS_REGION, which Lambda automatically sets.
When you deploy this cached version, you’ll notice a significant speedup for subsequent invocations after the initial cold start. The cold start will still include the Secrets Manager API call, but all "warm" starts will be much faster. The print statements in the code will confirm whether the secret was fetched from the cache or at runtime. You’ll see "Secret fetched and cached during initialization" on the cold start, and "Using cached API key" on subsequent warm starts.
The one thing that many developers miss is that the initialization code runs before the context object is fully populated with details about the specific invocation. For instance, context.aws_request_id will not be available during this global initialization phase. This means you cannot use invocation-specific information to dynamically determine which secret to fetch or how to process it during this initial load. The secret name and region must be static or derived from environment variables set before the Lambda environment starts.
The next hurdle is managing secret rotation and ensuring your Lambda function can gracefully handle updated secrets without requiring a redeploy.