A Lambda custom runtime allows you to run code written in any language on AWS Lambda, bypassing the need to use a pre-built runtime provided by AWS.
Here’s how you can build your own bootstrap for a custom runtime:
First, you need to understand the Lambda Runtime API. This API is a set of HTTP endpoints that your runtime uses to communicate with the Lambda service. The two primary endpoints are:
/runtime/invocation/next: Used by your runtime to fetch the next event to process./runtime/invocation/{awsRequestId}/response: Used by your runtime to send the response back to Lambda./runtime/invocation/{awsRequestId}/error: Used by your runtime to send an error back to Lambda.
Let’s consider a simple Go program that acts as a custom runtime.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
)
const (
runtimeAPIBaseURL = "http://127.0.0.1:9001/2018-06-01/runtime"
)
func main() {
for {
// Get the next event
req, err := http.NewRequest("GET", runtimeAPIBaseURL+"/invocation/next", nil)
if err != nil {
log.Fatalf("failed to create request: %v", err)
}
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
log.Fatalf("failed to get next invocation: %v", err)
}
defer resp.Body.Close()
awsRequestID := resp.Header.Get("Lambda-Runtime-Aws-Request-Id")
if awsRequestID == "" {
log.Fatal("Lambda-Runtime-Aws-Request-Id header is missing")
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalf("failed to read response body: %v", err)
}
// Process the event (this is where your custom logic goes)
log.Printf("Received event: %s for request ID: %s", string(body), awsRequestID)
// Example: Echo the event back as a response
responseBody, err := json.Marshal(map[string]string{
"message": fmt.Sprintf("Hello from custom runtime! You sent: %s", string(body)),
})
if err != nil {
log.Fatalf("failed to marshal response: %v", err)
}
// Send the response back to Lambda
postResp, err := http.Post(fmt.Sprintf("%s/invocation/%s/response", runtimeAPIBaseURL, awsRequestID), "application/json", bytes.NewBuffer(responseBody))
if err != nil {
log.Fatalf("failed to send response: %v", err)
}
defer postResp.Body.Close()
if postResp.StatusCode >= 400 {
log.Fatalf("received error response from Lambda: %d", postResp.StatusCode)
}
}
}
To make this a Lambda function, you need to package it correctly. This involves creating a deployment package that contains your executable and any necessary libraries. For Go, you’d typically build a static binary.
Here’s a Dockerfile example to build a Go binary and package it for Lambda:
FROM golang:1.18-alpine AS builder
WORKDIR /app
COPY . .
RUN go build -o bootstrap main.go
FROM alpine:latest
COPY --from=builder /app/bootstrap /lambda/bootstrap
# Set the handler to your bootstrap executable
# This is crucial for Lambda to know what to run
CMD ["/lambda/bootstrap"]
You would then build this Docker image and push it to a container registry (like Amazon ECR). When creating your Lambda function, you’d choose a container image as the deployment package and point it to your image.
The bootstrap executable is the entry point for your Lambda function. When Lambda starts your function, it looks for an executable file named bootstrap in the root of your deployment package. If you’re using a container image, this executable needs to be present at the root of the container’s filesystem or in a location specified by the CMD instruction in your Dockerfile.
The core loop of the bootstrap program is to continuously poll the Lambda Runtime API for new events. When an event is received, your code processes it and sends the result back via the API. If an error occurs during processing, you use the /runtime/invocation/{awsRequestId}/error endpoint to report it. This allows Lambda to manage the execution lifecycle, including retries and error handling.
The environment variable AWS_LAMBDA_RUNTIME_API provides the base URL for the Runtime API, making your code portable. Your bootstrap executable is essentially the heart of your custom runtime, orchestrating the entire interaction between your application code and the Lambda execution environment.
The most surprising thing about Lambda custom runtimes is that they don’t actually run your code directly; they act as a proxy, fetching events from Lambda, passing them to your actual application code, and then returning the results. Your bootstrap is just the intermediary that translates Lambda’s expectations into something your language/framework understands.
Consider a slightly more complex scenario where you want to use a web framework. Your bootstrap could start a web server (e.g., a Go net/http server) listening on 0.0.0.0:8080. Then, for each Lambda event, it would simulate an HTTP request to that local server and return the server’s response to Lambda.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"os"
"time"
)
const (
runtimeAPIBaseURL = "http://127.0.0.1:9001/2018-06-01/runtime"
)
// Simulated HTTP request to a local web server
type SimulatedRequest struct {
Method string `json:"method"`
Path string `json:"path"`
Headers map[string]string `json:"headers"`
Body string `json:"body"`
}
// Simulated HTTP response from a local web server
type SimulatedResponse struct {
StatusCode int `json:"statusCode"`
Headers map[string]string `json:"headers"`
Body string `json:"body"`
}
func main() {
// Start a local web server that your application code will use
go func() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
// This is where your actual application logic would live
// For this example, we'll just echo back the request details
bodyBytes, _ := io.ReadAll(r.Body)
response := SimulatedResponse{
StatusCode: 200,
Headers: map[string]string{
"Content-Type": "application/json",
},
Body: fmt.Sprintf(`{"message": "Processed by local server", "method": "%s", "path": "%s", "body": "%s"}`, r.Method, r.URL.Path, string(bodyBytes)),
}
respJSON, _ := json.Marshal(response)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
w.Write(respJSON)
})
log.Println("Starting local web server on :8080")
if err := http.ListenAndServe(":8080", nil); err != nil {
log.Fatalf("local server failed: %v", err)
}
}()
// Give the server a moment to start
time.Sleep(2 * time.Second)
for {
req, err := http.NewRequest("GET", runtimeAPIBaseURL+"/invocation/next", nil)
if err != nil {
log.Fatalf("failed to create request: %v", err)
}
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
log.Fatalf("failed to get next invocation: %v", err)
}
defer resp.Body.Close()
awsRequestID := resp.Header.Get("Lambda-Runtime-Aws-Request-Id")
if awsRequestID == "" {
log.Fatal("Lambda-Runtime-Aws-Request-Id header is missing")
}
lambdaEventBody, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalf("failed to read Lambda event body: %v", err)
}
// Simulate an HTTP request to our local web server
// The structure of this simulated request can be customized
simulatedReq := SimulatedRequest{
Method: "POST", // Or GET, PUT, etc.
Path: "/process",
Headers: map[string]string{
"Content-Type": "application/json",
"X-Lambda-Request-ID": awsRequestID, // Pass through request ID if needed
},
Body: string(lambdaEventBody),
}
simulatedReqJSON, _ := json.Marshal(simulatedReq)
localResp, err := http.Post("http://127.0.0.1:8080/simulate", "application/json", bytes.NewBuffer(simulatedReqJSON))
if err != nil {
log.Fatalf("failed to send simulated request to local server: %v", err)
}
defer localResp.Body.Close()
localRespBody, err := ioutil.ReadAll(localResp.Body)
if err != nil {
log.Fatalf("failed to read response from local server: %v", err)
}
// Send the response from the local server back to Lambda
postResp, err := http.Post(fmt.Sprintf("%s/invocation/%s/response", runtimeAPIBaseURL, awsRequestID), "application/json", bytes.NewBuffer(localRespBody))
if err != nil {
log.Fatalf("failed to send response to Lambda: %v", err)
}
defer postResp.Body.Close()
if postResp.StatusCode >= 400 {
log.Fatalf("received error response from Lambda: %d", postResp.StatusCode)
}
}
}
This pattern is powerful because it allows you to leverage existing web frameworks or libraries without significant modification. Your bootstrap acts as the adapter, translating Lambda’s event-driven model into the request/response model your framework expects.
When deploying a custom runtime, you must set the runtime parameter in your Lambda function configuration to provided or provided.al2 (for Amazon Linux 2). This tells Lambda to expect a custom runtime and to look for the bootstrap executable. The CMD in your Dockerfile or the entry point specified during deployment will be used by Lambda to find and execute this bootstrap file.
The memory and timeout settings for your Lambda function directly influence the resources available to both your bootstrap executable and the application code it invokes. If your bootstrap is inefficient or your local server requires significant resources, you might hit Lambda’s limits.
The next challenge you’ll face is managing cold starts effectively, as your bootstrap needs to initialize any long-running processes (like web servers) on its first invocation.