Ackermann on AWS Lambda with S3 Trigger

AWS Lambda is a serverless compute service – run code without thinking about servers. Lambda functions can be implemented in a variety of programming languages and may be triggered from other AWS services (e.g., S3, SNS, API Gateway). Amazon S3 (Simple Storage Service) is an object storage service which can be used to store all kinds of data organized into buckets

In this example, a Java AWS Lambda function is triggered by S3 events. Whenever triggered, it reads computation input in JSON format from S3, computes the Ackermann function, and writes back the result into S3. In case you are wondering: there is no deeper meaning to computing Ackermann in this way other than to showcase the technology stack.

The repository containing the code is here: https://github.com/sbtznb/aws-lambda-ackermann-s3

AWS Java SDK

To interface with AWS Lambda and Amazon S3, we'll use the AWS Java SDK. Using Maven, these are the core dependencies required for our implementation:

<dependency>
  <groupId>com.amazonaws</groupId>
  <artifactId>aws-lambda-java-core</artifactId>
  <version>1.2.1</version>
</dependency>
<dependency>
  <groupId>com.amazonaws</groupId>
  <artifactId>aws-lambda-java-events</artifactId>
  <version>3.11.0</version>
</dependency>
<dependency>
  <groupId>com.amazonaws</groupId>
  <artifactId>aws-java-sdk-s3</artifactId>
  <version>1.12.181</version>
</dependency>
</dependency>

Request Handler

The core functionality is implemented in AckermannS3Handler.java. The class inherits from com.amazonaws.services.lambda.runtime.RequestHandler<S3Event, String>. It reads the computation request from S3, computes the Ackermann function, and writes back the result to S3.

public class AckermannS3Handler implements RequestHandler<S3Event, String> {

    /* ... */

    public String handleRequest(S3Event s3event, Context context) {

        /* */

    }

}

When the function is invoked in AWS Lambda, the handler method handleRequest is called. The Lambda runtime receives an event as a JSON-formatted string and converts it into an object. In this case, the input object is of type com.amazonaws.services.lambda.runtime.events.S3Event which represents an S3 event. Lambda passes the event object to our function handler along with a context object that provides details about the invocation and the function.

Read JSON from S3

First the S3EventNotificationRecord is accessed which contains information on the S3 entity as well as the operation (in our case "ObjectCreated:Put") that caused the S3 event to be triggered. To uniquely identify an object S3 we need its key and the name of the bucket in which the object is placed. Using com.amazonaws.services.s3.AmazonS3 we can read the source object which is in JSON format. The method S3Object.getObjectContent reutrns an input stream representing the content of our S3 object. The contents are then converted to a POJO AckermannComputation using Gson.


final private AmazonS3 s3Client = AmazonS3ClientBuilder.defaultClient();

/* ... */

// Read S3 event
final S3EventNotificationRecord record = s3event.getRecords().get(0);
final String srcBucket = record.getS3().getBucket().getName();
final String srcKey = record.getS3().getObject().getUrlDecodedKey();

// Read computation request (JSON to POJO) using AmazonS3.getObject
final S3Object s3Object = s3Client.getObject(new GetObjectRequest(srcBucket, srcKey));
final AckermannComputation computation = new Gson().fromJson(new InputStreamReader(s3Object.getObjectContent(), StandardCharsets.UTF_8), AckermannComputation.class);

To get a better understanding of the contents of S3EventNotificationRecord you may check out AckermannS3HandlerTest.java in which a record is instantiated using dummy data:

S3EventNotificationRecord record = new S3EventNotificationRecord("eu-central-1",
                "ObjectCreated:Put",
                "aws:s3",
                "2022-05-21T00:30:12.456Z",
                "2.1",
                new RequestParametersEntity("174.255.255.156"),
                new ResponseElementsEntity("nBbLJ4PAHhdvxmplPvtCgTrWCqf/KtonyV93l9rcoMLeIWJxpS9x9P8u01+Tj0OdbAoGs+VGvEvWl/Sg1NW5uEsVO25Laq7L", "AF2D7AB6002E898D"),
                new S3Entity("682bbb7a-xmpl-4843ca-94b1-7f77c4d6dbf0",
                 new S3BucketEntity("ackermann-test-bucket",
                   new UserIdentityEntity("BA3XMPLFAF2AI3E"),
                   "arn:aws:s3:::" + "ackermann-bucket"),
                 new S3ObjectEntity("input/ackermann-computation.json",
                   Long.valueOf(1),
                   "d132690b6c65b6d1629721dcfb49b883",
                   "1.0",
                   "005E64A65DF093B26D"),
                 "1.0"),
                new UserIdentityEntity("AWS:AIDAINPONIXMPLT3IKHL2"));

The S3 object itself is a very simple JSON file containing two numbers, m and n:

{
    "m": 3,
    "n": 1
}

Compute Ackermann Function

Using the POJO AckermannComputation we feed the computation input m and n to our Ackermann implementation which is imported statically. The computation result is fed back into the POJO.


import static de.zenbit.aws.Ackermann.ackermann;

/* ... */

// Compute Ackermann function
long m = computation.getM();
long n = computation.getN();

computation.setResult(ackermann(m, n));

The actual implementation in Ackermann.java isn't of much concern to us. It's a naive, stack-based implementation as a fully recursive implemetation quickly blows the JVM's heap. It computes Ackermann A(4, 1) = 65533 for m = 4 and n = 1 in a couple of minutes on my machine.

/**
* Stack-based Ackermann implemenation following 
* "An inherently iterative computation of ackermann's function" 
* by Jerrold W.Grossman and R.Suzanne Zeitman
*/
static Long ackermann(long m, long n) {

    if (m < 0 || n < 0) {
        throw new IllegalArgumentException("Undefined for negative inputs.");
    }

    final Stack<Long> stack = new Stack<>();

    stack.push(m);
    stack.push(n);

    while (stack.size() > 1) {
        
        n = stack.pop();
        m = stack.pop();
        
        if (m == 0) {
            stack.push(n + 1);
        } else if (n == 0) {
            stack.push(m - 1);
            stack.push(1L);
        } else {
            stack.push(m - 1);
            stack.push(m);
            stack.push(n - 1);
        }
    }

    return stack.pop();

}

Write JSON to S3

Finally the computation result is converted back into JSON using Gson and stored into S3 using AmazonS3.putObject. The result is stored under a new key so separation between input and output data is ensured. This is important as we are writing into the same bucket we are being triggered from. In general, this is not recommenend practice as it may lead to a Lambda function triggering itself recursively.

To avoid recursive invocations between S3 and Lambda, it’s best practice to store the output of a process in a different resource from the source S3 bucket.

private static final String INPUT_PREFIX = "input";
private static final String OUTPUT_PREFIX = "output";

/* ... */

// Write computation result (POJO to JSON) using AmazonS3.putObject
final String dstKey = srcKey.replaceFirst(INPUT_PREFIX, OUTPUT_PREFIX);
final String result = new Gson().toJson(computation);
s3Client.putObject(srcBucket, dstKey, result);

Set up Lambda Function

  1. In AWS, create an new Lambda function, setting runtime to Java. Assign a basic execution role that based on AWSLambdaBasicExecutionRole and AmazonS3FullAccess permissions. Edit the Runtime Settingsand set Handler to de.zenbit.aws.AckermannS3Handler.

  2. Build the deployment artifact (JAR) using Maven.

$ git clone git@github.com:sbtznb/aws-lambda-ackermann-s3.git
$ cd ./aws-lambda-ackermann-s3
$ mvn clean package
  1. Deploy the JAR file to AWS Lambda using the AWS CLI. Of course, may also deploy the JAR file using the Lambda console In the Code Source pane, choose Upload from and then .zip file.
$ cd ./target
$ aws lambda update-function-code --function-name YOUR-FUNCTION-NAME --zip-file fileb://aws-lambda-ackermann-s3-0.0.1-SNAPSHOT.jar
  1. Test your function using using the AWS CLI or the Lambda console using JSON input.
$ aws lambda invoke --function-name YOUR-FUNCTION-NAME --cli-binary-format raw-in-base64-out --payload '{"m":3,"n":2}' out.json
$ cat out.json
> 29

Set up S3 Trigger

  1. Create an S3 bucket
$ aws s3api create-bucket --bucket YOUR-BUCKET-NAME --region eu-central-1 --create-bucket-configuration LocationConstraint=eu-central-1
  1. Add a trigger to your Lambda function
    • Type S3 trigger for your bucket
    • Event type All object create events
    • Prefix input/
    • Suffix .json

Be carefeful setting up the trigger to avoid recursive invocations between S3 and Lambda.

Final Test

Upload a file ackermann-computation.json into the input-folder in your S3-bucket, e.g.;

{
  "m": 2,
  "n": 2
}

Your Lambda function should be triggered and process the input.

A new folder "output" will be created containing an output file with the same name as the input file. The output file contains the result of the computation:

{
  "m": 2,
  "n": 2,
  "result": 7
}