Reducing Clojure Lambda Cold Starts Part 5 - Native JVM vs Node Performance

Comparing the performance of ClojureScript vs Clojure Lambda got me wondering what the performance difference is between them and the comparable JavaScript and Java Lambda with the same dependencies and essentially the same code. I'll create such Lambdas and run my SQS blaster against them and compare the results to the Clojure/Script versions.

Set Up

JavaScript

JavaScript Lambdas are super easy, I can just add the code inline with no need for any build tools:

...

  RunJavaScriptCalculationsQueue:
    Type: AWS::SQS::Queue
    Properties:
      QueueName: !Sub "${AWS::StackName}-run-calcs-queue-js"
      VisibilityTimeout: 5400        

  RunCalculationsJS:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: !Sub "${AWS::StackName}-run-calcs-js"
      Handler: index.handler
      Runtime: nodejs14.x
      Timeout: 900
      MemorySize: 128
      Policies:
        - AWSLambdaBasicExecutionRole
        - S3ReadPolicy:
            BucketName: !Ref TransactionsBucket
        - S3WritePolicy:
            BucketName: !Ref CalculationsBucket
        - Version: '2012-10-17' 
          Statement:
            - Effect: Allow
              Action:
                - s3:ListAllMyBuckets
              Resource: 'arn:aws:s3:::*'
      Environment:
        Variables:
          TRANSACTIONS_BUCKET: !Ref TransactionsBucket
          CALCULATIONS_BUCKET: !Ref CalculationsBucket
      Events:
        SQSEvent:
          Type: SQS
          Properties:
            Queue: !GetAtt RunJavaScriptCalculationsQueue.Arn
            BatchSize: 1
      InlineCode: |
        const AWS = require('aws-sdk')
        const s3 = new AWS.S3()

        exports.handler = async function(event) {
           return s3.listBuckets().promise();
        }

Deploying, running the SQS blaster, and running this query in Log Insights:

stats avg(@initDuration), avg(@duration), count(@initDuration), count(@duration)

Gives the results:

avg(@initDuration),avg(@duration),count(@initDuration),count(@duration)
445.55,150.6677,16,637

As I had hoped, the durations are nearly identical to the ClojureScript version.

Java

Add src/core/tax/core.java:

package tax;

import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.ListBucketsRequest;

public class core {

    public Object calculationsHandler(Object event) {
        S3Client s3 = S3Client.builder().build();
        ListBucketsRequest req = ListBucketsRequest.builder().build();
        return s3.listBuckets(req).toString();
    }
}

Update build.clj:

(ns build
  (:require [clojure.tools.build.api :as b]
            [clojure.java.shell :refer [sh]]))

(def lib 'taxbit/tax-engine)
(def version "0.1.0")
(def clj-target "target/clj")
(def java-target "target/java")
(def clj-class-dir (format "%s/classes" clj-target))
(def java-class-dir (format "%s/classes" java-target))
(def basis (b/create-basis {:project "deps.edn"}))

(defn uber-file [target-dir]
  (format "%s/%s-%s-standalone.jar" target-dir (name lib) version))
(def clj-uber-file (uber-file clj-target))
(def java-uber-file (uber-file java-target))

(defn clean [path]
  (b/delete {:path path}))

(defn uber [_]
  (clean clj-target)
  (b/copy-dir {:src-dirs ["src/clj" "resources"]
               :target-dir clj-class-dir})
  (b/compile-clj {:basis basis
                  :src-dirs ["src/clj"]
                  :class-dir clj-class-dir})
  (b/uber {:class-dir clj-class-dir
           :uber-file clj-uber-file
           :basis basis
           :main 'tax.core}))

(defn uber-java [_]
  (clean java-target)
  (b/copy-dir {:src-dirs ["src/java" "resources"]
               :target-dir java-class-dir})
  (b/javac {:basis basis
            :src-dirs ["src/java"]
            :class-dir java-class-dir
            :javac-opts ["-source" "11" "-target" "11"]})
  (b/uber {:class-dir java-class-dir
           :uber-file java-uber-file
           :basis basis
           :main 'tax.core}))

Update template.yml:

...
  RunCalculationsCLJ:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: !Sub "${AWS::StackName}-run-calcs-clj"
      Handler: tax.core::calculationsHandler
      Runtime: java11
      CodeUri: target/clj/tax-engine-0.1.0-standalone.jar
      Timeout: 900
      MemorySize: 512
      Policies:
        - AWSLambdaBasicExecutionRole
        - S3ReadPolicy:
            BucketName: !Ref TransactionsBucket
        - S3WritePolicy:
            BucketName: !Ref CalculationsBucket
        - Version: '2012-10-17' 
          Statement:
            - Effect: Allow
              Action:
                - s3:ListAllMyBuckets
              Resource: 'arn:aws:s3:::*'            
      Environment:
        Variables:
          TRANSACTIONS_BUCKET: !Ref TransactionsBucket
          CALCULATIONS_BUCKET: !Ref CalculationsBucket
      Events:
        SQSEvent:
          Type: SQS
          Properties:
            Queue: !GetAtt RunClojureCalculationsQueue.Arn
            BatchSize: 1

  RunCalculationsJava:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: !Sub "${AWS::StackName}-run-calcs-java"
      Handler: tax.core::calculationsHandler
      Runtime: java11
      CodeUri: target/java/tax-engine-0.1.0-standalone.jar
      Timeout: 900
      MemorySize: 512
      Policies:
        - AWSLambdaBasicExecutionRole
        - S3ReadPolicy:
            BucketName: !Ref TransactionsBucket
        - S3WritePolicy:
            BucketName: !Ref CalculationsBucket
        - Version: '2012-10-17' 
          Statement:
            - Effect: Allow
              Action:
                - s3:ListAllMyBuckets
              Resource: 'arn:aws:s3:::*'            
      Environment:
        Variables:
          TRANSACTIONS_BUCKET: !Ref TransactionsBucket
          CALCULATIONS_BUCKET: !Ref CalculationsBucket
      Events:
        SQSEvent:
          Type: SQS
          Properties:
            Queue: !GetAtt RunJavaCalculationsQueue.Arn
            BatchSize: 1
...

Running the SQS blaster on the CLJ and Java versions and running Log Insights queries gives:

---

---

Yikes, I was not expecting such a difference in cold start times. I have been operating under the faulty assumption that most of the load time is due to the JVM, not to the additional Clojure loading! I also was not expecting the cold start for the Java version to be less than our JavaScript version! The durations after cold start are along the lines of what I was expecting, a little more for Clojure than Java, but @duration is the total duration and also includes the @initDuration, so the actual average durations are probably a bit closer.

So, to wrap up, ClojureScript and JavaScript Lambdas with the same dependencies seem to have nearly identical cold start times and overall run times. Java cold starts seem pretty comparable to JavaScript ones, but the run times were actually significantly slower for listBuckets. Clojure cold starts turn the base JVM ones from good to terrible, but the overall runtimes aren't too much worse. Seems to be another win for ClojureScript.