Reducing Clojure Lambda Cold Starts Part 3 - ClojureScript

Reducing Clojure Lambda Cold Starts Part 3 - ClojureScript

Larry Christensen's photo
Larry Christensen
·

5 min read

Me and ClojureScript

At various companies, I spearheaded migrations many front-end migrations, from unresponsive JSPs and ASPs to responsive but out of control jQuery SPAs; to more structured but slow-compiling and extremely verbose GWTs; back to pure JS SPAs with ExtJS, which was super-heavy-weight and took forever to load; to Backbone.js with jQuery, which gave lightweight structure but still got out of control with rampant jQuery state mutation. Then came Angular, which felt like a true front-end framework. The migration went well, but it quickly turned into a complicated web of callbacks and state mutation where callback loops would cause the UI to start doing mysterious (and sometimes comical) things that were difficult to debug. The stage was set for React. A unidirectional rendering loop inspired by functional programming? Yep, I need that. React was a huge step forward for my team and the migration went so well that all other teams at that company quickly jumped on board.

Something was still missing, though. I've always loved the high-level expressiveness of functional programming languages. There were libraries like Underscore that filled the functional gap, but I'm a minimalist when it comes to dependencies because of additional load times, frustrating dependency conflicts, and security vulnerabilities they introduce. JavaScript, at the same time, was getting better, but to use some of the newer features one had to use something like Babel to 'transpile' a better version of the language to one that most browsers could support (this is really compilation, but the term caught on for JS->JS compilation and now for *->JS compilation). If I'm going to be compiling from a better language to common JS, why not pick a great one that has all of the functional features my heart desires and not just one that adds a few extra features?

Enter ClojureScript. I've loved Lisp since my exposure to programming in CS 101. I had dabbled with Clojure since its early days and fell in love. Now I can build a SPA Web application in Clojure as well?! I can use the same build tool I've been using in Clojure, rather than the dozens of ephemeral ones required for a JS application?! I can even write my CSS in Clojure?! This led me to a journey that I would describe as the closest thing to front-end programming bliss I could imagine, but that's a subject for another post.

Alas, though, the days of the true full-stack developer were coming to an end. With the proliferation of tools on the front end, front-end development had become so specialized and rapidly evolving that only a dedicated front-end developer could keep up. My favorite challenges involve the back-end anyway. The challenge of building massively scalable, performant, data-intensive applications speaks to me. This brings us back to Lambda.

ClojureScript Lambda Experiment

Maybe ClojureScript is the answer to Clojure Lambda cold starts? I've had some success with it in the past, so let's give it another go. The first thing we need is a ClojureScript Lambda handler. While we'll be able to use the same code for our business logic using CLJC, the code at the edges will be significantly different because of the differences in the JVM vs Node runtimes:

>> emacs src/cljs/tax/core.cljs

Notice the 'cljs' in the path and 'cljs' extension. The 'cljs' in the path is arbitrary, but in the extension, it is required or the CLJS compiler will ignore it. 

(ns tax.core
  (:require [cljs.core.async :as async :refer [<!]])
  (:require-macros [cljs.core.async.macros :refer [go]]))

(defn handler [event context callback]
  (js/console.log "HELLO EVENT" event)
  (go (<! (async/timeout 3000))
      (callback nil event)))

ShadowCLJS seems to be the easiest way to compile a ClojureScript project, so I'll install it and initialize the project configuration:

>> npm install -g shadow-cljs
>> shadow-cljs init

This will create a shadow-cljs.edn file, which I modify to look like this:

;; shadow-cljs configuration
{:source-paths
 ["src/cljs"]

 :dependencies
 []

 :builds
 {:calcs-lambda {:target :node-library
                            :exports {:handler tax.core/handler}
                            :output-to "target/lambda/calcs/index.js"}}}

And update my template.yml:

AWSTemplateFormatVersion: "2010-09-09"

Transform:
- "AWS::Serverless-2016-10-31"

Resources:

  RunCalculationsQueue:
    Type: AWS::SQS::Queue
    Properties:
      QueueName: !Sub "${AWS::StackName}-run-calcs-queue-2"
      VisibilityTimeout: 5400

  RunCalculationsJS:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: !Sub "${AWS::StackName}-run-calcs-2"
      Handler: index.handler
      Runtime: nodejs14.x
      CodeUri: target/lambda/calcs
      Timeout: 900
      MemorySize: 128
      Policies:
        - AWSLambdaBasicExecutionRole
      Events:
        SQSEvent:
          Type: SQS
          Properties:
            Queue: !GetAtt RunCalculationsQueue.Arn
            BatchSize: 1            

Outputs:

  QueueUrl:
    Value: !Ref RunCalculationsQueue

And try running it locally:

>> sam invoke local
Invoking index.handler (nodejs14.x)
Skip pulling image and use local one: public.ecr.aws/sam/emulation-nodejs14.x:rapid-1.35.0-x86_64.

Mounting /Users/larry/Documents/code/tax-engine-experiments/target/lambda/calcs as /var/task:ro,delegated inside runtime container
START RequestId: fb598ebf-3ef3-4eb2-8213-cda77e07c9f5 Version: $LATEST
2021-12-24T23:00:36.239Z    undefined    INFO    HELLO CLJS!
2021-12-24T23:00:36.302Z    fb598ebf-3ef3-4eb2-8213-cda77e07c9f5    INFO    HELLO EVENT {}
END RequestId: fb598ebf-3ef3-4eb2-8213-cda77e07c9f5
REPORT RequestId: fb598ebf-3ef3-4eb2-8213-cda77e07c9f5    Init Duration: 1.29 ms    Duration: 992.57 ms    Billed Duration: 993 ms    Memory Size: 128 MB    Max Memory Used: 128 MB

Boom! Now let's deploy:

>> sam package --s3-bucket larrys-cool-bucket
>> sam deploy --stack-name tax-engine-experiments-2 --s3-bucket larrys-cool-bucket --capabilities CAPABILITY_IAM

Now running a test in the Lambda console:

Screen Shot 2021-12-24 at 4.48.02 PM.png

Init duration of 185.39 ms, now that's more like it. Let's try the SQS blaster again to trigger a ton of cold starts:

>> clj -Aprofile

After some time for the queue to drain and the logs to populate, here's what I get in Log Insights:

Screen Shot 2021-12-24 at 4.57.58 PM.png

An average of 182.5228 ms for 82 cold starts, that's pretty dang fast, at least in comparison to Clojure on the JVM! ClojureScript might indeed be the way to go. I'm still skeptical about how performant it will be on heavy computations, so next time we'll add some heavy computations and compare the performance.

ClojureClojureScriptAWSaws lambda