Build and Deploy Serverless Application with webpack

Hey guys!!!
Recently I got chance to solve an interesting as well as a challenging problem. We were working on a Node.js / Typescript application build on top of serverless framework. We were using aws cloudFormation template to deploy the aws lambdas after manually packaging the build artifacts. It was all good when the codebase was small, the deployment was working fine. But as we started adding new features, the codebase started to grow exponentially. As a result , the size of the build artifact kept increasing and finally one day …… BOOM !!!!!!!

The deployment started failing because the size of the build exceeded the limit ( 250MB) AWS provides. We were nearing the first release of the application and it was a shocker for us.

By the time this issue occurred , our codebase already had over 100 files. Initially we thought we’ll have to restructure the entire codebase , rethink about the packages we have used.
Imagine you’ve to sit and restructure the codebase having 100+ files and find a light weighted alternative of each npm package being used…………… Yeah ! it was going to be painful. We needed something intelligent , quick and robust.

So this is how I managed to reduce the build size of the application from 300MB to 2–3 MB per lambda using webpack:

Webpack as you might already know , a module bundler which is primarily used in the UI frameworks i.e. Angular/React etc. But Kudos to the creators, it is so configurable that it can be used anywhere.
Serverless provides a plugin called serverless-webpack which rescued us.
Let’s go step by step and see how serverless can be configured to deploy the lambdas individually with drastically reduced size.

In this tutorial :

1. Create a new repository and install required packages
2. Create lambda handlers
3. Setup tsconfig.json
4. Setup webpack.config.js
5. Setup serverless.yml
6. Deploy and test the lambdas

To start , open terminal , make a directory serverless-webpack-app and go inside it.

Run npm init , it’ll ask you a set of questions , fill it according to your preference(or run with -y flag to have default values). Once you’re done , it will create a package.json file in the root folder.

Install following npm packages :
install serverless globally :

npm i -g serverless

then add below packages:

npm install -D webpack typescript ts-loader serverless-webpack webpack-node-externals terser-webpack-plugin fork-ts-checker-webpack-plugin

once the installation is completed , install lodash :

 npm install aws-sdk lodash @types/lodash 

Now run following commands :

touch serverless.yml
touch webpack.config.js
touch tsconfig.json

Create a folder src and create 2 files with names helloLambda.ts and testLambda.tsinside it. and add following content inside it :

helloLambda.ts

testLambda.ts

By this time , you have :

lambda handlers : actual code to be executed by lambda
package.json : It has all the required dependencies and scripts
webpack.config.js : Blank right now but we will add config later in this tutorial.
serverless.yml : Blank right now, will be updated by cloudFormation template
tsconfig.json : Blank right now, will be updated with typescript rules later.

Now let’s update the typescript compilation rules , open tsconfig.json and copy the below content :

tsconfig.json

the tsconfig.json will be used by ts-loader to transpile the typescript code into Javascript. We will add this loader in the webpack.config.js in the next step.

Now open webpack.config.js and copy paste the below code :

webpack.config.js

lets understand what we did here :
Webpack expects a configuration file to understand how the app will be built. The configuration file can be in json , yml or js format.
in the module we are using ts-loader to transpile the typescript. You need to write rules based on the type of file you have in your project. for example if you have css/scss files , you need an appropriate loader to resolve such file and get your code compiled successfully. ( see loaders ).

node-externals library enables the third party dependencies to be installed separately and not included in the build file. You can avoid it if you want the the third party libraries to be included in the build file. ( I like to keep it separate).
TerserPlugin is one of the most popular code minification library. We are using this to minify the code.
ForkTsCheckerWebpackPlugin is used for Typescript’s type-checking and linting.

Next step is to setup serverless.yml file to get the application deployed. open the serverless.yml file and paste copy paste the content below :

serverless.yml

So the serverless.yml contains the usual template for lambdas and other resources you need to deploy the application ( serverless starter ).
Apart from the followings :

package:  
  individually: true

this option enables the individual packaging of the lambda. When this option is set to true , serverless will package the lambdas individually. You can set it to false if you want a single package for all the lambdas deployment ( not recommended).

plugins:  
   - serverless-webpack

This plugin enables serverless to package the lambda using webpack . You need to provide webpack configuration as custom property.

custom:  
  webpack:    
    webpackConfig: './webpack.config.js'   
    packager: 'npm'    
    includeModules:      
      forceExclude:        
        - aws-sdk

We are passing the configuration for the serverless-webpack plugin. You can explicitly exclude and include third party libraries based on your requirement. Since aws-sdk is already provided in the lambda environment by AWS , we don’t need to package it in the build hence it’s been excluded forcefully.

Now we are almost done. If you haven’t configured aws-cli on your machine you need to configure it to be able to successfully deploy the application. Alternatively you can export your secret key as following :

serverless config credentials --provider aws --key <Access Key ID> --secret <Secret Access Key>

once this is done, run the following command in the terminal to deploy the app :

sls deploy --config serverless.yml

serverless deployment result

congratulations !!!! your serverless-webpack app has been deployed successfully.

To invoke use the following command :

sls invoke -f  testLambda  -l

lambda invoke result

Now to remove the stack , run the following command :

sls remove 

remove serverless stack

So after 2–3 weeks of rigorous efforts finally we were able to deploy our application using webpack. This not only reduced the deployment package size but also helped us catching typescript errors.

That’s it for this time, I hope it was useful for you. Give a clap if you liked it. Suggestions are always welcomed!

Here is the complete source code :

programoholic/Serverless-Webpack-App (github.com)

In the next blog I’m going to explain How I streamed a zip file having multiple csv files with more than 1 million records into kafka producer.
Stay tuned :)