v0.3 - last updated November 2020
If you want to read why Serverless is part of the SJGAR stack read the Introducing the SJGAR stack post.
What is serverless? In the context of the SJGAR stack serverless basically means you have no servers you can SSH into. Think of AWS Lambda to run your compute workloads. Or DynamoDB to host your NoSQL database. Or S3 to store and serve your files. Usually Docker and Kubernetes do not fall in this category since you can SSH into a running container.
Basically you have no servers you can SSH into
The main advantage of serverless is cheap ops, or no ops. No machines to maintain, no security patches to apply, no log files to purge to prevent disks from running full. All of this is outsourced to the provider.
A serverless architecture runs on components that are operated and managed by the provider. It frees a development team from ops work so the team can focus on the development of business value and therefore focus on customers. Usually this comes at the expense of flexibility.
The tradeoff usually is worth it when starting new projects and therefore a good fit for innovation. Pay as you go means you usually can start almost for free and scale towards your first million users comfortably. For larger scale use planning to move towards containers makes sense once you get close.
It frees a development team from ops work so the team can focus on the development of business value and therefore focus on customers
When AWS Lambda launched in 2014 this gave rise to the notion of serverless computing. The two elements that are important to consider:
- A place for your code to run in the cloud - fully managed;
- Your code can be triggered by different events.
Your code will run in the cloud when you need it and automatically scale up and down
When your pipeline deploys the code to the Lambda it will just run. The service uses, rather than exposes, container technology. When the code is triggered a container will spin up to run your code. The runtime reuses warm containers to speed up subsequent calls to the service. The cold start penalty can be minimized by choosing lightweight runtimes like Node.js and GO. For that same reason it makes sense to not use heavy runtimes like Java.
Your code can be triggered my multiple events
Some examples:
- Exposed as an API endpoint when used with API Gateway;
- When a file is uploaded to S3;
- From DynamoDB when a record is created, updated or deleted.
Software that fulfills requirements is never just about compute. There are so many things that need to be done. Data to be persisted. Files to be stored. Events to be handled. Caches to be used. Logs to be written.
Services like these give you a pretty complete toolbox to create API's, backends and integration services.
Some examples of service types you can get with AWS Serverless:
- NoSQL Database
- Managed API Gateway (REST and GraphQL)
- User management
- File storage
- Logging
- Pub/sub messaging
- Queue
- Event Bus
- Workflow and orchestration
- DNS management
- CDN
Services like these give you a pretty complete toolbox to create API's, backends and integration services.
Going all in an on a serverless architecture means you get the benefit of a wide set of services that all can be integrated easily with code being triggered at key points. These code invocations are handled by the AWS platform in a robust event-driven way that allows your application to scale.
Some example stories with possible solutions:
Data archiving
Story: You want to automatically remove data from your database and archive it two years after creation.
Use Dynamo TTL to schedule automatic deletion after two years. Set up a Lambda trigger to handle events on the Dynamo stream. On deletion of a record store the data sent as the event payload in S3.
Handling user creation
Story: Whenever a user signs up for your service you want to add their email address to your mail software.
Use a Cognito trigger to run a userCreated Lambda. In the lambda call the API of your mail software to register the new users email address.
Thumbnail creation
Note: This is the scenario that was first extensively used by Amazon when they introduced Lambda.
Story: When a user uploads a full resolution picture you want to create thumbnails for use in your apps.
Trigger a Lambda whenever a file is uploaded to a certain S3 bucket. Resize the images and store them in a thumbnail bucket.
Data enrichment
Story: A user enters their cars license plate which is stored in your database. You want to asynchronously enrich this data with an external database with car details to help your ML algorithm perform better.
Set up a carDetailsEnrichment Lambda that handles events for your car Dynamo database. Whenever the licensePlate changes and the enrichment field for that license plate is not set to CARDETAILSENRICHED call the external system and update the record car details and status field.
Adding search capabilities
Story: You want to make your database searchable.
Set up a streamToElastic Lambda that handles events for your Dynamo database. Map the incoming data to the required model and write it to ElasticSearch;
Logging
Story: You want to add logging to your application to help with troubleshooting.
A simple console.log statement in your Node.js code will automatically be streamed to CloudWatch. You can use a library like lambda-log to help simplify adding things like log levels and creating proper JSON logs. Use CloudWatch Log Insights to run powerful queries over your logs.
If you want to hook your logs up to a company wide used tool like Splunk set up a Lambda trigger to handle Cloudwatch log events and write the data to Splunk.
Monitoring, tracing and alerting
Story: You want to be able to view key metrics for your services and get alerts when certain criteria are met.
Use the built in metrics for a Lambda function or build your own dashboard in CloudWatch. Use X-Ray to trace service dependencies and behavior. Set up alerts in CloudWatch and hook up your own code in a Lambda to send notification to Slack. Use Cloudwatch ServiceLens and Lambda Insights to get more insight in the behavior of your services.
Paragraph to be written
Paragraph to be written
Paragraph to be written
Paragraph to be written
Paragraph to be written
- Serverless Architectures - read on martinfowler.com