Published October 28th, 2020 – 20 min read
In this article I will introduce you to a new software stack that can be thought of as a spiritual successor to the MEAN or MERN stack. The technologies in the SJGAR stack were carefully combined to create something where the whole is greater than the sum of its parts. Focus on your customers, on business value and the developer experience. Do more, and keep things simple.
If you want to find out why the mix of Serverless, JavaScript, GraphQL, AWS and React could be interesting to you please read on.
Disclaimer: This is not a sponsored post. All views expressed in this article are mine and may or may not be shared by my employer.
I remember writing software in the period roughly between 2005 and 2015 where sometimes it could feel like every week something new was thrown at us. We were transitioning from desktop to the web and then cloud and mobile. We were just getting good at Object-oriented programming when a shift towards Functional Programming required us to rethink our mental models around our code. Cloud scale computing brought us from solely using SQL databases to study and use NoSQL databases. We went from embracing XML and SOAP to transitioning to JSON.
Focus on your customers, on business value and the developer experience. Do more, and keep things simple.
Note: If you would like to skip the story you can skip ahead to the 'Introducing the SJGAR stack' paragraph.
Looking at JavaScript alone there was also a lot of turmoil. The language started almost as a toy language to glue together simple elements on a web page. Then it slowly but steadily grew into becoming the most widely used programming language on the web. It even moved on beyond the browser and started to conquer the server with Node.js. A realm where Java and .NET were dominant before, now had to create some room for people starting to use JavaScript on the server.
On the web, framework after framework was becoming popular. Much to the point where they are still being refactored away ten years later. I remember moving from jQuery to Backbone or Knockout. I used Ember a bit and then also Angular. All of these seemed to offer some improvements over what was previously there. There were still enough rough edges however. It was clear we were not there yet.
In 2015, I got the chance to work on a corporate startup project that was completely greenfield. We set out to create a platform with web and mobile apps with state-of-the-art user experience, and we wanted to fail or succeed fast. This meant we had to ensure a short time to market. We had to focus on developer experience and productivity. After having tried some pieces of technology on smaller projects, this was the perfect chance to put them together and figure out if they would play nicely together. In theory, it should all work, but previous experience had shown that you really need to start using them for a decent amount of time to figure out if the new advantages will really outweigh the drawbacks.
It worked. It really worked.
We built a web application using React and an accompanying mobile app using React Native. The backend was built using AWS Serverless technologies. We used DynamoDB as the single source of truth for data. The built-in event triggering system was used to run some Node.js code in a Lambda to stream this data to a managed Elasticsearch cluster, so we could support the search and query capabilities we needed. We created a GraphQL API using the reference server implementation graphql-js (Apollo Server and Client weren't a thing back then). Again this was run in Lambda. To make sure the web application was performant enough for our end users we used a Lambda to implement Server Side Rendering (Gatsby and Next.js also weren't a thing back then). We also used the Serverless framework, a bit shaky at that time, to write our infra as code.
With a handful of full stack engineers we were able to deliver this project successfully to production. We could deliver new versions of our microservices independently, and we would do this multiple times a day.
It worked. It really worked.
We started with almost zero costs for infrastructure. The most expensive service was the Elasticsearch service. The other services would easily service our first customers on the free tier. I think we never spent over 100 euros a month. Without Elasticsearch it would probably not have been higher than 10 euros a month. We launched and started running ad campaigns to lure customers to our new offering. For these ad campaigns we were spending over a thousand euros per month. It was clear that we now could go to the business and tell them we could do innovation projects with IT running costs of close to zero.
I remember we were able to move some logic from the web app to the authentication provider (we used Auth0 back then since Cognito was still a bit too new) by just copying some JavaScript code. Similarly, we were able to move code from frontend to backend. More generally we were able to take what we learned in a certain context, and apply it to a different one. JavaScript allowed us to learn a single programming language and write code for the Web, Mobile and Cloud. React allowed us to use a single framework to structure applications and do the state management for both Web and Mobile. Using Flexbox we could even apply the same UI layout system in these apps. With the complex and time-consuming nature of layout work, it was great that we were able to reuse our knowledge and tools to enjoy a great productivity boost.
Going Serverless meant we did not have to spend time on managing servers. The code and the system would just work. No disks filling up, no security patches to be applied, no certificates to be updated. AWS had given us a platform with great performance and stability. Everything would just run.
After running this project in 2015 and 2016, in the end it was shuttered because it did not align with our business goals. Only later I came to realize this set of technologies would stand the test of time, and would do so pretty well.
Fast-forward a bit to when I joined my current team and company early 2019. We formed a new innovation department and dreamed of a future where we helped turn our 175-year-old financial company into a fintech. Teaming up with the department that was our business counterpart, in the first year we built and delivered a total of fourteen projects with a relatively common two pizza team. In the previous nineteen years of my career I had never witnessed anything like this. We built smaller projects like landing pages, but also large ones like a new platform to sell houses using data and AI. We also built a fairly complex API to serve as the backend for an IoT driven car insurance. We even managed to surprise ourselves and win a hackathon in our first year with a mobile app for planting trees.
Five years after having first used this set of technologies it was clear that they still were around. Not only were they still around, they had become even stronger with a bigger community supporting them.
Many things fell into place to make this a success. For one, it were the amazing people we joined. I consider myself lucky to this day that we met them and were able to make all of these things happen. It was almost as if we started in a performing phase as a team, skipping the forming, norming and storming phases altogether.
Besides the people the technology played an important role here. Five years after having first used this set of technologies it was clear that they still were around. Not only were they still around, they had become even stronger with a bigger community supporting them.
What was really cool was that we were able to deliver all these different types of applications using basically the same architecture. For the IoT insurance, the API we delivered we implemented a lightweight event sourced system using mainly DynamoDB and Lambda. We later added a future event system where we combined DynamoTTL and Step Functions to combine high and low resolution timers, all of which were inspired by a set of blog posts and a tweet we found on the web.
For the housing platform we combined Cognito, Azure AD, DynamoDB, Lambda, AppSync, S3 and CloudFront to deliver a platform that was highly performant. Gatsby helped us leverage React’s power to build statically rendered, Seo-friendly pages, hosted on S3 and made available to the wide internet using CloudFront and Route53. On the backend we created a GraphQL API to, on the one hand, power the build-time data requirements. On the other hand it also powered the real-time admin panel we needed for providing customers support.
There were many more projects that used very similar architectures. Five years after first having used these technologies we were still able to use them to build modern applications. In our hiring process I remember seeing the smiles on the faces of the engineers when we would mention our tech stack.
Which parts are going to be successful for at least the next five years?
There was this one thing I noticed though. Whenever I wanted to explain our technology stack, I was summing up this combination of technologies one by one. Time after time saying things like “Our stack is JavaScript based, uses the React ecosystem and runs on AWS Serverless... And oh yeah we use GraphQL as a BFF for our apps.” A couple of days later I would say something like “We build apps using React and use JavaScript/TypeScript. Our APIs are running on AWS Lambda using NodeJS.”. And then many more variations of these sentences.
Okay. This thing needs a name.
Back in the days we had the LAMP stack, and some time after that there was the MEAN stack. Acronyms that became part of our vocabulary to help explain the stack we were using, or even inspired greenfield projects to combine these technologies. People would think, hey, I recently have been hearing a lot of buzz around the MEAN stack. Why don't we give it a shot for this new project we are going to work on? A named stack can also help with learning and education tracks. Engineers looking to update or broaden their skill set can find guidance in what to invest in.
It was never the parts of these stacks that were new. What was new was the call to combine these parts to create a synergetic whole with a value greater than the sum of its parts.
I sat down and looked carefully through the technology set we were using to determine the most important ones. I asked myself: Which parts are going to be successful for at least the next five years? Which ones are essential? Which ones can be taken out?
The SJGAR stack builds on the concepts of the MEAN stack, but importantly adds guidance on where to run your code and databases — use a serverless architecture on AWS.
I settled on JavaScript, React, AWS, Serverless and GraphQL. This was the core of everything we had been doing for the past 5 years.
To pick a name I used an online tool to create all possible combinations of these five letters and started to read through them to see if there was something there. With only one vowel it was clear that it was going to be tough. And tough it was. Finally, it happened. After glancing through the entries this one combination was staring me in the face. SJGAR. If you squinted a bit, it was easy to read it as SUGAR.
Sweet.
We have our name.
/pronounce the sugar stack/
The SJGAR stack is a software solution stack for creating modern apps on web and mobile that embraces cloud and APIs. The stack is a collection of widely used technologies where each part comes with a vibrant community. As a whole, because of this, the stack comes with a large full stack talent pool.
Serverless
In the context of the SJGAR stack serverless basically means you have no servers you can SSH into. One of the main advantages of serverless is cheap ops, or no ops. No machines to maintain, no security patches to apply, no log files to purge to prevent disks from running full. All of this is outsourced to the provider. This frees a development team to work on creating business value and focus on customers.
JavaScript
JavaScript is the world's most popular programming language. It basically runs everywhere. In browsers, on servers, devices and the desktop. From a somewhat quirky start, slowly but steadily, it has evolved into an elegant programming language that is used to write high grade production software in many industries.
GraphQL
GraphQL is making a significant impact on how systems are connected in the modern world. One of the uses is serving as a Backend-For-Frontend for modern web and mobile apps. The rich community and tool set with for instance Apollo server and client are giving companies a great chance to get a head start when dealing with state management and server client communication. Because the libraries do the heavy lifting, engineers can focus on creating apps with a stellar user experience and performance.
AWS
AWS is the largest cloud provider and introduced serverless compute in the form of AWS Lambda in November 2014. AWS has managed to keep its lead in this field by introducing new services and improving existing ones relentlessly. Amazon is known for their focus on their customers. Not driven by sales targets, their solution architects have KPIs that are based on how successful you are. Even stronger, these architects are said to have bonuses paid out to them when they help their customers save money. Pair this with their self-service capabilities and pay as you go fee model to build autonomous teams that create rock solid software that scales from zero to millions of customers with ease.
The AWS Serverless toolbox can help you create almost any product or component you want. Use AWS Lambda to run your Node.js code, S3 to store and serve files and websites and DynamoDB as your single source of truth NoSQL database. Use Step functions to manage the workflow of your applications, Cognito for your IAM and CloudFront as your CDN. Use Route53 to manage your domains, Api Gateway to power your REST APIs and AppSync to do so for your managed GraphQL APIs. Use CloudWatch for your logging purposes and SES for your emails.
React
React is the most used frontend framework to power web applications. Uniquely it also allows the creation of native mobile applications with React Native and Expo. It has inspired frameworks to embrace the unidirectional data flow and component driven development. Unlike its alternatives it embraces JavaScript at its core and has a very small footprint.
You start at T Minus Zero in the cloud.
Werner Vogels coined the term “Undifferentiated heavy lifting”. It refers to all the work engineering teams have to do before “lift off”. Work that needs to be done by all companies before they can start working on code that differentiates them from the others. In other words, engineering work that keeps teams busy without actually adding business value.
With the SJGAR stack you are building off existing rock solid platforms and the work of thriving communities.
JavaScript and AWS Serverless help you to outsource a large part of operations, performance engineering, reliability engineering, security patching and scaling to name a few. Hit those nines of reliability without having to do anything yourself.
A lightweight core will help you perform better.
The JavaScript runtime is lightweight. Start testing something on a developers machine within minutes and get continuous feedback on every change you make within seconds. Upload your code to Lambda and enjoy lightweight services that quickly start and auto-scale. The AWS Serverless infrastructure uses Linux to run its workloads.
Learn once, apply everywhere. Conquer the Web, Native Mobile, Devices, Cloud and Desktop.
JavaScript and React help you to “learn once, apply everywhere”. With JavaScript, you build code that runs everywhere. In a browser, on servers or on devices. Reuse patterns, libraries or code over the different environments. React helps you write apps that run in the browser, as native mobile apps or desktop apps. On top of the earlier mentioned reuse possibilities, also reuse the Flexbox layout system over the different environments. Layout work is complex and time-consuming and being able to reuse your knowledge and tools delivers a great productivity boost.
Build world-class apps and APIs without re-inventing the wheel.
Combine JavaScript, React and GraphQL to build world-class apps. Adding GraphQL to the mix unlocks access to a rich community and tool set when dealing with server client communication. Don't re-invent the wheel but use the Apollo tool set to implement complicated elements like caching, optimistic UI, state management and subscriptions.
In the opening paragraph I stated that the SJGAR stack can be seen as a spiritual successor to the MEAN or MERN stack. Why is that? The SJGAR stack builds on the concepts of the MEAN stack, but importantly adds guidance on where to run your code and databases — use a serverless architecture on AWS. Next to this it emphasizes the usefulness of a GraphQL API to power modern web and mobile applications.
So here you go. I have put my thoughts on paper. And now the thing I am looking forward to the most is your feedback. For me to learn, and for us to have meaningful conversations.
I have created a website on sjgarstack.org to document more about the why of the stack. The website itself, even though very simple, was built using the SJGAR stack. Starting with a Gatsby React web app, which is deployed on AWS S3, Route53 and CloudFront. When adding more functionality I probably will throw in a GraphQL powered API running on AWS Lambda and Node.js to power the static and dynamic parts of the website.
I have ideas on including related external blog posts, adding templates or getting started articles on the SJGAR stack. First I will work on getting the initial content somewhat complete.
Your stories, experience and thoughts please.
If you are reading this on a website that supports comments, please use it to voice your thoughts. You can also reach me at my gmail address which is a rather boring combination of my first and last name. If you have been using the SJGAR stack for your projects at work or at home please drop me a message. I am very curious to learn what worked or didn't work for you. Also, if you have ideas about what could be added to the website, please share your thoughts.
I am looking forward to hearing from you.
But what about Vue?
Vue is great. You should definitely use it. There are four major reasons why the SJGAR stack chooses React over Vue.
- React embraces JavaScript at its core. People have described React as bringing HTML to JavaScript, where Vue is bringing logic to HTML. The SJGAR stack is centered around JavaScript and choosing React allows more of the 'learn once, apply everywhere' approach.
- With React Native you are able to build native mobile apps with the same skill set and engineers.
- React is the most used frontend framework, making it easier to find talent.
- The React core is small and stable which positively impacts its potential lifetime.
But what about Azure?
Azure is great. You should definitely use it. There are three major reasons why the SJGAR stack chooses AWS over Azure.
- The AWS Serverless services enable a great of the box performance for your products. Zooming in on the example of cold starts look at the following articles Comparison of Cold Starts in Serverless Functions across AWS, Azure, and GCP and Visualizing cold starts.
- AWS is famous for their focus on their customers. There are many anecdotes, both online and offline, of how this can be of benefit.
- AWS offers a great developer experience and large community. Deploying new versions of your code can be easily done without worrying about downtime for your users without have to do any engineering on your own side. Also tools like the Serverless Application Framework pioneer on the AWS side.
But what about Kubernetes and Docker?
Kubernetes is great. You should definitely use it. There are some clear examples where serverless compute may not work or does not make sense for your use case. When writing expensive processing jobs that could take up to an hour or more to complete, it makes sense to use worker nodes in the Kubernetes cluster. Also, when operating at a huge scale for a service it may become cheaper to run on Kubernetes. In general the advice would be “Use serverless unless...”. There are two major reasons why the SJGAR stack recommends to use Serverless over Kubernetes as much as possible.
- Kubernetes comes with more operations and engineering work for infrastructure, performance, reliability and security. Also when using managed clusters like AWS EKS or Azure AKS.
- Kubernetes is a field of specialism on its own and will most likely require specialized engineers.
But what about Flutter, Svelte, Rust, Go, Swift, Clojure, Angular, ...
All fantastic options.
Currently, though, when compared on the axes described in this document they are not there, at least yet. It could be a lack of maturity, limited community or less 'Learn once, apply everywhere'.
But what about vendor lock-in?
This one is a bit harder than the other ones. What if you put all your eggs into one basket? What if you go all in the integrated serverless services of AWS? What if you have to move out? It could be because of government rules, or for instance if Amazon starts operating in your industry.
One approach that is sometimes considered is using open source components that can run on multiple clouds. To quote Gregor Hohpe from his article Multi Cloud Architecture: Decisions and Options:
While this works relatively well for pure compute (hosted Kubernetes is available on most clouds), it may reduce your ability to take advantage of other fully managed services, such as data stores or monitoring. Because managed services are one of the key benefits of moving to the clouds, you need to consider your options carefully.
There is no easy answer, and it is about making trade-offs. One thing is for sure. When going for the managed cloud services you will be able to do so with a better time to market. You will at least bring something to your customers.
For further information you can also read the following article by the same author: Don't get locked up into avoiding lock-in.
Vijai Ramcharan has been working in the software engineering field for over twenty years. He is passionate about using software technology to put a smile on the users face, and help them get something done. He likes to peel away layers, both in software architecture and companies, in order to simplify things and get stuff done.
Currently, as an innovation architect, he is helping to build a highly talented team at NN that is at the heart of a technology infusion for the company. The team is helping the company to transform into a fintech and serve their customers even better. The group is bringing working software, a helpful attitude and the spirit to learn from each other.
If you are a software engineer or architect interested in what they are doing send out an email to vijai.ramcharan @ nn-group.com. You can also find this post on the NN Tech Blog.
Besides working on software, Vijai loves to spend as much time as possible with his wife Sharmila and five-year-old son Jayce. Together they like to travel around the world, where they enjoy the sun, local food and culture.