Over the past couple of years I’ve had plenty of time to consider what technology use to build Sharetheirday.
I’ve been lucky enough to work with a variety of languages & frameworks suitable for building web APIs including a number of JavaScript servers like Express or Koa, Ruby on Rails, Django, Flask, and back in the day, good old ASP & PHP (shudder).
For the front end, I started working with HTML & CSS decades ago, and for interactivity I’ve got a lot of experience with vanilla JavaScript, through the jQuery years, to the modern glut of JS UI frameworks.
When thinking about the tech stack I wanted to use for this project I started with the question of whether I should go with something I’ve used before or try something new.
I built BorrowMyDoggy with Rails but my day job since then has been full-stack JS focussed, so after a bit of thinking about what my requirements looked like I decided to go with JS.
Context matters
For this project I’m pacing myself not aiming to build it in a weekend (obviously!) and I’m trying to keep the costs as low as possible without going too crazy.
I’m also trying to keep it as simple as possible so that it’s easy to onboard other developers if I need to in the future.
JavaScript has improved a lot since I built BorrowMyDoggy, there are mature technical ecosystems built on it, and I know a large number of JS developers.
It also means that I can use JS in the frontend and the backend of the application - although that’s potentially just laziness not wanting to context switch when dealing with the API & the UI codebases!
As much as I love the simplicity of Rails, and how easy it is to get up and running, it felt like I would need some time to get back up to pro-level speed, and as I use JS every day for work, it made sense to go with that.
No deadline = going nowhere, quickly
Even though my previous thought was about getting up and running quickly, aiming to get the initial version of the API done without a specific deadline gave me plenty of time to spin my wheels, and try various ideas around solutions.
Spending a few weekends trying out serverless solutions on AWS (because that’s what everyone’s doing these days for web-scale, right?), and struggling with a list of issues (mostly getting things to run locally with any kind of consistency - things have moved on & there are more tools for this now), I eventually realised I was falling into the trap of pre-emptive optimisation.
It doesn’t make sense to worry about how many users I can support when I don’t have any at all yet. I’m still going to keep those lovely masses of people in mind as I make progress though - it’s just not the primary focus.
Future proofing is a myth, or is it?
I spent a little time considering how to prevent major rewrites in the future. Over the years a large number of projects I’ve been involved in have been dealing with & rewriting “legacy” systems that were built only a few years previously.
Quite often this has been down to the original design of the systems and how tightly coupled they were, making them hard to adjust or extend.
One of the newer JS frameworks on the scene that takes up this challenge is called Fastify.
Of the things I initially found attractive about Fastify was that it uses a concept of plugin architecture that means you can build a server or API in a way that enables you to split out parts of the application so you can scale them independently as needed.
It’s also super fast, which made it even more appealing - who doesn’t love a bit of speed? Yes. I know. Pre-emptive optimisation.
I thought this sounded like a great option to get going with.
Sometimes going with what you already know is ok
Making decent initial progress with Fastify building out the initial routes I got the API to a point where I could login new users and create various resources.
I soon found I didn’t get on with the plugin architecture, at times struggling to know where the best place was to locate parts of the system. As Fastify was relatively new at the time it was tricky finding best-practice examples.
I find this a common issue with a lot of online resources for JS frameworks - lots of hello world examples, but very little in the way of how to build robust production ready applications.
After going in circles with a specific issue one weekend, I cast my mind back to when I’d been leading a team building large scale applications for Science Direct.
Although as an engineering department we’d been using Hapi, a JS API framework created by the Walmart web application teams, we’d done a bit of research into Koa another JS framework.
Koa is built by members of the original Express team, and contains some fundamental differences - including being several times faster than Express.
It also benefits from using a similar middleware architecture to Express, which means a lot of the things you need to solve with a framework have been covered by a lot of people in the Express ecosystem - meaning it’s easier to find documentation & online resources that help with issues.
After chatting with Michael, my friend now helping on the project and part of the Science Direct engineering team, we decided that it was probably worth investigating the effort needed to switch to Koa for the API.
Going backwards to move forwards
In one weekend I was able to replicate most of the functionality that I’d built in the Fastify version and the codebase was now in a much more familiar structure.
Since we made the decision to switch progress has been rapid. Both of us are able to contribute to the API development and coding new features has become much easier; testing & communicating the changes has improved a lot too.
Problems that have arisen have mostly been around finding out-of-date solutions for Koa issues but they’ve been mostly straightforward to solve as the pattern & concepts are more familiar.