April 2020 Notes: Hugo, JAMStack, Dart and more

March 2020 was a good month. I finally got a personal blog running and figured out how to use Hugo. I was struggling with generating content and can’t even put aside some time to build a portfolio page. I did a lot of work in the background though, so I’m quite happy in terms of personal development and getting something published is a cherry on top.

April 2020 was a brewing month. I built and learned tons of new things but don’t have anything yet to publish, be it blogpost, page or product. In brief, this month I tried building an app in a static web (the JAMstack thing), managed to setup a working environment for it but decided to go with a native app for now.

After many unfinished in-depth blogpost drafts, I decided maybe a quickbites is more doable. After all, quantity matters when you’re still sucks, so I think it’s better to have something out consistently for now. It’s not pretty for public consumption so I put it on Personal category. Always happy to chat if anything interest you.

JAMstack app with Hugo and Dart

When you access a website, usually a server app processes your request by collecting data from various sources before presenting them to you for your personal pleasure. I worked in backend of an ecommerce company before, and witnessed first hand the staggering vast intricacies behind an innocent product page.

JAMstack is basically an easier web. Instead of collecting data whenever someone asked for it, why don’t we just prepare all the pages beforehand? After all, in a typical apps, 99.9% of the time people are viewing something instead of changing something. Why spent most of the computing power to solve what is basically repeatedly processing the same thing?

JAMstack approach comes with shocking benefits. First of all is speed, you get what you want without no processing from CDN. This is already one less “hops” than typical server behind load balancer setup. Second is all changes are atomic. Every change is a git commit, thus you can view its history, roll back change, use different branch for different environment and so on.

I was quickly sold on those ideas (also, that’s how we do our work in BBC Visual Journalism). I built this website using Hugo, a static site generator (SSG, basically the tool to do prepare all the pages bit) and set an AWS Amplify project to automatically publish change whenever I push a commit to master branch in Github. The Hugo documentation could be better, but the concept is way easier to comprehend than Wordpress, for example, so I find it really easy to customize and work with.

Hugo works with special folders and naming conventions instead of a huge configuration. Finding what those names mean takes time, fortunately there’s not much of them.

One interesting concept that Hugo does is the content of theme folder is actually a full Hugo folder with same special folders and naming conventions, and you can have multiple of them! By using this concept, I managed to keep my top directory to store only contents (blogpost, images) and implement other features inside different module folders. This is super neat because the module folder is also a git repository. Therefore my main folder git history only contains content changes and module updates, while each module has their own history.

Git submodule is software engineer version of Marie Kondo organizing.

Adding dynamic content

Some parts of a web app are ever changing, for example a comment section or an ads box. Those parts can’t be prepared beforehand. Ads, for example, need to know roughly who you are before showing up in your screen. Fortunately, they are not important for the most important reader in the world, the Google Search Bot, therefore there is no need to prepare them beforehand.

But we still need to present them at some point to lowly humans, and that’s where the JA in JAMstack come into play. I can’t speak much about what kind of project I’m working on (not an ads, at least), suffice to say that interactivity is the meat of it. Thus I need to build those on top of Hugo.

To build interactivity, I decided to go with Dart instead of the usual JavaScript (making it DAMstack). I need strongly-typed language at all cost because unlike work project, I don’t have resource for rigorous testing. With strongly-typed language, I can focus on unit testing behaviours because the type system already enforces some degree of correctness.

I was going to go with TypeScript and Webpack at first, but decided I can’t afford to spend time with high configuration system like Webpack (also there’s some broken pieces to make it work with TypeScript smoothly). I gave Dart a run few months ago when learning Angular and it was good so I gave it a try again, this time without Angular. It turned out things like installing packages and starting project is faster than npm (perceived speed, might be because it’s Google project and Google has datacenter in Singapore). I’m also sure that compilation will be faster than Webpack or alternatives simply because there’s less “build steps” with Dart, it simply compiles Dart code to ES2015 JavaScript.

Dart development in Hugo module

This was my first couple of weeks of April 2020, trying to run Hugo hot-reload-blazing-fast server side-by-side with Dart compiler.

At the beginning, I tried to run Dart’s webdev package on watch mode and set it to write the output to Hugo’s static folder. It’s right away not great because Hugo now serves thousands of build artifacts scattered inside static folder. Also, webdev watch mode crashes a lot on my Windows 7 PC whenever Hugo tries to copy things.

After some tinkering, I tried to put webdev output to assets folder and load the JS file as a resource. This solves the thousand files to serve issue, but I got an error because the Dart’s JS output actually loads a couple of other JS files and expects them to be at the root directory of the website. I need them to be at Hugo’s static folder because that’s the only way to put them at the root directory.

Finally I found the most natural solution. I simply start a Dart project as usual and put them on a separate repository, then add to the website as a Hugo module. In Hugo config, I picked the JS files to copy into static folder, thus Hugo no longer serve thousands of build artifacts. On development, however, I simply set the script tag to point to webdev development server, essentially running two dev servers at once. This magically works, whenever I make any change to either Dart code or Hugo content, the page gets reloaded automatically.

This is great and I was ready to churn Dart codes to implement my app. Right before I went on a week leave to do the big work, one thing bugged me.

I know I can make this work using JAMstack, but suddenly I’m not sure if web is the right platform for the project. I can’t speak much about the project I’m setting myself to work on (it’s not an ads, at least), but copyrights is a huge conundrum, and I think the web is simply too open for this. I can hash the API content, but eventually the content will become text in DOM, and it’s just way too easy to retrieve those in browser, not to mention the deciphering logic can also be found and reverse engineered quite easily from browser.

I spent a whole weekend thinking about the alternatives, e.g. creating the project on HTML Canvas and put the logic in Web Assembly module, they all sounds too infeasible. Thus the only way forward now is a mobile app.

Last couple of weeks in April was spent to work on this. And it’s quite interesting for me, I finished a Redux pattern in C++ for the app and currently working on the UI itself. Alas my annual leave is gone now, so I’ll at least write down that bit in the part two of April 2020 Notes. Hopefully that helps to fuel works on May!

Keep striving!