Modern web is a mess. That’s why I think I loved the idea behind 1MB Club, and the offshoots 512KB Club and 250kb Club. They are lists of websites where the pages are under 1MB, 512KB, and 256kb respectively.
I had two main goals with this project. First, adding a new website should not require someone to go through the cumbersome process of forking the repo and sending a pull request. Second, ongoing maintenance of the website should be reduced as much as possible.
As you can guess, there’s an xkcd for that:
While I don’t expect the project to go viral, it was a fun learning opportunity. I decided to fully operationalize it through Github: Github Actions for adding new websites to the list, and Github Pages for hosting the resulting webpage.
If you are not familiar with Github Actions, this is a good intro from Github docs:
GitHub Actions help you automate tasks within your software development life cycle. GitHub Actions are event-driven, meaning that you can run a series of commands after a specified event has occurred. For example, every time someone creates a pull request for a repository, you can automatically run a command that executes a software testing script.
The full flow in nojs.club is like this:
Someone opens an issue suggesting a new website. The issue must follow the set template.
A Github Actions Workflow kicks off and
a. Extracts the domain of the request (code ref)
b. Find any use of
<script>tags on the web page. Good ol’
grepcome in handy here (code ref)
Once that happens, someone (me) must manually approve the request. I must post a comment in the format
/approve example.com 123.45 where the numbers refer to the total size of downloaded assets in KB. Another workflow then kicks off and adds the domain to a CSV file in the same repo (code ref).
Note: While I can also automate the approval itself, I did not want to risk adding bad data to the list.
Excluding the latency for me to respond, this takes under a minute.
When a new commit is pushed to the repo, a different workflow builds and pushes the Jekyll-based site to the
gh-pages branch. That process takes a couple of minutes.
While the resulting process and code is relatively easy to follow and reason about, it did take a lot of head-banging for me to get it working end-to-end.
The default shell that Github provides fails fast:
-eo pipefail. What that means is that any command with a non-zero exit code will fail the job.
Little did I know that
grep by default returns an exit code of
1 when no match is found:
$ echo "hello world" | grep hi $ echo $? 1
The bash script in a fail-fast shell, thus, ends up failing with an exit code of
1, and no useful logs.
No local dev environment
The lack of a full-fledged local dev environment for Github Actions meant that my development flow was slow and unpredictable. I would modify the action, push it, then test using real issues. This results in so much time being spent just waiting to test.
I would urge Github to put out a real test environment that stubs events.
The website is built using Jekyll like I mentioned. Because I wanted the list of websites to be sorted by size of the web page, I had to do an in-memory sort (code ref). That worked well locally, but not when rendered by Github Pages.
After some digging, I found that Github uses Jekyll 3.9.0 while my local install had 4.0.1. That seemed like a likely culprit, and as I found was widely reported. Github has shown no interest in upgrading Jekyll, so the workaround for me was to build the Jekyll site on every commit using - you guessed it - Github Action (code ref).
Trigger action on commit
When a commit is made through a Github Action, by default the resulting action does not trigger any Github Action Workflows. As Github explains:
When you use the repository’s GITHUB_TOKEN to perform tasks on behalf of the GitHub Actions app, events triggered by the GITHUB_TOKEN will not create a new workflow run. This prevents you from accidentally creating recursive workflow runs.
This is a good default. However, this means that when my Github Action adds a new website to the CSV, it does not trigger a Jekyll build. A simple solution for this is to use a personal access token instead of the
GITHUB_TOKEN (code ref).
There are of course many ways to implement a similar project, some better than others. In the end, this was a good project to learn some new skills, build something that aligns with my principles, and join a
viral trend fad.