How was this blog made?

10 July 2021

It is fascinating to observe how technology evolves, and it is irresistible to play with them! That is why every few years some of us feel the urge to completely rebuild our blogs.

In this latest iteration I wanted to shift from ghost to a static site generator, but I ended up building a complete state-of-the-art serverless CI/CD system! That last ghost blog was run on a Google Compute Engine, in the usual configuration: served by Nginx, using mySQL and an SSL cert from LetsEncrypt. However, to update any of the components (and you should keep things facing the seething internet updated for security), I’d have to take the site down.

The static site generator I chose is Hugo. An interest in Go-Lang and shiny things tipped my hand.

In the first iteration, after converting the site to Hugo and setting up a theme, I ran hugo to generate the website on my local machine. I containerized Nginx in Docker and simply copied Hugo’s public/ directory to the correct location within the Docker container, then deployed it on Google Cloud Run.

The Dockerfile is literally 4 lines long and supported by an Nginx template with 1 additional line added to the default to explicitly tell Nginx to listen to a port (to abide by Cloud Run’s contract).

# Dockerfile
FROM nginx
COPY public /usr/share/nginx/html
COPY nginx/default.template /etc/nginx/conf.d/default.template
CMD envsubst < /etc/nginx/conf.d/default.template > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'

Then to build and deploy is a single line each:

$ gcloud builds submit --tag gcr.io/<project-name>/<container-name>
$ gcloud beta run deploy static-google-cloud-run --image gcr.io/<project-name>/<container-name>

It is an attestation of the advanced technologies today, where so much of the underlying processes and logic have been abstracted away, that the above is possible. Cloud Run even handled the SSL certificates for me automagically. This is indeed a golden age of computing.

The site was deployed but there was a problem. To update my blog I’d have to destroy the old one and push a new one to Cloud Run. (Let’s not even discuss the fact that the solution has not been optimized: for instance the Nginx could run on Alpine Linux which would optimize persistent storage for the container.) And so I started thinking about CI/CD.

I could use Cloud Run with GitHub triggers, but I did not want to add the complexity of another 3rd party platform even though it looks fairly tightly integrated into Google Cloud. The solution I settled on is a serverless (microservices) architecture, specifically Google’s Firebase. The tight integration with GCP means I can use familiar Cloud APIs such as Cloud Build.

The inspiration came from Jeff Levine’s (a Google Customer Engineer) blog. In essence his suggestion was to use Firebase with Google’s own Cloud Source Repository, and trigger builds that way. Static sites play extremely well on Firebase.

But there was still a piece missing. For true CI/CD, I’d want to be able to fetch the latest versions of software in my containers as part of the workflow.

It took me some time to figure it out. The resultant workflow in my cloudbuild.yaml looks like this:

Check the version (in my project’s Google Container Registry (GCR)) for both Hugo and Firebase tools.
Download the latest Docker image of python from docker.io as we use gitpython downstream. Run a python script (based on a GitLab example) that checks if our GCR versions above is out of date against GitHub (for Hugo) and npm (for Firebase tools). If no, exit, if yes pull the Docker files and update our Cloud Source Repository (CSR).
Run a test on the Hugo image to check if it works. If so, run the Hugo command “hugo” to build the website in public/. Test that the website has been generated.
Run Firebase tools in another container and deploy the new website build to Firebase.

Like Cloud Build, deploying to Firebase is a single line of code. Here is an extract from my cloudbuild.yaml:

# Part of cloudbuild.yaml
- name: 'gcr.io/$DARKMATTER_PROJECT_ID/firebase'
	args:
	- '--project'
	- '<project-ID>'
	- '--non-interactive'
	- 'deploy'
	- '--only'
	- 'hosting'
	- '--message'
	- 'Build $BUILD_ID, repo $REPO, branch $BRANCH, sha $COMMIT_SHA'
	secretEnv: ['ENCR_FIREBASE_TOKEN']

The process is actually more nuanced, for instance, nothing is done if we detect that we already have the latest versions of the software, saving compute and storage. I also use Google’s Cloud Key Management Service (KMS) to pass around my secret Firebase Token through a few lines in the cloudbuilder.yaml:

secrets:
- kmsKeyName: '<path/to/cloudbuilder/cryptoKeys/firebase-token>'
	secretEnv:
		ENCR_FIREBASE_TOKEN: '<encrypted-firebase-token>'

To encrypt your firebase login:ci token, follow the suggestions in Cloud Builders Community here.

And finally, I set my build trigger as a specific git tag. This is to allow me push changes without building a new Firebase site and affords more control than just triggering off changes to the Cloud repo. Plus a gut feel tells me that just building based on any repo change could cascade into a loop if I am not careful.

So my Hugo workflow now is as simple as:

Write on local Hugo, running “hugo server” to get real-time local feedback on changes.
When I am ready, I commit the changes and tag the master head:

$ git tag -a <trigger-tag> -m "new posts"

and

$ git push origin <trigger-tag>

Now that is a full CI/CD workflow! Firebase takes care of SSL certificates, scales up if this site gets slashdotted, or drops usage to zero if it’s a quiet day.

It is truly remarkable that this kind of resource is available to anyone with a computer and an internet connection. Not too long ago these kinds of resources would only have been available to the enterprise. In addition the tools are mature and sophisticated enough that anyone with experience in git and Docker should be able to pick up.

The Hugo theme is based on Alexandre Vicenzi’s soho theme. Some elements you see, such as transclusions, are custom Hugo shortcodes that I created. Different sections behave in different ways from customization of Hugo layouts.

If you are interested in how all this is done in nitty gritty detail, drop me a line and I might write a post about it.