Eddie Hinkle suggested I write up how I use Amazon Web Services (AWS) to build this website, so here’s a high-level walk-through.
As a warning to someone who is a bit new to this stack, this may seem like a lot of acronyms or terminology - it is but I’m not sure how much of that is avoidable from AWS’s side of things. Just break things down and do one part at a time and it is not so bad - I find the responsibilities are logically separated.
I did find a lot of tutorials online, but many of them had outdated screenshots of the AWS console interface (missing fields or things that had changed in how they worked.) That said, these were generally the most helpful and should be read before hand:
CloudFront as a content distribution network to speed up content delivery and serve up the contents under SSL
Other factors: I’ve been using Hugo for years and I was already very familiar with AWS S3, AWS CloudFront, and have a deep understanding of HTTP, Content Distribution Networks, Caching, DNS, etc. It does not necessarily matter for this setup but I work entirely from an iPad Pro and this configuration works quite well with my development and writing workflow which relies heavily on Working Copy, Drafts, GoCoEdit, and Siri Shortcuts.
Why CodeCommit over Github or Bitbucket? No reason other than I wanted to use AWS exclusively as a learning exercise. CodeCommit’s free tier is rather generous - private repos, 50gb, 5 users limit. More than enough for my needs.
For CodeBuild, I choose to use the Ubuntu 14.04 “long term support” image. It is not ideal from my point of view to be using an image that will no longer receive support from Ubuntu by next year (2019) but that’s life.
I have a CodeBuild “buildspec” file in my CodeCommit repo that defines the build steps I want to have happen after I push a commit to the repo. It is important to remember that CodeBuild has everything from Ubuntu 14.04 (and the AWS CLI tools) already installed when building. In this case the steps are:
Install Pygments for code highlighting via python’s pip tool.
Download and install the latest Hugo binary
Build the website using Hugo (no additional command line flags were necessary for my needs - everything is defined in my Hugo config file)
Use the AWS CLI S3 tool to sync content over to my S3 bucket. Make sure to use (and set a reasonable for your website) --cache-control setting here.
Use the AWS CLI S3 tool again to set a different --cache-control for my index.html and index.xml - basically a “no-cache” directive. This is important later for CloudFront to not aggressively cache files that will change frequently.
Use the AWS CLI S3 tool again to set a different --cache-control setting for content that is really old (things I posted years ago.) This way CloudFront can hold on to that content much much longer than I might want for my newer material where I might still be fixing a typo or adding an “update” that corrects things.
After the contents have been copied to S3, I issue a curl command to send a WebSub ping to micro.blog letting them know my RSS feed has changed.
Publish a SNS message that have an email subscription to so I can get notifications that my builds have completed.
This an optional step - you could just as easily point your domain name straight to the S3 bucket. I wanted to use SSL and it seems like AWS really wants you to use CloudFront for that need. As a bonus the content is cached and served up quickly to folks depending on their network’s physical location. Presumably if S3 is offline but CloudFront is not - basic HTTP GET requests could conceivably work as long as the file is still in the CloudFront cache.
Website Creation & Posting
Like any static website generator, Hugo defines certain folders and files it expects to find in order to generate your website depending on whether it is building an index page, a single page, or a list of pages.
Here’s what the workflow looks like for me then.
I design and maintain the website templates in GoCoEdit and commit the results to my CodeCommit git repo using Working Copy.
I write posts in Drafts and use a Siri Shortcut to automate creating a new blog post file in Working Copy as well as the git commit/push steps.
CodePipeline monitors CodeCommit & CodeBuild for changes. CodeBuild is ultimately responsible for pushing content out to my S3 bucket but CodePipeline is the glue that automates CodeBuild picking up on CodeCommit changes.
The Siri Shortcut I wrote does a number of “nice” things - it looks for input from an iOS Share Sheet:
website (from Safari)
If it finds no input it’ll give me a little text prompt to type out a quick entry but I mostly trigger this Shortcut from Draft’s “share” action - but it could be triggered from any app that integrates with the iOS share sheet. After that the Shortcut goes to work formatting a Hugo blog entry with all the correct “YAML” front-matter, dates strings, file names, etc. It does all the heavy lifting. This makes posting as frictionless as using Wordpress or any other fancier system, from my POV at least.
Tonight I dug into all the AWS toys I’ve been too busy to deal with. This means this site is now served off S3 and is built by AWS entirely without any servers on my end. It was also interesting (?) to experience the full AWS console user experience across multiple tool-chains.
The main problem seems not to have been the proliferation of the Semantic junk such as RDF/XML and SPARQL, as is sometimes argued (references below), but rather that the Web side did not provide an open and decentralised system in which to host the Semantic side. Ongoing efforts are being made to rectify that, but there are no promising solutions in that domain and so many of the Semantic Web ideas will remain dormant.
And so we come back to Solid, TimBL’s new creation:
Solid (derived from “social linked data”) is a proposed set of conventions and tools for building decentralized social applications based on Linked Data principles. Solid is modular and extensible and it relies as much as possible on existing W3C standards and protocols.
Solid then looks to offer a means of “hosting the Semantic side of things”.
I had lunch with TimBL once at a W3C gathering and the man is brilliant and it is easy to get swept up in his optimism. This was years ago though, before Facebook and Google vacuumed up the web into giant silos. Maybe an angry but boyishly optimistic TimBL will finally get the Semantic Web transparently into millions of people’s hands.
If nothing else all this Solid talk made me fondly remember the active days of the old swhack IRC channel and left me shedding a tear for Aaron Swartz one more time.
Since July we’ve been fostering a family of five kittens. Today they are off to the pet shelter to start the process of finding their “forever homes.” It was quite a wild ride to watch them grow up together and I’ll miss their playful exuberance when I’m working at home.