Automating Blog Posts

WordPress wasn’t working for me. I wanted something cleaner and lighter weight. If it was easier to produce content, so much the better. I found almost all that I needed with the static site generator, Hugo. I say Hugo was almost all that I needed because I still needed some way to automating publishing the blog posts the way WordPress and other providers can. Hugo converts markdown files to HTML. Obsidian, which I use for taking notes and writing, stores its data in markdown files. There had to be a way to make that work for me. It turns out there is. My thanks to Network Chuck for the inspiration.

Overview #

Markdown is a lightweight markup language that allows users to format text using simple, readable syntax. Markdown is easy to learn, readable in its raw form, and platform independent. If you (or an app you use) create files in markdown format, you can take them anywhere. Another advantage of Markdown is that it enables writers to create formatted documents using plain text editors without needing to know complex HTML tags.

Lots of websites and applications use markdown, including GitHub, Discord, and, of course, Obsidian. I can draft new posts in Obsidian using markdown, which I would do anyway. The basic work flow would go like this:

I could manually copy over files from Obsidian, activate Hugo to convert them, and upload them to my site. But why in the wide. wide. world of Linux would I do that when I can make it happen automagically?

How it Works #

As Chuck identified in his video, there are a couple of issues to deal with. One is getting the files from Obsidian to Hugo. That’s easy (he and I use rsync). The other is dealing with images. Obsidian stores images in its data vault, so its links are to that vault. Those links, when translated to HTML by Hugo, will reference images that don’t exist in Hugo’s world.

My automated workflow uses a bash script and a python script to process a markdown file (produced by Obsidian) to make those image links Hugo-friendly and copy the updated markdown file and the images to a new sub-directory in the Hugo project directory for the website. I expanded a bit on Chuck’s concept. The expanded scripts are available on GitHub.

Cover and Thumbnail Images #

The Hugo theme I use, (Congo), will create dedicated cover images and thumbnail images for articles if they are provided. To do this, Hugo looks for images with the ‘cover’ or ’thumbnail’ in the name. I wanted to be able to provide these along with the post content directly in my Obsidian note. Otherwise, I’d have to add a manual step to the automated publishing process, which defeats the purpose of automating it in the first place. To do this, I put the images in my note with special formatting. The Python script looks for an image that is embedded in the markdown file using the format [image.ext|cover], moves that image to the subdir as cover.ext, and removes the reference to the image from the markdown file. It does likewise for the thumbnail image, looking for [image.ext|thumbnail].

Front Matter #

Front matter is a metadata section at the beginning of a Markdown file that contains information about the document. In the case of Hugo, the front matter contains the title of the article, the post date, categories, tags, a summary, and data, which it uses to create the HTML files. Rather than entering this section by hand each time, I created a template in Obsidian that automatically creates the front matter for a new blog post note by prompting me when a new note is made using the template. This ensures that the front matter always includes the basics I need and is formatted correctly.

Publishing a New Post #

To publish a new blog post (including this one!) using these scripts, I:

1. Draft the post in Obsidian, adding cover and thumbnail images (those I create using GIMP).
2. Run the bash script, which: a) Moves the Post notes from the Posts directory to the posts subdir in my Hugo site’s directory; b) Calls the python script to analyze any .md files at the root of the posts directory, make a new subdir in the Hugo posts directory for each post, take care of the images as noted above, and move the markdown files from posts dir to the new subdirs, renaming them to index.md; c) Calls Hugo to build the site locally, which converts all the markdown files to HTML. Hugo also works a lot of other magic; d) Calls git to commit and push changes to a private github repo for the site; and e) Calls rysnc to publish the site to the webserver (in my case, a VPS running apache in a docker container).

I sync my Hugo project files to a private GitHub repo so I have a backup of the site and a history of changes.

The flowchart below lays it all out.