They static now? They static now!
27 February 2024 ·Origins
When I started this site back in 2014, I only had a couple of years of experience writing Python, but I wanted to try writing a real (meaning “dynamic”) site, so I went with the Flask framework.
The site started as a single Python file and a few static CSS files and Jinja2 templates. When I did the conversion this weekend, it was still a single Python file, a few static CSS files, and the same Jinja templates.
My early inexperience resulted in a site with some odd design decisions, and in the first year or two I added features (like an RSS feed) by adding more functions to my single file. Don’t get me wrong, it’s cool that you can write a real web server so easily, but the result was an unmaintainable mess with some very questionable decisions.
One early decision I made was to avoid using any kind of database. My
blog posts were (and are) simple Markdown files. To add a new post to
the site, I simply copied it to a directory on my server and the Flask
application did the rest. I went with the easiest possible solution to
most problems; for example, the URL paths to individual posts were
simple numbers counting up from the first post, /post/0 for example.
This led to problems, because the post numbering is just determined by
the .md files the Flask application finds. Flask is not an
especially friendly framework for, say, running a background task at a
specific interval and setting a global variable containing the markup
for all the blog posts it finds. The easy solution is to look up the
files every time someone visits the page, so that’s what I did.
Even worse, there’s not a one-to-one relationship between the post
paths and the Markdown files on disk. If someone sends GET /post/5
then what post does that represent? For all the Flask app knows, a new
post has been added with a modification time in between what used to
be posts 3 and 4, so the only thing to do is reload every single post
on every single page view.
I partially rewrote the application in 2019. Because the code had turned into such an ugly mess, it wasn’t fun to work with, so I once again turned to easy solutions. I put a bunch of caching in front of the heavy calls, so that only the first request for a post would be slow, and then the rest would be fast, inside the cache window. I also finally switched to URL paths based on the title by default. This still didn’t provide a one-to-one relationship between URLs and Markdown files (since characters that aren’t safe get removed), so this didn’t solve the underlying issue of having to fetch all the posts on every page view.
For a long time, I intended to migrate the blog off the old server (running Ubuntu 18.04!) to a new one. Over the weekend, I finally got around to this, and in the process fixed the issues with the site.
Changes
My criteria for the transition were the following:
- The site must not visually change.
- The site should be static HTML.
- The static HTML for existing posts must not be created via scraping the old site and saving the files.
- The process for writing new posts should be relatively painless.
The original development of the site, as I have said, was a learning opportunity for me. I’ve made other (and much better) sites since, and these don’t have quite so many problems! I was stuck on the old site primarily because the obvious transition (to static HTML) seemed so difficult given my requirements.
When I finally found the time, I decided to try an interesting
approach. Rewrite my existing code once more, to remove bugs, and also
remove the dependency on Flask. Instead, I have a main() function
that collects changes to the source files and renders them via the
same functions called by Flask. The resulting HTML is then written to
disk in a directory specified by a config file, and I rsync the
result to my server.
Overall, this turned out to be way easier than I expected, and in fact I fixed quite a few issues with the legacy code in the process.
Perhaps most importantly, I decided to just use Jinja2 directly. This allows me to use my existing templates, meaning that the site doesn’t visually change at all. I performed some cleanup, but in other respects this required minimal work.
You can replace Flask’s ubiquitous render_template with a simple
shim:
def render_template(template_name, *args, **kwargs):
template = JINJA.get_template(template_name)
return template.render(*args, **kwargs)
JINJA, here, is just a global Jinja environment.
Perhaps most importantly, I’ve developed a workflow for shipping Python apps that I’m pretty happy with. With my deployment scripts, I can recreate my sites on a new server in a matter of minutes.
I’m also no longer relying on file modification times for my “last updated” dates on posts, thanks to the meta extension for Python-Markdown, which I actually added back in 2019.
Anyway, this update isn’t terribly interesting to most people, I’m sure. I’m writing this to reward myself for getting it done. I do think the idea of rewriting existing server code to just render templates is a good one, so if you’re in a similar situation with your own site, perhaps give that a try.