Adam Fontenot

Fixing a broken QR code in FEZ

2021-01-13T05:20:19+00:00

I’ve been playing FEZ for the first time recently, and came across this room where some NPCs are constructing a QR code on the wall:

This seemed very likely to be a puzzle. It’s almost a complete QR (most of what’s missing is the top left positioning square), and anything under the scaffolding can be reconstructed, so it stands some chance of scanning after a few fixups.

I imported the screenshot in GIMP. I combined it with another screenshot of the same scene to remove the characters, leaving only the scaffolding. I used the Color Exchange tool (Colors > Map > Color Exchange) to replace the lighter purple color in the background with white. I then used the Threshold tool (Colors > Threshold) to make anything else in the image black.

Then, in another layer, I created a grid precisely aligned with the blocks in the image to help me see and fill in the missing portions. GIMP has a tool for that (Filters > Render > Pattern > Grid). I replaced the partial blocks hidden by scaffolding with completed blocks. This result looked like this (the grid is at 25% opacity):

Unfortunately the result still wouldn’t scan, even when I added the third position symbol back to the top left. Too much seemed to be missing, at least for the apps I tried. But since this seemed to be a puzzle that the developers intended you to solve, I assumed there was something more here, so I broke out the QR code spec.

The first thing to understand about QR codes is that they’re relatively simple ways of encoding binary data (ones and zeros) using black (1) and white (0) blocks. Certain parts of the image are only used to align scanners: they don’t contain any data. Other parts contain format information: this tells a scanner what kind of QR code it is and how to decode it. I was hoping this QR code would contain valid format information, as otherwise I’d have to test all the possibilities. Here’s an illustration of the different parts of a QR code. Image credit: Wikipedia.

From the size (25x25), we know this is a V2 QR code, which doesn’t have version information. The first question to answer was whether the QR code still contained valid Format Information. This information is error corrected and duplicated within the image. It appears in the QR code in the sections highlighted in blue:

One of the Format Information bit strings seems complete: it combines the top-right portion with the bottom-left portion. I’m working with the assumption (that I might need to change later) that anything not in the top left quadrant is “finished” by the NPCs, meaning it’s trustworthy. Hopefully this means I won’t need to apply error correction to the Format Information.

Information in QR codes is masked (XORed with a known pattern) to break up large chunks of white or black that give scanners difficulty. The Format Information uses the static mask of “101010000010010”¹. Wikipedia incorrectly says this mask is “101011001010101”.² The data in the QR code will use a different mask, chosen dynamically by the encoder so as to minimize large white or black chunks.

Here is where I began to be surprised by how far I could go using only GIMP. I expected to be importing the data into Python relatively quickly, but being able to toggle masks on the QR code directly and observe the result visually turned out to be very handy.

Using the new “Exclusion” layer mode, you can selectively invert parts of the underlying image using a black and white layer. Take care to observe the correct bit order when creating the mask (MSB: for the Format Information, that means bottom to top, left to right), as seen here:

Applying the mask, we see the following:

Note that because of the way the GIMP Exclusion layer works, we have to use white as the “on” bit for the mask. Black would be more appropriate since that’s what the QR standard uses for an “on” bit.

According to the spec, the first two bits give the error correction level, and the third through fifth bits give the data mask information. That’s once again in MSB order, so bits 14 and 13, and 12, 11, 10, respectively. In this case, we have “11” for the error correction level, and “001” for the data mask. That tells us that we have error correction level “Q” (which we’re ignoring for now), and “i mod 2 = 0” for the mask (this is one of several options for a mask that is chosen by an encoder). This means that the mask is active (flipping the underlying data bits) for every even number row, starting at 0 for the top row.

We can quickly create such a mask in GIMP by making a small portion of it (just two blocks, in this case), then expanding that with Filters > Map > Tile. We then trim out any portion that is part of a position symbol, timing symbol, alignment symbol, quiet zone, or Format Information (so that only the data will be masked). The spec (and other references) are pretty clear about what these areas are, so I’ll skip to the result here:

We’re now ready to begin trying to read the raw data! The order of bits is detailed in section 8.7 of the spec. We start with the bottom right block, then go left, then go back and up, then left again, reading upwards in a step-wise pattern. When we reach the position symbol at the top we move to the 2 block wide column to the left, and this time step down. (Discovering that QR code data starts with the bottom right and that it ends with error correction data gave me hope that the QR code was decipherable.) The reading order for the intricate sections is given precisely in the spec. I’ve indicated the interesting parts below.

Section 8.4 of the spec indicates that the data stream is prefixed by a header, where the first four bits are a “mode indicator”, followed by a length indicator. If you’re just encoding raw binary data in a QR code, you may as well just use 8 bit bytes, which is mode “0100”. But if you’re representing numeric or alphanumeric data, you can encode it more efficiently. The FEZ QR code begins with “0010”, which indicates an alphanumeric QR code.

This means that the next 9 bits (for a V2 QR code) are used to give the number of characters in the data. The next 9 bits of our QR code are “000010111”, which is the number 23 represented in binary.

For maximum efficiency, alphanumeric mode encodes 45 possible characters into blocks of 2 characters per 11 bits. (45^2 / 2^11 ≈ 98.9%, which is pretty good efficiency). To access the characters, we divide the 11 bit integer by 45. The result without the remainder is the first character, and the remainder is the second character.³

Each number represents a character according to its position in the following string:

0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ $%*+-./:

Note that we have an odd number of characters, so to save every bit possible the last character will be encoded as a single 6 bit value, instead of wasting another 5 bits.

With all this information in hand, we can transcribe the data from the QR code. Anything after the last piece of data will be part of the error correction bits, which we’re hoping we don’t need. Now that we know how much data there is (22 × 11 / 2 + 6 bits), we can also identify each portion of the QR code by its function:

Blue represents Format Information (as before). Green represents data. Red represents error correction. Anything purely black or white is a static part of the QR code.

I have transcribed the data like so:

10011011100: 27 29 (R T)
11001101111: 36 27 (Space R)
10100111101: 29 36 (T Space)
01111001110: 21 29 (L T)
11001101111: 36 27 (Space R)
10100111101: 29 36 (T Space)
01111001110: 21 29 (L T)
11001101001: 36 21 (Space L)
10100111101: 29 36 (T Space)
01111001110: 21 29 (L T)
11001101111: 36 27 (Space R)
011101: 29 (T)

In other words, the data says “RT RT LT RT LT LT LT RT”. This is a sequence of inputs you can make with the controller in FEZ. You usually receive a collectable for doing so.

I also wrote some Python code to do the decoding automatically, which you can find here: fez_qr.py.

With all the information we now have, we can actually generate the final, corrected version of the QR code. To do that, we’ll need a library that allows us to set the parameters manually (because an automatic QR code creation tool might choose a different mask or error correction level, giving a completely different result). I used the one here. Exporting the resulting SVG and layering the result on the original, we can see that it’s nearly a perfect match!

Having finished all this, I reopened FEZ, and entered the input sequence. And NOTHING HAPPENED. I tried multiple times to make sure I wasn’t screwing anything up.

I looked up the puzzle online, and apparently a completed version of the QR code is in a later stage of the game, and this unfinished one is just for “lore”. I find this explanation frustrating, because it clearly seems to have been designed so that an enterprising person could reconstruct the code themselves. Everything you need is there. Yet I have to admit… it doesn’t work. You can’t enter the code in this room, you have to enter it elsewhere.

I think this is a mistake on the game designers’ part. If a puzzle gives you everything you need to solve it, the solution should work. It’s rather frustrating to put that much effort into solving a puzzle only to find out that your effort wasn’t anticipated, and the puzzle is actually solved for you later in the game.

According to section 8.9 of the QR code specification. ISO/IEC 18004:2000. ↩
As of 11 January 2021. https://en.wikipedia.org/w/index.php?title=QR_code&oldid=999748921#Encoding ↩
It’s pretty easy to understand how this works. In a base 10 system, you can “encode” 2 characters out of a set of 10 possible characters in a 2 digit number: “12” for example. Dividing by 10 and ignoring the remainder gets you back the first character (“1”), and dividing by 10 and taking the remainder gets you the second character (“2”). This system works exactly the same, it’s just that it’s a base 45 system. If we used base 45, you’d be able to just read off the two characters after printing the number. ↩

Solving the vehicle routing problem for research biologists

2021-06-20T15:05:15.494622-07:00

Code accompanying this blog post is available on Github.

My partner is a research scientist working in a lab group. Every year, members of the lab are expected to go on long driving trips to collect specimens for research. This labor is shared among the members of the lab (in practice, the graduate student members of the lab). So even though one lab member’s work might only focus on a single species, a research trip might collect many different species, even if the lab member who requires them is not present on the trip.

In practice, the planned destinations are known at the beginning of the academic quarter. The trips need to be planned out in such a way that effort and route duplication is minimized, while each individual only goes on as many trips as their workload will bear.

While not a professional programmer or computer scientist by any means, I’ve been tinkering with software long enough that coming up with ways to solve problems like this is not that hard for me, and so I’ve been asked on several occasions now to solve the routing problem for them. Having done that, I thought I’d document my process (using entirely open source software and data) and provide a few hacked together scripts to simplify the process.

The core piece of the puzzle is Vroom, a software project explicitly designed to solve the vehicle routing problem in an efficient way. It’s designed to integrate with OSRM, a routing engine built with OpenStreetMap data. If you’re curious about the system, I recommend this underappreciated talk about Vroom.

Dealing with constraints can be complicated. Fortunately, I never had to worry about blackout dates, because the routes once calculated could begin whenever the driver was ready to start. That said, Vroom has one significant limitation: the only metric it allows you to optimize is total drive time. If there is 3 weeks worth of work to do, and it minimizes total drive time to send one vehicle on a very long trip, Vroom will do that. You can’t tell Vroom to, for example, minimize the length of the longest trip, or to make all trips as close together in length as possible. I thought at first that this would make Vroom unusable for my tasks (it certainly doesn’t seem like it would work for hot pizza delivery, for example), but with a lot of handheld tinkering you can get it to work using availability constraints on the vehicles. I also didn’t find any software that would handle the problem better.

While OSRM does provide a very generous free API, it’s still too limited for the kind of routing I needed to do. Vroom works by requesting a routing matrix from OSRM that contains the travel times between every pair of points you submit. Sometimes I would be given a thousand or more destinations. This meant I had to run a routing server myself. Fortunately I have an always-on server that was more than up to the task.

You can find the routing server at osrm-backend. There’s a Docker image, but as I prefer to run software in different containers I simply built it myself. The map data I needed (the US West subregion) is available to download from Geofabrik. After pre-processing, the data for the western United States is “only” 14 GB.

Very clear instructions for setting up the software are available on the project’s wiki page on Github. You’ll probably want to use the MLD pipeline as the pre-processing step is much faster. The page recommends the CH pipeline specifically for large distance matrices (exactly what we have), but as we’re only going to submit a small number of requests to this server in its lifetime, we can afford for them to be slightly slower.

Vroom has a documented JSON format that describes the available vehicles, the destinations they need to reach, and other constraints. I wrote a script to create the necessary input for Vroom based on data in a spreadsheet (exported to a TSV file).

The format of this file is

Latitude <tab> Longitude <tab> Skills <tab> Time

Skills is probably the most interesting feature I made available. Sometimes one scientist is particular good at identifying one species, and so the plan is to make sure they’re in the vehicle for every destination that involves collecting that species. In the spreadsheet, the scientists can put a number (or several numbers separated by a comma) in the corresponding Skills cell. Each vehicle has a corresponding list of numbers which identify the scientists in each vehicle (a scientist can go on multiple trips and so be in multiple vehicles). Vroom will use the skills feature to only send a vehicle to a destination if there is a scientist in the vehicle who can identify the specimen there.

The Time field is occasionally useful too. It’s an estimation of how long the vehicle will spend at the destination not moving (in seconds). In most cases, the scientists I was routing for would simply park the car, clip a branch of a tree or shrub, and get back in the car, so this was effectively zero. In some cases, however, the GPS point could be a mile or more from the road, meaning that a longer hike was in order.

The scripts that I’m making available are very much hacked together, not up to my usual standard for code I make public. For example, there are important constants in the makejson.py file that have to be set or adjusted when the script is run. I didn’t come up with a convenient way to use argparse for this.

We have the already-discussed list of VEHICLES:

VEHICLES = [[1,2],[3,4]]

This identifies two vehicles / trips, where two scientists (numbered 1 and 2) are in the first, while two other scientists (numbered 3 and 4) are in the second.

We have lists of starting coordinates and ending coordinates for each vehicle. These are in [longitude, latitude] format because that’s what Vroom itself uses.

STARTCOORDS = [[-118.44, 34.068], [-118.44, 34.068]]
ENDCOORDS = [[-118.44, 34.068], [-118.44, 34.068]]

Note how the starting and ending coordinates for the vehicles are all the same in this case. That’s a common pattern (starting and ending at the university), but you’re not locked into doing that by any means.

Last we have TIMEAVAIL:

TIMEAVAIL = [400000, 400000] # seconds

This is the messiest part. It’s the total drive time allowed for each car. In theory, this would be perfectly known to the participants in advance (they could allow more time for stops, breaks, sleeping, etc., as needed). In practice, though, it’s a balancing act. The research trips have to be done, so even if finishing all of them would take more time than the scientists theoretically have, put together, that just means that they have to find more time from somewhere. So in practice the only workable approach that I found was setting this value unreasonably high for each car, then reducing it selectively to get routes that looked reasonable. When you can’t reduce it any more without the router failing, you have a roughly optimal solution.

Vroom will print a large JSON file with the results (and the routes themselves from OSRM in polyline format), and so I have another script that takes the result and prints the result as a long list of coordinates for each vehicle, and also exports the routes (as lists of coordinates) to text files, so that they can be shown on a map. Theoretically I ought to create KMLs or something more useful, but this turned out to be more complicated than I had hoped to do in Python and so I ran out of time.

My repository contains a run.sh file that shows how I run the program for my specific setup. Obviously you’ll need all the pieces in place (Vroom, OSRM on a server, etc) in place for this to work. Running the script, even for a large list of destinations, takes only 3-5 seconds, so re-running it while changing the parameters is no big deal.

The results are extremely nice looking:

Thanks to GPS Visualizer for making it easy to plot these routes.

This solution shows three different routes, constrained to a limit of about a week per trip. Several of the points have additional constraints, such that only one scientist can collect the sample, and this has still allowed for a nice solution. In many cases, it’s possible for human intuition to come up with a map that looks a lot like this one, but getting the exact order of coordinates right turns out to be crucial in saving hours of drive time.

At the end of the day, I suspect my exact approach won’t be completely workable for most people. You’ll have slightly different constraints than me, a different input format for your data, or some other problem. That’s why I’m just sticking these scripts online instead of spending the time to make sure they’re as functional and well-written as they could be. Hopefully, others will be able to look to them for help in solving their own routing problems.

How I defeated Wordle with Python

2022-01-11T19:07:12.866986+00:00

Wordle is a simple online word game. It is played by similar rules to the classic game Mastermind, which is known in the United States through the version licensed to Hasbro. Wordle is played with words instead of arbitrary configurations of pegs or numbers, and there is one added twist: each of your guesses must also be a word in Wordle’s dictionary, not any arbitrary configuration of letters.

For those unfamiliar with Mastermind, I recommend clicking through to try the game or reading this brief description of the rules: every day, the game randomly chooses a five letter word from a predetermined list of possible words. On each of your turns, you will guess a word that must be in a dictionary contained by the game. If you guess the word within 6 turns, you win. The game tracks your streak of wins and the number of guesses it takes you to solve each one.

For each guess, the game will tell you whether each letter

does not exist in the correct answer at all
exists in the correct answer, but is at the incorrect location
is the correct letter and is at the correct location

Each letter in the guess has a 1-1 correspondence with the same letter in the answer, if there is one. This means that if the correct answer is “means”, and you guess “green”, you would see the following:

The yellow color indicates that the chosen letter appears in the solution, but that it is in the wrong place in our guess. The second “e” appears in grey, because there is no second “e” in the solution. There must be a 1-1 correspondence between each guessed letter and a solution letter, and if they were both highlighted in yellow, the “e” in the solution would correspond to two letters in the guess.

Likewise, a green letter, indicating a letter in the correct location, takes precedence over a letter in the incorrect location. So if the solution was “rules”, we would see the following:

Here as before only one “e” is highlighted, but the second is in green rather than the first being in yellow, since the letter in a correct location takes precedence over a letter in an incorrect location.

I’d like to solve the puzzle

When I come across a puzzle like this, I’m immediately compelled to think about optimal ways to play it. For example, the player is in the same knowledge position at the beginning of the game every time. Since the opponent is not adversarial (a random 5 letter word is picked, not a word intended to be problematic for any particular guessing strategy), the optimal strategy necessarily means guessing the same word first every time. Few human players likely play this way.

In fact, given that the solution word is drawn from an unchanging list of words before the game begins, this means you can just pre-calculate the optimal guess given each previous guess and response from the game, laying out every possible game state in a tree-shaped diagram.

I have written a solver for Wordle in Python which searches for an optimal (per several rules, see below) game tree, and saves it in a JSON file, which turns out to only contain about 15 KB when compressed. The optimal game tree for Wordle therefore turns out to be pretty small!

How the solver works

The ideal game tree would have 3 constraints:

Every guess is a word that the game will accept
The maximum path to a solution is less than seven guesses
The average path to a solution is minimized

I have made several simplifications in order to quickly get a reasonable solution:

Every guess is a word that the game will consider using as a solution
Each guess results in the smallest number of live possible solutions, on average.

A constraint to keep the maximum path under seven guesses turns out not to be necessary, because a solver with just the two constraints above will never take more than five.

As it turns out, the game actually has two word lists: one is the set of words it will consider using for solutions, just over 2000 words. The other is the full dictionary of words it will accept as guesses. I found both these lists in the Javascript source code of the game. Clever use of the full list of guessable words would in same cases allow faster solves, but my solver is so efficient even without this that I haven’t seen fit to implement it yet. So my program only guesses words that could, in theory, be used by the program as solutions.

Likewise, always guessing the word that will result in (on average) the smallest number of possible solutions is only an approximation of optimal guessing. There are 2314^6 = 153525361154699100736 different possible routes through the game, and so brute forcing your way to an optimal solution is (while not unthinkable as it is in chess), probably unworkable in Python. Clever pruning of the search tree should help (and I’m going to look at this at some point), but is not as easy as it is in games with an opponent. In chess, you can simplify the tree by assuming that the opponent will always make the move that is worst for you. In Wordle, because the solution is random, you have to always consider all possibilities and try to find the guess that is optimal on average.

Still, though, reducing the average number of live solutions as much as possible is a very good approximation of ideal play. Consider the following situation:

After the second guess, the computer makes the interesting choice “pygmy”. This may seem counter-intuitive. Not only do we already have two vowels confirmed to be in the word, “i” and “e”, the guess actually contains “y” twice, which reduces the number of letters contained in the guess. As a matter of fact however, this guess is rather astute. The computer will always find the solution on the fourth guess depending on the outcome of this query.

Using ‘A’ to mean absent, ‘P’ to mean present, and ‘C’ to mean correct:

AAAAA → liver
AAAPA → timer
AAPAA → giver
AACAA → tiger
PAAAA → viper
CAAAA → piper

Being in a point in the game tree where there are six descendent guesses means that there are six possible solutions. Clearly, optimal behavior would be to always find the solution on the next turn. A brute force search would find this solution, but so would the heuristic of eliminating as many possibilities as possible. With “pygmy”, we always eliminate five possible solutions, the best result possible in this case.

Using a heuristic like this is much faster than a brute force search, because it generates a guess in each situation without needing any recursion at all.

Results

This was just a quick little project for me, taking a few hours, so the code is relatively unoptimized. Determining the best first guess requires searching every combination of possible guess and possible solution, and this takes several hours. The complete tree is generated in only a few minutes after that. Because it is stored as JSON, a player for the game is included that doesn’t have to do any searching - it simply reads the next guess out of the game tree.

Stats:

The solution is found, on average, in 3.51 guesses. In the worst case scenario, the solution is found is 5 guesses. The histogram of outcomes is as follows:

1: 1 (0.0%)
2: 65 (2.8%)
3: 1085 (46.9%)
4: 1074 (46.4%)
5: 90 (3.9%)

If anyone improves on my solution by utilizing the complete dictionary or achieving a brute force solve of the game, I would be curious to hear how much you manage to improve on these statistics.

Updates

Jan 11: I discovered that the program was treating all guesses that resulted in the same average reduction to the live possibilities as equivalent. We can, without violating the constraints given above, choose to prefer guesses that are also possible solutions to the current puzzle. Adopting this change reduced the average solution distance from 3.68 to 3.51 guesses.

Code

All of my code for this project is open sourced, under a GPL3 license, on Github. This includes:

words.txt - a list of the words used in possible solutions on Wordle
solver.py - uses words.txt to generate the solution (solution.json)
solution.json - a pregenerated copy of the solution
stats.py - analyzes solution.json to generate the stats given above
player.py - an interactive prompt that uses the solution to play the game
game.py - in case you want to get addicted to Wordle rather than play once a day, this game will generate endless puzzles for you to solve

All the Python code should run on recent versions of Python 3, including the Pypy implementation for additional speed.

Looking at the distribution of ratings on Goodreads

2022-02-27T07:49:04+00:00

Goodreads is a popular site for tracking books you have read and finding new ones using your ratings. Many people have noticed that Goodreads ratings have a surprisingly small variance: it seems like most popular books have ratings around ~3.8. This suggests that a book the community assesses as fairly poor and one that it says is great will be closer in rating than the descriminatory capacity of a single user (you can’t grant half-stars in your Goodreads ratings)!

Is the situation really as bad as all that? Can anything be done about it?

One user scraped Goodreads’ API fairly recently and made the data publicly available. I downloaded this data and performed some analysis on the distribution of ratings on the site.

One thing I found right away is that “only” about half of the films in the dataset have ratings between 3 and 4. There’s a bigger issue, though: about 10% of the films have a rating of precisely 4. That’s because this includes a ton of films with only a few ratings, and 4 is the most common rating on the site. What we need is a model.

What we’d like to have is a distribution function describing our knowledge about the dataset. This is usually called a Bayesian prior. Let’s consider a simplified example.

Bayesian update

If the data were in the form of thumb-ups and thumb-downs, our data would consist of a proportion of thumb-ups out of the total number of ratings. In other words, it would be binomial data. This entails that our prior distribution would be the corresponding conjugate prior, the Beta distribution. If we fit a Beta distribution to the data, we would get a function with two parameters, α and β.

These parameters have a deeper meaning. Bayesian update allows you to introduce new information to the prior knowledge you have, and tells you what your resulting beliefs about the question at hand should be. When your prior is a Beta distribution, the parameters are simply added to the new evidence. If you have a new book with x thumb-ups and y thumb-downs, the prior evidence suggests that the book’s true proportion of thumb-ups to thumb-downs is x+α : y+β. Because the resulting rating is adjusted by adding these “fake” ratings, α and β are sometimes called pseudocounts.

Alternatively, with a simple substitution, you can think of yourself as adding c = α + β fake ratings with the average value m = α / (α+β). This approach is given the name “bayesian average” by Paul Masurel, here.

We don’t have just two possibilities, of course, we have five: five different star ratings. A distribution with several options seems like it would be appropriately characterized by the generalization of the beta distribution known as the Dirichlet. This approach is what Masurel suggests:

If you have the relevant data and infinite time, you may set these two values by fitting a Dirichlet distribution on the dataset of the ratings of all your computer books. However it is very common to just choose a pair of parameter that mimick the behavior that we are looking for.

I’ve seen this approcah recommended by several others as well. Okay, I have the relevant data and infinite time, but let’s consider whether fitting a Dirichlet really makes sense here. I believe that it actually does not!

The problem is that the Dirichlet (and actually, the beta distribution) are conjugate priors for distributions of categorial variables, not discrete ones. The binomial distribution, for instance, has success and failure conditions, not 0 and 1. The multinomial distribution that has any number of distinct categories.

Why is this an issue? Well, suppose we had distinct categories. Users could describe their reaction to a book as “green”, “orange”, “red”, “yellow”, or “blue”. (Assume that users wouldn’t immediately derive a mutually agreed upon ranking system for these colors, i.e. they really are just colors.) In that case, there would be no inherent correlation between colors, and you couldn’t average them into an overall color. Star ratings are precisely the opposite. They’re numerical ratings giving a quality score, and usually what we care about is just the average.

If you fit a Dirichlet distribution to star ratings, the resulting distribution will be underconfident about the behavior of the average star rating, because it treats the ratings not as comparable numbers but as incomparable categories. More specifically, a set of star ratings will converge to their individual true values as new ratings come in more slowly than their average will converge to its true value.

As proof of this, here is the result I get when fitting the Dirichlet distribution to the Goodreads data.¹ I am plotting the Dirichlet reparameterized to show the strength of the prior on the average value.

As you can clearly see, the result is underconfident. Don’t use the Dirichlet as a prior for star ratings, especially when you only care about the average!

What should you use instead? I think the beta distribution is actually more reasonable here. Instead of modeling the proportion of successes and failures in a series of Bernoulli trials, we can think of it as modeling the average value of a series of individual equally weighted fractional successes, where e.g. 5 stars means perfect success and 1 means complete failure.

If anyone knows of a better way of motivating the use of a particular distribution here, please do contact me by email or on Github and let me know.

The results

I filtered out books with fewer than 50 ratings, to eliminate books for which no rating consensus has yet been formed. (I also implemented some special handling for books that have at least one star rating which no one has yet given them, though this is a technical detail which I’ll let the interested read the code to see.)

The beta distribution appears to be a very good fit for the data. Experimentally, the distribution of dirichlet averages seems to become more and more beta-like as the strength of the prior increases.

The smoothed and filtered values show the Goodreads ratings to be worse (less informative) than we initially feared. Fully 80% of adjusted ratings are between 3.5 and 4.2, a range considerably less than the one star’s worth of discrimination available to individual users when rating a book. Virtually every book on the platform that has enough ratings to be reliable gets at least a 3, and virtually none gets more than a 4.5.

A partial solution

Given that we have a model of the data, we can adjust the average rating of books with too few ratings to give ourselves a more accurate estimate of the true average of the book. Furthermore, we can use the model to determine what percentile a given rating corresponds to: my analysis suggests that a book with a Bayesian rating of 4 is better rated than 2/3 of books on Goodreads.

For those who are interested in more accurate ratings (for books with a small number of them) and the ability to see a book’s percentile, I have made a user script which does just that!

Does the ability to see rankings instead of a raw score solve the problem with Goodreads? It certainly makes things better, but in my opinion it doesn’t solve the basic problem, which is that Goodread ratings just kinda suck. They’re inconsistent, prone to audience biases, and fairly arbitrary.

A sampling:

As it happens, I think all of these works are great. (Books I hated are similarly all over the place.) The problem is not just that the ratings are so widely varied, it’s that there’s no apparent rhyme or reason to the order these books are in. When a film I think is great has a relatively low rating on IMDb (example: I’m Thinking of Ending Things), there is usually an obvious, comprehensible reason why most people didn’t like (or were indifferent to) the film. Likewise, I thought Star Trek Into Darkness was quite bad, but I understand why most people liked it. Nothing of the sort can be said for Goodreads: not many people dislike Shakespeare — so what’s happening here? Unlike with IMDb, where the ratings (though clearly inflated, with most movies between a six and an eight) do frequently indicate something about the quality of the movie, I’ve found no way to extract the same value from ratings on Goodreads.

Here, I’m eliding how we arrive at the precise α and β values: the Dirichlet is parametized by a vector α that behaves exactly the same way as the α and β values of the beta distribution. Since our data has 5 possibilities, α will contain 5 values. To update a Dirichlet prior, each of the values is added to the corresponding value in the evidence vector. As Masurel shows, since we care only about the average value we can simplify the resulting α vector down to two variables; the sum of the vector, and the average (number of stars) that a book with exactly α reviews would have. These values are just the c and m of the bayesian average approach. ↩

Statistics-assisted weight loss with pyWeight

2023-04-12T05:13:18.329538+00:00

Content note: this article contains a discussion of weight loss and healthy eating, and includes my before and after pictures at the end. You can view a version of this document without these pictures here.

Quick summary

I lost 80 pounds following a diet focused on calorie restriction and satiety by writing software to track my weight and suggest adjustments. The program is PyWeight and it is available as a free download for Windows or Linux.¹

The premise of the diet is that it’s possible to achieve a precise rate of weight loss when restricting your intake without counting the calories in all the food you eat or knowing your daily energy expenditure. A reasonable estimate of how much you’re eating is good enough, when assisted by technology.

Because a calorie deficit and the resulting rate of weight loss are mathematically related, software that measures your average rate of weight loss can accurately determine whether you’re eating too many or too few calories, even if you don’t count them exactly. You can use the information the software provides to hone in on your desired rate of weight loss by making small adjustments to your diet over time.

If this sounds interesting to you, read on.

People who want to follow this diet should talk to a doctor beforehand.

Why everyone is gaining weight and why it’s so hard to lose it

Some people believe that their bodies are genetically programmed to reach and maintain weight at a particular “set point”.

This clearly can’t be the whole story. In the last 40 years, the average weight of the adult population in most western countries has increased significantly. If these weights reflect our natural set points, then we are forced to conclude that just about everyone else who ever lived somehow never reached their body’s natural weight. We’re the lucky few — except that apparently we’re unlucky, since this supposed biological imperative just so happens to be bad for us!

Yet neither can the set point view be completely wrong. It accords too well with several common experiences.

When someone in a country without a fast-food culture moves to one with an urban environment, they frequently gain a moderate amount of weight and then stabilize at a new point from which they find it difficult to gain or lose.

Similarly, people who do manage to lose weight often hit plateaus — low points at which they find further weight loss to be much more difficult. Some people regain lost weight until they’re about where they started. What’s really happening?

I suggest that the truth lies somewhere in between. You do have a distinct set of variables that dictate the weight you will approach in the long term. These include your genetic makeup (with its effects on your metabolism, skeletal structure, and so on), age, and hormone levels. But these variables also include diet and exercise, two variables that are partially under your control.

In the rest of this article, I will refer to the weight you are approaching given your current daily intake (and other variables) as your “equilibrium weight”, to distinguish this from the simplistic “set point” view.

Holding everything else constant, if you are currently maintaining your body weight and you permanently reduce your intake by 100 calories per day, you will eventually stabilize at a new equilibrium point about 10 pounds lighter than your current weight.² A cut of this size is equivalent to drinking one fewer can of Coca-Cola each day at work (five days a week).

Why losing weight is hard, and maintaining weight loss is even harder.

There’s one additional but very important wrinkle: one of the variables that dictates your equilibrium weight is your personal history of weight changes, because of its effects on your metabolism and body composition.

Consider three genetically identical people of my height and age. They all weigh 235 pounds.

Person 1: has weighed 235 pounds for many years
Person 2: recently lost weight from 350 pounds
Person 3: recently gained weight from 150 pounds

According to one model of diet performance³, they will need to eat as follows to maintain their present weight:

Person 1: 3050 calories
Person 2: 3000 calories
Person 3: 3400 calories

In other words, if all three want to lose weight, the person with the thin “history” has an advantage. Because their body has not yet adjusted to carrying this extra weight, they will burn an extra 400 calories a day for free.

A 400 calorie per day difference is a whopping huge one. It’s about the equivalent of eating a pint of your favorite Ben and Jerry’s ice cream every Monday, Wednesday, and Friday. On top of that, the previously 350 pound individual was able to eat about 3800 calories a day at maintenance. So just to maintain their current level of weight loss, they have to get used to eating 800 fewer calories than they are used to.

This may seem unfair. These are people who are genetically identical, the same age, the same height, and the same weight. But one of them will maintain at 400 calories a day higher than the other. It is a little unfair, but remember that these are ultimately temporary metabolic effects. Eventually Person 3, who gained 85 pounds, will become Person 1, who has carried that weight for a long time.⁴ The body doesn’t provide you with a metabolic advantage forever.

This leads to two important principles for maintaining weight loss:

Until your body becomes similar to that of a typical person who maintains at your new weight, you will face transitory metabolic effects that resist your weight loss. This means that until your body adapts, you will have to eat less than others of the same weight can eat.
When you lose weight, you become a smaller person. Even once your metabolism stabilizes, you will need fewer calories to maintain your weight than you did at your original body weight. Your hunger systems may have accepted dieting, but when the time comes to maintain they may be slow to adapt to your new maintenance intake, and you may find them in revolt.

Most people ignore one or both of these principles. It’s tempting to treat diets as one-time interventions similar to a medical procedure. This simply will not work, as the foregoing discussion indicates. You can’t eat like an overweight person who has temporarily starved themselves to fit into a smaller pair of jeans. What you must do if you want to keep weight off is to permanently eat like the thin person you now are. You must lower your equilibrium weight through consistent, long term improvements in diet and physical activity.

Most people recognize on some level that when they regain weight after dieting, it’s because they “ate too much again”. What many do not realize, however, is that “too much” means something different after dieting than it did before.

The worst part about all this is that you’ll begin to feel the effects of the two principles while you’re still trying to lose weight.

Most people start a diet expecting to lose weight at a relatively quick rate — and they do their first week or two. Unfortunately, most of this weight loss is fake. Running a calorie deficit means that your body burns through its store of glycogen, which is what causes it to tap your energy reserves (fat). Along with the loss of this weight, you will see a one-time drop that’s just water weight; your body has less food temporarily stored in your stomach and intestines as it’s being digested. Your body will also adapt to reduced intake relatively quickly, and your metabolic rate will drop slightly.

Metabolism is not magic. Your body obeys the law of conservation of energy when your metabolism “slows”, and it doesn’t have a more efficient “starvation mode” that uses fewer calories to perform the same tasks. (If it did, natural selection would surely have dictated that we would always be in that mode, and thereby need to eat less.) Rather, a “slowed” metabolism means that you will either be slightly colder (unlikely to be by enough to make a serious difference) or that you will have less energy available, and therefore perform fewer tasks. Maybe you will feel like exercising less, or maybe you will perform tasks in a lazier fashion, or maybe you’ll simply sleep more. Putting all this together, it means that you can expect to lose some fraction of your caloric deficit to temporary metabolic changes.

If you don’t get discouraged and continue dieting, your weight loss will continue apace, but unfortunately you start to run into the problem discussed above: your reduced size is resulting in a smaller energy expenditure. Research suggests that for most reasonably large deficits, it would take years for this effect to neutralize further weight loss (at which point you would have achieved a new equilibrium), but in the mean time your losses each week will get smaller and smaller. No wonder most people get discouraged! They end up cheating or adding calories to their diets little by little, and eventually they stall out. This is why many people hit plateaus in their diets. A slowing rate of weight loss leads to discouragement, and discouragement leads to a failure to adhere strictly to the diet, and this leads to you hitting a new equilibrium point much earlier than expected.

Because of your body’s remarkable capacity to adapt to changes in food availability, you have only two choices if you want to overcome this problem. You can start out with a lower calorie budget, precisely calibrated to match the metabolic changes you anticipate. Or you can adjust your calorie intake as you go along, to maintain a constant deficit. I strongly suspect that the latter approach is easier to follow and less discouraging, so it is the approach PyWeight is based on.

Where is the obesity epidemic coming from?

This talk of equilibria makes an obesity epidemic rather surprising. Even if people are eating more than before, why aren’t they simply reaching new slightly higher equilibrium points, like our theoretical immigrant? After all, if the way one developed a body that required 4000 calories just to maintain itself was to start eating 4000 calories a day, no one would ever do that. Most people with normal body weights couldn’t even stomach 4000 calories for many consecutive days.

But the evidence for epidemic obesity is clear. Here in the U.S., the state of Mississippi had the highest rate of obesity in 1995, at 19.4%. In 2010, a mere 15 years later, Colorado had the lowest rate of obesity in the country, at 19.8%, and 12 states were over 30%.⁵ Today, more than 40% of American adults have obesity.⁶ This is happening quickly enough that it doesn’t work to posit genetic or behavioral changes in a new generation of adults: the same people who were thin 10-20 years ago have obesity now. So what’s happening?

Of course, when we’re talking about the obesity problem, we’re talking about averages. Some people do maintain an equilibrium point that’s just overweight. Other people don’t become overweight at all. Yet in this country, we seem to have reached the point where the majority of people put on weight year over year, endlessly.

You can square an equilibrium view of body weight with this fact using the suggestion that we’ve reached some kind of tipping point. For some reason, people are driven to eat slightly more than they need to maintain, and for many of them the rate at which they increase their intake has begun to outpace the body’s compensatory mechanisms.

The body is calibrated to give you hunger signals that will result in you eating approximately what you need to eat to maintain your current weight, on the sort of diet most people had in the past. Someone who could maintain at 2000 calories a day and significantly overate⁷ would gain weight, but after doing that their body would still only be giving them hunger signals appropriate to maintain their new body weight — or more precisely slightly less until they adapted. (Effectively, the two principles mentioned above are functioning in reverse here.) So this person would not automatically continue to gain weight. The body’s negative feedback of constantly feeling too full would tend to keep their weight in a healthy range.

Our hunger signals are no longer accurate. The body still says “I need 2000 calories”, but it doesn’t feel full until you’ve eaten 2100, thanks to the low satiety calorie-dense diets of today. When you gain enough weight on such a diet, your body will start to say “I need 2100 calories”, but now it takes 2200 to fill you. And because eating food takes time and costs money, your increased appetite will be drawn towards quicker and cheaper options, further exacerbating the problem. Before, the body gave you you a signal that you should eat less food the day after a big meal. Now, even a big meal isn’t enough to make you feel full.

This suggests we should change our view of personal responsibility. People in the last 30 years are facing a problem that virtually no one in the past faced: their stomachs are lying to them about how much they should eat.

Recall the equilibrium equation: your equilibrium weight is a function of your genetics, body composition, age, hormones, diet, and exercise. If you consider this equation on a societal level, you can replace diet and exercise with just one variable: environment. In the absence of a strong effort to control these two variables, the health environment you are in will dictate them to you, and therefore control your weight.

Faced with the environment that most people in Western countries face, the average person is going to gain weight over time. So on a broad social level, that’s what we see. Blaming individual people for the consequences is not just counterproductive, it’s an inaccurate accounting of what has happened.

Consider the following scientific model of average weight gain⁸:

What this model indicates is that the weight gain of the average American over the last decades can be explained by a relatively tiny gap between energy intake and energy expenditure. In fact, the average gap for the period examined in this study was only about 7 calories a day. The problem doesn’t amount to 7 calories here or there, the epidemic stems from the fact that this gap is so reliably sustained. High calorie junk food is causing many of us to overeat, day after day, with no feedback telling us to cut back for a while. This has gone on for so long that in 2005 the average person needed to cut out more than 200 calories a day to achieve the equilibrium point of the average person in the 1970s. This number, of course, has only continued to rise.

Watch this excellent five minute clip detailing some of the causes of calorie overproduction in the United States:

Source: The Weight of the Nation, 2012. HBO Documentary Films.

You might be tempted to decide the takeaway of this analysis is political: corporations are profiting off of human suffering on a massive scale, so some social action is needed to fix the problem. Insert regulation, social democracy, or whatever your preferred solution happens to be. That’s all well and good, but it’s of little use to someone suffering the effects here and now. This country will happily let you die in the street. In all probability, things are not going to get better on a timescale that matters to you. The only option is to take matters into your own hands and exploit the two variables under your control to the maximum extent you can. For some people, that will be enough.

The good news

The one bright spot is that for most people the food environment is not yet bad enough to make it impossible for them to control their weights. If you are not very constrained in your diet and exercise choices, then you have a fairly good chance of overcoming your environment.

As for the other variables in the equilibrium equation that are outside of your control (e.g. genetics and metabolic effects), the good news is that these almost always have a relatively minor impact on your weight. Most people have a basal metabolic rate (the number of calories you burn each day just by being alive) within 10 percent of the rate estimated by the Mifflin-St Jeor equation. “Slow” metabolism is not usually the cause of weight gain.⁹

This means that many people can change their equilibrium point by managing their calorie intake and becoming more physically active.

Addendum: it’s okay to try to lose weight, I think

Some authors, including philosopher Kate Manne, claim that encouraging dieting and even dieting yourself are immoral. I agree that negative self-image and exploitation in the diet industry are both things we should condemn, but for this to have consequences for individual dieting decisions something more is needed.

I think that’s why Manne’s article, though it defends an ethical conclusion, ultimately rests on two empirical claims. She writes that “fat bodies can be not only healthy but also athletic” and “the vast majority of diets fail to make people any thinner or any healthier in the long term”. These claims are both true, and suggestive, but don’t on their own deliver her conclusion.

It is true that some fat bodies are healthy. Similarly, many people who don’t get the recommended 30 minutes of exercise every day are also healthy. Despite this, since exercise and a reduction in overweight tend to result in health improvements, it can make sense to pursue both.

Likewise, the fact that most diets fail cannot deliver the conclusion either. If this claim is to show that it would be a bad idea for me to start a diet, it needs to be true that I can have no expectation of success, so that trying to lose weight is similar to playing the lottery. I think this is false. There are reasons that most diets fail, but dieting is not tantamount to fighting genetic or environmental determinism, as I argued in the previous section. I explain further why I think it is reasonable to expect success in the next section.

I suspect that any versions of Manne’s empirical claims strong enough for her conclusion will be false.

The claim that some fat bodies are healthy suggests a stronger version that would suffice: most fat bodies are healthier than the same bodies would be if they were thin. Dieting would then, in all likelihood, make you less healthy. In this form, the route to Manne’s conclusion that encouraging dieting or dieting yourself “is a morally bad practice” is obvious, but the premise seems clearly false. There certainly is a correlation between body weight and health. As I will discuss later, some (contentious) studies suggest that being slightly overweight does not increase mortality, but no medical authorities will endorse the notion that obesity is healthy.

Likewise, if diets could be shown to be inherently dangerous, unhealthy, or doomed to fail, it would be easy to reach the conclusion. Manne hints at this: she says that in one month early in her diet, she didn’t eat for 17 out of 30 days. This is indisputably a bad approach to dieting. If most people trying to lose weight go for diet fads and ineffective products, or take dangerous or disordered approaches like extreme fasting, that would help explain the failure rate.

However, it seems false that these problems are inherent to dieting. I ate food every day on my diet. Manne herself seems to have ended up with a sensible diet: she lost about 50 pounds in a year, close to the common recommendation of a pound per week. The diet industry is certainly blameworthy for distracting people from methods likely to be successful and promoting fads, but this alone can’t show that dieting is wrong.

How to succeed at dieting

Most diets fail. In a major longitudinal UK study, less than 20% of people with obesity managed to decrease their BMI by at least one category without a subsequent increase, over a 9 year period. 78% of people who lost at least 5% of their body weight gained enough back that after 5 years they were back within 5% of their original weight, or even higher.¹⁰ This indicates that while plenty of diets do succeed (the popular claim that 95% of diets fail is incorrect), successful dieters are by no means the majority.

I don’t believe a 20% success rate should be discounted. It suggests that diet success is possible and that we should be looking at what the successful people do. If we were to discover that a particular intervention to help people quit smoking had a 80% recidivism rate, we would still encourage people to take the program (because 20% is much better than zero).

Part of the issue with assessing the success of a typical diet is that if you lower the bar of what counts as successful weight loss, then the success rate goes up. If you demand studies involving weight loss of a quantity closer to what I was hoping to lose, examples become even rarer. Most studies are testing specific short-term interventions to see what works and what doesn’t, not looking at how to achieve a long-lasting high level of weight loss. It’s hard to get reliable data about the prevalence of successful diets resulting in long term weight loss of 25% of body weight or more. A large portion of attempts likely fail.

The reasons why most diets fail go beyond simple misunderstandings of what it takes to succeed, of course. Food is tempting. Not eating food when you are hungry sucks. The restrictions that a diet imposes can be difficult to implement. The nutritional deficits imposed by Western eating can become even more severe on a calorie restriction, especially for those who can’t afford to make their own food. For many people, a personal trainer and gym membership are unaffordable, and for some even a doctor visit is out of reach. (It was for me when I began my diet! Thanks, America.)

Given the high rate of failure, what I went looking for was a diet that could guarantee success to those who stick to it and and don’t cheat. My hope was that following such a diet would not only bring me the weight loss I was looking for, but that the certainty of success might provide additional motivation to stick with the diet.

As a result of studying physics in college, I was naturally inclined to follow a “calories in, calories out” diet. This comes with something like a success guarantee: if you consume fewer calories than your body uses each day, you will lose weight. Other diets may be successful as well, but only if they end up resulting in calorie reduction anyway. Diets may have some individualized metabolic effects but these appear to be minor in comparison.¹¹¹² Perhaps some other diet is easier to maintain than calorie restriction, but no other diet could provide the guarantee of success.¹³

Counting calories has a reputation for being difficult, which is part of the reason diets that dress up the calorie counting by using “points” systems are so popular. It’s not only easy to underestimate the number of calories that you’re eating, it’s also difficult to determine with accuracy how many calories you’re burning. Moreover, if you weigh yourself once a week or so, your measurements might get swamped by variations in water weight or having an unusually large lunch, so it’s hard to know if you’re making progress. If you weigh yourself once a day, the value on the scale is likely to fluctuate so much that you constantly fly into a panic over an imaginary weight gain.

Furthermore, you might already be disheartened by reading the previous discussion of why it’s hard to lose weight. As you lose weight, the number of calories you need just to stay at maintenance is going to be decreasing. How is it possible to reliably stay ahead of the curve? It turns out that there’s a simple trick that can be summed up in a single paragraph, and this trick obviates most of the complexity of weight loss. This trick is:

Record your weight every day. Every two weeks, calculate your approximate weight loss using your record. If you are losing weight too quickly, eat more the next two weeks. If you are losing weight too slowly, eat less the next two weeks.

That’s it. Weight loss, summarized. By only worrying about whether your rate of weight loss matches your predetermined goal, and not calories or points or what have you, you can hone in on a reliable rate of weight loss over time with relatively little effort. Accurately counting calories or determining your daily expenditure is not required.

You can do this yourself with just pencil and paper if you like. But technology will help you in two ways: you can more accurately calculate your rate of weight loss using techniques like linear regression, and you can estimate how many calories you missed your target by (instead of pounds), to make it easier to adjust your intake in the future.

This idea was inspired by the book The Hacker’s Diet. The book’s version of this scheme goes something like this: if we assume that each pound of fat lost contains about 3500 calories, then the math becomes quite simple. If you need to lose x pounds, and you maintain a deficit of about y calories a day, it will take you about 3500 * x / y days to do that. At a deficit of 500 calories a day, you will lose about a pound a week. If after two weeks you’ve only lost 1 pound, you know you’re missing your target by 250 calories per day.

Unfortunately, this relies on the myth that a pound of fat contains 3500 calories. It turns out the correct number is closer to 4280.¹⁴ The slightly better version of the theory, which says that the average caloric density of the weight you lose is 3500 calories, is an inaccurate approximation. You do lose some fat free mass while dieting, and this contains around 824 calories per pound. So 3500 calories per pound is only accurate if about 3/4 of the pounds you lose are from fat. This is only a reasonable estimate for very overweight people at the beginning of a diet. Later on, as you approach an optimal body weight, you will lose a larger proportion of lean mass. If you were to continue losing weight indefinitely, you would of course eventually have nothing to lose but lean mass. This would be extremely bad for your health.

Because of this complication, PyWeight uses an equation from a real diet model¹⁵ to estimate how many calories were contained in the weight you lost in a given interval, and therefore how many calories you are away from hitting your weight loss rate target. This means it’s possible to target a linear rate of weight loss despite all the non-linear effects. (For what it’s worth, when you’re targeting a constant rate of weight loss rather than a constant, pre-determined calorie budget, the problems caused by inaccurate estimates are greatly lessened. In fact, you’d probably be fine with the old 3500 calories per pound estimate. The upshot of PyWeight’s advice is usually just “try to eat a little less / more next week”, just as it would be if you were using pencil and paper.)

With the Hacker’s Diet, the calculations are all done in a series of complicated spreadsheets which are provided with the book. PyWeight allows entering your measurements directly in a built-in log, and all calculations are done for you automatically. It also provides intake adjustment advice at a user-selectable frequency.

The linear regression method used by the Hacker’s Diet can also be improved upon. Doing a naive linear regression by breaking the data into intervals lasting 2-3 weeks means that only the data from those weeks figures into the calculation of your current rate of weight loss. But this is throwing away data; you can determine a much more accurate starting weight for each interval if you also make use of the data from the previous interval. PyWeight takes exactly this approach with a technique called spline interpolation. A possible alternative would be to use a moving average, but I believe this would give worse results because of the high degree of variability in weight measurements and the fact that the diet is built around attempting to lose weight at a constant rate.

As a side effect of this improvement, PyWeight is able to generate very nice looking graphs of your weight loss. Here is mine from the last ~7 months. (I lost the first 20 pounds using the methods described in this article before I wrote PyWeight.)

The idea behind using a program to track your weight is that despite the fact that our hunger systems are broken because of modern calorie dense diets, we can use technology to replace this body function, letting simple math regulate how much we eat. This method actually works even better than simple calorie counting, because it adapts to your changing energy needs as you lose weight. If you want to keep up a one pound per week deficit, you’ll need to reduce your intake over time, and this approach will help you do so automatically.

Even though you’re measuring your weight every day, you don’t have to worry about what the scale says in the way you do with ordinary calorie counting. An unusually large meal might send your apparent weight up 1-3 pounds for several days, but this is mostly water weight that will soon go away. Looking at a statistical measure of your weight loss over a period of weeks smooths out these misleading bumps in the graph.

Looking at my graph, you can see several of these blips, some of them quite large. My daily scale measurement increased 6.8 pounds over the course of four days in late December, but this did not represent any long term weight gain.

I’m not going to tell anyone that this approach makes dieting a sure thing. A great deal of willpower is still involved. But knowing exactly what you’re doing and trying to keep up a consistent rate of weight loss makes it much easier. Steven Novella helpfully writes:

I would conclude that weight control is difficult, but not impossible. In addition, I believe that the popular weight loss industry is making the problem worse by distracting the public from those strategies for which there is at least some evidence of efficacy and focusing on minute, short term, and probably insignificant effects from manipulating macronutrient proportions.

My goal with this diet is to focus on what we know works (consistently eating fewer calories than you burn) and find a way to make sure I am doing that with small course corrections over time. I began this diet with a high degree of confidence that it would work if I didn’t get burned out and give up, and that confidence was rewarded. As you can see from the graph above, I was able to maintain an extremely consistent rate of weight loss over time; I never deviated from my monthly target by more than a few percent (except around the holidays and a strenuous four day hiking trip in mid-February during which I snacked almost continuously).

It’s not just losing the weight that gives me confidence this method works. It’s the fact that I lost the weight at almost exactly the rate I planned in advance! I think many individuals can see success in dieting if they can follow a similar approach.¹⁶

No method for individual correction can solve the underlying problem. So long as the United States fails to get a handle on its poor nutrition environment, obesity will continue to be a serious issue. Proposing diets as the solution is like thinking that encouraging people to recycle can prevent climate change. However, until the social causes of obesity are resolved, individuals are on their own. Diets will not reverse the long term obesity trend, but they can help dedicated individuals who are fortunate enough in their circumstances.

Setting goals

I started out just above 240 pounds, which at my height was a hair into the obese range (30+) by the BMI metric. BMI is not necessarily a good indicator of individual health, but the truth is I felt bad. I did not feel like exercising, or even going for a walk. Hiking long distances, one of the activities I’ve enjoyed the most over the years, was certainly out of the question. While it’s not impossible to be in good shape at the size I was, it’s certainly much more difficult.

How bad is it, in general, to be overweight? There has been some recent controversy on this point in the medical world. The long-time consensus has been that while individuals with BMIs between 25 and 30 are not necessarily unhealthy, overweight is correlated with heart disease and diabetes, as well as increased all-cause mortality.

One 2005 study suggested that being in the overweight category was associated with reduced all-cause mortality relative to the normal (18.5 - 25) BMI category.¹⁷ A follow up meta-analysis¹⁸ by the same primary author confirmed this finding, and also found the difference between mild obesity and normal body weight to be insignificant. However, a much larger third analysis¹⁹ disagreed, finding that after controlling for smoking, prior chronic disease, and length of follow-up, both overweight and obesity were correlated with increased all-cause mortality risk. Moreover, they were able to reproduce the findings of the second paper by removing those quality controls. (The danger of leaving in smokers is that a disproportionate number of smokers will have healthy body weights because nicotine functions as an appetite suppressant. Smoking is much worse for you than a few extra pounds, so these people make “normal” body weights look less healthy on average.)

I’m not here to say that the third study was right and the first and second were wrong. I don’t have the expertise to do that, to begin with. The primary author of the first two papers wrote a response paper which claimed that the approach taken by the third was biased and suggested that the exclusion of smokers might lead to more bias than leaving them in.²⁰ The result has been an increasingly bitter academic feud over claims of “bias” with no obvious conclusion (to me).

However, a very nice fourth paper²¹, a systematic review of the issue, reached more or less the same conclusions as the third. It was significantly larger than either of the others, containing almost 10 million individuals in the never-smoking category alone, and none of the authors overlapped with those of the third study. This study found that an ideal BMI was approximately 23, with negligible differences between BMIs from 22 to 25. Furthermore, the paper reproduced the most important finding of the third: the conclusion was reversed when filtering was removed. A BMI of 27.5 was associated with a lower mortality risk than one of 23, and a BMI of 32.5 lower than one of 20!

Because this study subdivided “normal” BMI into many categories, the conclusions are a bit more fine-grained. The results may help explain why in some studies the 18.5-25 BMI range compares unfavorably with the 25-30 range. The dangers of being underweight are quite extreme: among never-smokers, the study found that a BMI of 17.5 was associated with a 35% increased mortality risk. Someone with a BMI of less than 20 contributes to the average outcome for the normal BMI range (if their BMI is 18.5 or higher), but they’re exposed to more of the risks of underweight compared to someone with a BMI of 23, which the study found to be optimal.

I believe it’s very important for casual interpreters of medical research to remember to look at papers in context. All this bickering is essentially over a 5-10% difference. If you believe the fourth paper, a BMI of 27.5 is associated with a 7% increased all-cause mortality risk versus a BMI of 23. If you believe the second paper, having a normal BMI might be associated with 6% more all-cause mortality risk versus an overweight BMI. Personally, I’m inclined to believe the former, because the studies are larger, it’s more in line with the medical consensus view, and because the exclusion of smokers in the analysis seems justified. But I see no reason for me to argue with someone who believes the latter. To my mind, the sane middle ground is found by keeping your BMI well out of the obese range, and at a level where you feel able to keep yourself in good shape. My guess is that the difference between the two is swamped by just getting plenty of exercise.

For me, the truth is that I wasn’t able to feel healthy or stay fit at the weight I was, and the BMI number merely served as a painful (though possibly useful) reminder of that.

For the sake of having a goal weight, I picked the arbitrary target of 160 pounds. This happens to correspond nicely to taking 10 points of my BMI, and struck me as unlikely to be unsafely low. It corresponds to a BMI of slightly above 20, and my plan is to put on a few pounds of muscle over the next year. The last time in my life when I was thin was around the age of 10 or 11 (when I had temporarily become a vegetarian), and I wanted to experience that again.

It remained only to choose a rate at which I was going to try losing the weight. The UK’s National Health Service says that a weight loss rate of 0.5 to 1 kg a week is safe. 1 kg a week is equivalent to 9.45 pounds lost every 30 days. I rounded this up to a target of 10 pounds per 30 days. I’m considerably taller than the average person, and so there is a larger gap between my daily energy expenditure and my basal metabolic rate than for the typical dieter. I think this was a safe target for me.

I emphasize that if you’re going to pursue fast weight loss, you should talk to a doctor about that. You may very well have more success and invite fewer risks by targeting a pound per week (for some people, even less). In my case, without access to professional advice, I made the best judgment I could about what would be safe and effective for me.

How to get enough nutrition on a diet that doesn’t suck

It can be helpful to get a rough estimation of how many calories you should budget for yourself. In my case, something like 1800 calories a day at the start of the diet was appropriate. One nice thing about taking a statistical approach to weight loss is that even if you screw up this estimate quite badly, you’re unlikely to put yourself in any danger as long as your target rate is reasonable. If you lose too much weight in the first weeks of the diet, you will receive a sharp corrective directing you to eat several hundred more calories every day. PyWeight is thus self-correcting.²²

Cutting a thousand calories out of your usual intake can make it difficult to get the nutrients your body expects to have available. It’s important to have a plan for getting the necessary vitamins, minerals, and essential amino acids.

This is an area where The Hacker’s Diet is not very helpful. It says

Since we’re efficient food processing machines, it’s possible to reduce all the complexity of food to a single number that gives the total energy the body can extract from it—the calorie.

The extreme form of this is the infamous Twinkie diet. You will lose weight on such a diet; it might also kill you. (It’s interesting that the actual Twinkie diet guy didn’t just eat Twinkies. A full third of his calories came from fruits and vegetables along with a protein shake. Eating no protein at all for months on end would probably not have gone well.)

To be fair, the book also suggests eating a balanced diet. As a vegan, I think there’s some cause for me to be concerned, especially on a diet as restricted as mine. I took a vegan multivitamin (DEVA tiny) every day of the diet, to ensure I wasn’t vitamin deficient. I really like this multivitamin because (a) it provides ~100% RDA of vitamins I was likely to be deficient on, instead of 10000% like some multis with potential side effects, and (b) it was small and easy to swallow. I also have a B-12 supplement that I take about once a week (as do most strict vegans).

So the remaining concern was protein. Getting enough protein isn’t generally a problem for vegans²³ and even on a restricted diet it wasn’t one for me either, at least in terms of quantity. The issue is that there are minimum amounts of specific amino acids that we have to consume for building cells and other important stuff our bodies do. I wanted to make sure I was hitting the WHO minimum recommendations fairly consistently, even though I didn’t think a mild deficiency for a few months was likely to be harmful. (After all, the Twinkie guy didn’t fall apart.)

In order to hit these targets, I ended up on a diet that was pretty protein loaded. I hit the recommended amino acid intakes almost every day, even though I was on a diet with considerably fewer calories. At the same time, I also emphasized eating a lot of fiber and low-density foods. The point of this is that both fiber and foods with a high bulk to calorie ratio will make you feel full. Note that neither of these strategies will directly cause you to lose more weight than otherwise. They’re simply ways to make a low calorie diet easier to maintain by reducing hunger.²⁴

Many vegan foods are complete proteins (that is, they include all of the essential amino acids). Most aren’t perfectly balanced between the amino acids, so you might have to eat more than enough of one to get enough of all the others. (This is one of the reasons why a balanced diet is often suggested, since eating many different protein sources will help balance the ratios out.) Note that this problem isn’t unique to plant-based diets — beef isn’t perfectly balanced either, nor are most other animal-foods. It’s only something to worry about for someone on an intense calorie restriction.

In trying to come up with meal plans, I found WolframAlpha indispensable. You can enter an amount of food, and it will tell you how much of each amino acid it has (assuming it’s in their database).

Lentils and soy turned out to be useful for hitting my high protein and fiber requirements on a low calorie diet. It’s actually possible to hit the WHO amino acid recommendations and much more than 100% of your fiber using mostly these sources on a mere 1200 calories a day.

You don’t have to eat the same thing constantly on this diet. (In fact the whole point is that you don’t need to be precise about what you eat or count calories exactly.) However, if you’re cutting so much that you need to be concerned about nutritional deficiencies, planning your meals may help. It’s probably still best to mix it up some of the time, just to ward off any additional danger of malnutrition.

On many days, I ate whatever kind of food I wanted (or even went out to eat) and just went with a rough estimate of my calories. The point is that as long as you are hitting your nutritional requirements, and can get even a rough estimate of the number of calories you eat, you’ll be fine. The best part is that you don’t even have to write anything down if you don’t want. It’s a calorie “counting” diet, but all the counting is quietly being done automatically for you by enhancing the quality of your estimations over time until they’re about as good as a real count. (Of course, should you struggle making the necessary adjustments to hit your targets, you might find a food diary indispensable, at least for a short time.)

If it is at all feasible for you, I do suggest making as many of your own meals as possible, and making them as highly nutritious as you can. Fast food is far worse than you or I probably realize, and it tends to be extremely calorie dense without satisfying.

Keys to success

I think adopting the following attitudes and practices while dieting helped me achieve success:

Commit to a lifestyle change, not a brief period of restriction.

As discussed above, this is actually mandatory if you want to keep the weight off afterwards. This means that you need to recognize that you’ll be giving something up (you will have to permanently eat less food), and decide that the cost is worth it.
Commit to weighing yourself every day.

Forming any habit can be irritating, but when your body has been sending you dishonest signals for years, this is really the only choice. Many people who are successful at maintaining weight loss continue to weigh themselves regularly for years afterwards.²⁵ You’ll need to do so at least until your hunger system is retrained.
Don’t declare any food off limits (other than pre-existing dietary restrictions).

There’s no good reason to set yourself up for a cycle of failure and self-blame. One of the most common reasons for dieting failure is that people prohibit certain foods entirely, crave them incessantly, and then binge when the pressure proves too much or they have a “cheat” day. The resulting feelings of shame may drive them away from the diet entirely, especially if the number on the scale goes up the next day. (Usually, this is a temporary result of eating a lot of food. The whole point of using a statistical measure of your weight loss rate is that you don’t have to worry about the daily fluctuations!) Throw out the concept of cheat days; every day is a diet day, but every day is also a day that you may have a doughnut if you absolutely must have one. As long as the line on the weight chart keeps going down at the rate it’s supposed to, you have nothing to worry about.

Try to develop a more healthy decision making process about what you eat. Ask yourself: am I willing to trade being hungry later for eating this sugary thing now? Speaking for myself, I ate almost no added sugar during the diet. It was never prohibited, though. I put a teaspoon of real sweetener in my tea. I had a little square of a chocolate bar once every week or two. I had a slice of pie or a few cookies over Thanksgiving and Christmas. I had a beer every now and then. Nothing off limits.
Never give up.

There will be days when you want to. If you have a tendency towards anxiety or depression, or you use food to cope with problems, you are going to have to face that head on. You’ll have to handle feelings of hunger and reduced energy. It’s even likely that there will be days where you screw it up. The difference between the people who succeed and those who fail is that people who succeed don’t tell themselves that they failed. Failure isn’t what happens when you have one bad day, it’s what happens when you give up.

Conclusions

I lost more than a third of my original body weight. The guarantee that you will lose the weight if you follow the simple instructions output by the program and dutifully weigh yourself every day was enough to push me through the hunger pangs and inclination to give up. Here are the usual before and after pictures.

Picture taken August 2019 at ~243 pounds. I was in a wedding.

Picture taken March 2022 at ~160 pounds. Appearance by choice this time. ☺️

Picture taken March 2022. This is the belt I was wearing in the first picture.

Is this a good way to lose weight? For me, the dominant consideration in answering this question is “is this a successful way to lose weight?”. That’s because most diets are unsuccessful, and so successfully losing a significant amount of weight is already a high bar to clear.

I think it’s reasonable to say that this can be a safe way to lose weight. If losing the standard recommendation of 1-2 pounds a week is a safe rate of weight loss for you, PyWeight will not tell you to do anything stupid. It is merely a tool to help keep you on track. You can choose a slower rate of weight loss and it will come off just as reliably, but will require more patience.

The other question, of course, is can I keep it off? That remains to be seen, but I’m planning to continue weight tracking for the rest of my life while eating at maintenance (net-zero calories in-out every month). This is something PyWeight can also help you do. If you set your desired deficit to zero, any time you begin to gain weight PyWeight will quickly warn you to reduce your intake.

The fact of the matter is that even if I knew for certain that I’ll gain it all back within five years, I’d still do it all again. I really do feel that much better, just as a result of the weight loss. You can’t assess the quality of someone’s life by what they happen to weigh when they die; even if I eventually regain weight I will have bought myself years of enjoyment in the mean time that can’t be taken away. That said, I understand that most people are looking for long term weight loss. I hereby commit to publish an update to this article with my current weight one year from the date this was posted, and then again a year after that.

Thanks to anyone who took the time to read all the way through this. Feel free to ask me any questions that you have.

Addendum (2023):

It’s now been one year since I published this article. Here’s my scale as of this morning:

I think it’s fair to call this a complete success!

PyWeight is free and licensed under the GPL v3, which means you can modify it under certain conditions to meet your own needs. ↩
Hall, K. D., Heymsfield, S. B., Kemnitz, J. W., Klein, S., Schoeller, D. A., & Speakman, J. R. (2012). Energy balance and its components: implications for body weight regulation. The American journal of clinical nutrition, 95(4), 989–994. https://pubmed.ncbi.nlm.nih.gov/22434603/ ↩
The Body Weight Planner by the National Institute of Diabetes and Digestive and Kidney Diseases: https://www.niddk.nih.gov/bwp ↩
One failing of the NIDDK model referenced in [3] is that it doesn’t seem to show this when modeling long term maintenance. When you put in a starting weight of 235 pounds, the model says you would require 3050 calories to maintain. For someone who gains 85 pounds to reach 235, it says they will maintain at 3400. No matter how long you run the simulation, it’s hard to get it to drop this maintenance level significantly below 3400. This is logically impossible. The only way someone becomes the first sort of person who maintains at 3050 is to first be a person who weighs 150 pounds and gains 85. It must be possible to transition from one to the other, although their model does not show this. Moreover, if we assume that your metabolism does eventually adapt, a 100 calorie decrease in your intake should eventually result in a loss of closer to 22 pounds. ↩
https://stateofchildhoodobesity.org/adult-obesity/ ↩
https://www.cdc.gov/obesity/data/adult.html ↩
This sort of sporadic overeating could actually be a helpful adaptation. If you live in a place where food is often scarce, a rare occasion when a fatty or sugary food was available might be a good time to gorge yourself well past the point of satiation, in anticipation of lean months to come. ↩
Hall, K. D., Sacks, G., Chandramohan, D., Chow, C. C., Wang, Y. C., Gortmaker, S. L., & Swinburn, B. A. (2011). Quantification of the effect of energy imbalance on bodyweight. Lancet (London, England), 378(9793), 826–837. https://pubmed.ncbi.nlm.nih.gov/21872751/ ↩
A study found that 82% of people are within 10% of the BMR estimated by the Mifflin St. Jeor equation (including 75% of people with obesity), and more than half are within 5%. See https://pubmed.ncbi.nlm.nih.gov/23631843/. See also https://pubmed.ncbi.nlm.nih.gov/27184275/ for evidence that metabolic issues are not (usually) the cause of obesity. ↩
Fildes, A., Charlton, J., Rudisill, C., Littlejohns, P., Prevost, A. T., & Gulliford, M. C. (2015). Probability of an Obese Person Attaining Normal Body Weight: Cohort Study Using Electronic Health Records. American journal of public health, 105(9), e54–e59. https://pubmed.ncbi.nlm.nih.gov/26180980/ ↩
For example, much has been made of the supposed ability of ketogenic diets to increase metabolism. https://pubmed.ncbi.nlm.nih.gov/16389240/ discusses this and attributes to a low-carbohydrate diet that resulted in ketosis an increased expenditure of less than 100 calories a day. (This was attributed to increased thermogenesis, not any special property of ketosis.) It’s not nothing, but it indicates that most of the success of a ketogenic diet will be from calorie reduction anyway, along with potentially increased satiety. A high protein diet could also potentially provide a feeling of having more energy, resulting in an increase in physical activity. In my diet, I decided to eat plenty of clean protein for exactly these reasons, along with the desire to achieve a complete amino acid profile. I think these reasons make sense, even though I’m not counting on “ketosis” as a weight loss mechanism. ↩
Given footnote [11], you might expect a typically high-carb vegan diet to show the opposite effect (reduced metabolism). Not so. On the contrary, a study looking at the effects of a vegan diet found that it increased the thermic effect of food by 16%, which was associated with a significant increase in weight loss compared to a control group. https://pubmed.ncbi.nlm.nih.gov/31021710/ Despite following a vegan diet myself, I don’t believe the overall results for this or any other diet show that you can “hack” the weight loss process by causing the body to run hotter, at least not significantly. In reading many papers while writing this, what struck me the most was how many of them were limited or flawed outright. I believe it is best to follow a diet that is healthy and sustainable over the long term, even if a few clinical trials would have you believe that one approach or another could net you a 10-20% faster rate of weight loss. On this note, one very large study found that following a vegetarian or high-carb diet was positively correlated with possessing a lower BMI. https://pubmed.ncbi.nlm.nih.gov/11320946/ Thus, regardless of its weight loss credentials, a high-protein diet may not be as successful at weight maintenance. ↩
A calorie-restriction diet can only guarantee success if you actually follow it, which means accurately calculating your energy expenditure (the number of calories you burn every day) and accurately counting the number of calories you eat. Both are hard to do. My methodology gives you a quasi-guarantee of success if you can follow instructions like “eat fewer calories this week than last week”, even if you don’t know your true metabolic rate or can’t count calories accurately. ↩
Hall K. D. (2008). What is the required energy deficit per unit weight loss?. International journal of obesity (2005), 32(3), 573–576. https://pubmed.ncbi.nlm.nih.gov/17848938/ ↩
Ibid. ↩
I mean this in the sense that well-situated people can have a high degree of confidence in their success. Luck is involved in the question of whether you are “well-situated” or not. Many people are not so fortunate: if you don’t have time to cook your own food, are in a “food desert”, have limited access to health care, are the victim of misinformation, or have mental health issues or an eating disorder, then you’re unlucky in the latter sense. The unfortunate fact of America’s public health crisis is that most people are in at least one of these groups. That’s why public health experts don’t see individual behavioral changes (dieting) as the solution to the problem. It’s not that relatively well-placed, high-information people can’t see success (if they approach dieting with a reasonable plan), it’s that most people don’t have that access. So when I say that dieters do have some control over whether they succeed, this doesn’t mean that a dieter who fails has only themselves to blame. Individuals are “free” to quit eating as much in much the same way as you are “free” to quit your job. It’s true, but it misses the point. ↩
Flegal, K. M., Graubard, B. I., Williamson, D. F., & Gail, M. H. (2005). Excess deaths associated with underweight, overweight, and obesity. JAMA, 293(15), 1861–1867. https://pubmed.ncbi.nlm.nih.gov/15840860/ ↩
Flegal, K. M., Kit, B. K., Orpana, H., & Graubard, B. I. (2013). Association of all-cause mortality with overweight and obesity using standard body mass index categories: a systematic review and meta-analysis. JAMA, 309(1), 71–82. https://pubmed.ncbi.nlm.nih.gov/23280227/ ↩
Global BMI Mortality Collaboration, Di Angelantonio E, Bhupathiraju ShN, t al. Body-mass index and all-cause mortality: individual-participant-data meta-analysis of 239 prospective studies in four continents. Lancet. 2016;388(10046):776-786. https://pubmed.ncbi.nlm.nih.gov/27423262/ ↩
Flegal KM, Ioannidis JPA. A meta-analysis but not a systematic review: an evaluation of the Global BMI Mortality Collaboration. J Clin Epidemiol. 2017;88:21-29. https://pubmed.ncbi.nlm.nih.gov/28435099/ ↩
Aune, D., Sen, A., Prasad, M., Norat, T., Janszky, I., Tonstad, S.,Romundstad, P., & Vatten, L. J. (2016). BMI and all cause mortality: systematic review and non-linear dose-response meta-analysis of 230 cohort studies with 3.74 million deaths among 30.3 million participants. BMJ (Clinical research ed.), 353, i2156. https://pubmed.ncbi.nlm.nih.gov/27146380/ ↩
Within reason, of course. If you modify the source code of PyWeight to let you target 5 pounds of weight loss per week, then the program will happily tell you that you need to eat a negative number of calories. Statistics can’t force you to start out with sane parameters, but as long as they are reasonable for someone of your height and weight (e.g. 1-2 pounds a week for most people), PyWeight will guide you onto the proper path. ↩
Craig, W. J., Mangels, A. R., & American Dietetic Association (2009). Position of the American Dietetic Association: vegetarian diets. Journal of the American Dietetic Association, 109(7), 1266–1282. https://pubmed.ncbi.nlm.nih.gov/19562864/ ↩
This basic strategy of eating plenty of vegetables and fruits, along with clean protein and fiber to help you feel fuller, is more or less that recommended by the UK’s National Health Service. https://assets.nhs.uk/tools/download-panels/data/weight-loss/pdf/all-weeks.pdf ↩
Wing, R. R., & Phelan, S. (2005). Long-term weight loss maintenance. The American journal of clinical nutrition, 82(1 Suppl), 222S–225S. https://pubmed.ncbi.nlm.nih.gov/16002825/ ↩

LibreOffice 7.4 has a new approach to text rendering

2022-08-18T17:36:37-07:00

I’ve used LibreOffice for quick word processing tasks for years now — since before it was forked from OpenOffice in fact. However, for years I’ve drafted documents in another tool, before copying them into LibreOffice for the final export to PDF. The reason is that the font rendering in LibreOffice is infamous for how terrible it is, with a large number of issues reporting going back years.

As an example, one of these reports was mine.

Here’s a brief explanation of the problem. Fonts set the amount of space (called “tracking”) around each glyph. Certain combinations of characters do not look right with the default amount of space, and a feature called pair kerning allows a text renderer (like LibreOffice) to adjust this space for individual letter pairs. For instance, the large open bowl of the uppercase “C” looks too empty by default, so the next letter is often brought closer to it.

Unfortunately, LibreOffice’s rendering on low resolution screens was so bad that enabling pair kerning (it has been the default for many years) made it worse. In some cases the failure is catastrophic:

This is a real screenshot taken on my computer. I wrote software to upscale the image so that the problem is clear. Each individual (red, green, or blue) sub-pixel as shown on my screen is being re-rendered at nine times its original size, in grayscale. This helps to capture the effect of sub-pixel anti-aliasing, which uses the shape of the pixels on modern LCD screens to triple the effective horizontal resolution of the text. Here is the original screenshot, for comparison:

Pair kerning is implicated here because it is actually this feature, when enabled, that turns “California” into “Califomia”. The font shown here is Liberation Serif 12pt, at 120% page zoom. Hinting is disabled system-wide.

Why is text rendering on LibreOffice 7.3 so bad?

The problem was caused by an approach to text rendering that predates the introduction of the codebase into version control in the year 2000. Back then, text renderers were rather stupid, and did not do a good job of rendering strings of text in a readable way on the screen. LibreOffice (then OpenOffice) therefore did its own glyph positioning.

LibreOffice knew where the glyphs should go (called the printing position, because it was where the glyphs would be placed when sent to a high resolution output like a printer), but just sending these “ideal” locations to the text renderer would result in subpar text.

On the other hand, if LibreOffice rendered the text at the preferred screen position, the width of the text would not match up with the printer’s output (because the screen metrics of the font differ, e.g. due to hinting). LibreOffice is a WYSIWYG (what you see is what you get) editor, so this is unacceptable. Lines that exceed the word processor’s margins must reflow, so you would end up with cases where the printed document would have different lines than the document as shown on the screen.

Instead the authors adopted a compromise: 75% of each glyph’s placement is determined by the ideal position according to the screen metric, and 25% is determined by where the glyph would be placed if its right border was aligned with the right border of the ideal printing position. WYSIWYG is made possible by realigning the text with the printing position every time white space appears in the text.

Unfortunately, the end result is not very good by the standards of a decade later. On a low resolution screen like mine, pixels are quite large relative to the size of the glyphs:

The above shows close to ideal kerning for the r/n letter combination for this font at 12pt on my screen. As you can see, being off by one pixel is enough to make “rn” into “m”.

Despite LibreOffice making its best effort to place glyphs intelligently, most of that work was thrown away at the last stage in the rendering process, because the glyph positioning engine did not support sub-pixel positioning. This is a different concept than sub-pixel anti-aliasing. While the font renderers used by LibreOffice have generally supported the latter, sub-pixel positioning support is much more recent. Sub-pixel anti-aliasing (sometimes more generally called sub-pixel rendering) is the use of the thin red, green, or blue components of LCD screen pixels to improve the resolution of text. Sub-pixel positioning uses anti-aliasing (potentially including sub-pixel anti-aliasing) in order to render text at positions more precise than a single pixel.

This is crucial to the proper appearance of text. Because pixels on low-resolution screens are so (relatively) large, rounding the location of glyphs to the nearest pixel generates consistently bad results. Combine that with pair kerning (which tends to move glyphs closer together, and also changes the heuristic described above to favor the printing position), and you’re asking for catastrophic failures.

What does LibreOffice 7.4 do differently?

For at least the last 22 years, LibreOffice has done its own glyph positioning under the assumption that the underlying font rasterization engines were not capable of creating high quality text on LCD screens when passed the “true” positions of each character (i.e. the printing positions).

This was true for most of LibreOffice’s history. After all, while rendering has been acceptable on Linux for at least a decade (with the proper settings and a recent enough FreeType), commonly used libraries like Pango / Cairo did not support sub-pixel positioning until very recently. Qt was ahead of the game for a long time, supporting it since some time around 2015.

During the LibreOffice 7.4 development cycle, Caolán McNamara experimented with this behavior, initially adding an option to just kick the printing positions down to the rasterizer and let it place them on the screen using its own algorithms. The result was successful enough and fixed so many bugs that it was made the default mode in LibreOffice 7.4.

Because the underlying rasterizers now support sub-pixel positioning, this change results in LibreOffice getting that for free, in addition to the intended change of eliminating the buggy positioning code from the render path.

The results are breathtaking:

It’s rare to see such an enormous improvement in a piece of software in a minor version bump, and this one appears to have gone almost unnoticed. It’s even rarer to see an improvement like this with a simplifying change, in this case one that makes a bunch of old code irrelevant.

My thanks to Caolán McNamara for working on this, and for backporting the patches to LibreOffice 7.3 at my request. LibreOffice is now a viable choice for me when drafting documents.

I hesitate to share the following before and after picture of some text at the original size for fear it may be misconstrued. Don’t expect this text to look great on your system, even if you view it at 100% zoom (which you should — right click to open the image in a new tab). Font rendering happens at a very low level, relying on details like screen DPI and pixel orientation that may vary from screen to screen. This is also a bit of a stress test, rendering some serif fonts with many details on a low resolution device at 12pt.

Rather than deciding which rendering looks better to you overall, focus on the differences in glyph placement. Where are the glyphs too close together? Where are there ugly gaps? To my eye, LibreOffice 7.4 has significantly reduced severe positioning errors (“met” in the fourth line of the Garamond sample) while achieving superior rendering even in cases where LibreOffice was already acceptable (every “th” pair is slightly too close in Caslon), thanks to sub-pixel positioning.

Improving font rendering in old UI toolkits with a one line Pango patch

2023-04-18T21:46:09.635740+00:00

Context

First let me give enough technical information to make this blog post comprehensible to newcomers, and then dive in with a little bit of history. Those already familiar with how font rendering works can skip the next section.

Font rendering basics

Most computer screens are made up of little square pixels. On most of these screens, each pixel is composed of three subpixels — vertical columns of red, green, and blue LEDs (usually in that order). In other words, the entire pixel is never “blue”; a pixel appears blue to you when its blue subpixel is turned on, and its red and green subpixels are turned off.

Modern fonts are vector based. This means that instead of being defined in terms of which pixels are to be turned on, and which are to be turned off, the font designer instead specifies precisely the shape of the corners and the curves for each glyph, and the font rendering system is responsible for deciding which pixels to activate. This is why you can zoom into the text on a webpage without it looking blurry or blocky.

So how does the font rendering system decide whether to turn a pixel on or off? Well, the simplest way to do that (for black text on a white background) is to pretend that the pixel is a square. Ask what portion of the square is “covered” by the glyph you are drawing; if that number is greater than 50%, make the pixel black. Otherwise, make it white. Unfortunately this looks terrible. The pixels on almost every screen are large enough that if you draw fonts this way, you’ll be able to see the blocky pixel edges.

The solution is called anti-aliasing. As a bit of a simplification, you can imagine that instead of drawing each pixel black or white, you instead take the percentage of the pixel that is covered by the glyph and make the pixel that fraction black. So if the 75% of the pixel is covered by the glyph, you set the pixel to 0.25 * maximum brightness (all white).

This is complicated slightly by subpixel anti-aliasing. On many screens, especially older ones, the pixels are still large enough that the text is hard to read or blurry when you render this way. However, remember how I said that each pixel actually has three vertical subpixels? What if your renderer treated each of these as separate for the purpose of anti-aliasing? It would then be able to increase the horizontal resolution of text by 3 times! That’s exactly what subpixel anti-aliasing is.

Done naively, this results in strong color fringing on the edge of characters, since if the left edge of a character is just barely in a pixel, you have to activate the “blue” subpixel (on the pixel’s right edge), and not the green or red ones. So it’s usually combined with what’s called LCD filtering, which is a set of complicated techniques for adjusting the anti-aliasing so as to minimize these color fringes as much as possible. The resulting approach has been around for decades now, most prominently as Microsoft’s ClearType. More generally, it’s also sometimes called subpixel rendering, although I find that label unhelpful.

I say all that because these techniques are well known, and it’s important to clarify that I’m talking about something different that has been around on most operating systems for much less time. Let me introduce this concept with a new question: where on the screen can you place a glyph?

This may seem like a silly question, but hold on. My screen has 1920 pixels in each row. Does that mean that there are 1920 positions at which a font renderer could insert an “a” character, for instance? No. Imagine looking at a book, which we’ll say is printed with traditional hot metal typesetting. No pixels were involved anywhere in this process. There was only the right place to put the “a”, and an infinite number of wrong places. Will the right place to put each glyph always align with the same position relative to the pixel? Of course not!

But that’s exactly what most font rendering engines assumed, for years and years. (Turns out there are good reasons to do this: it improves performance by reducing the size of the font cache.) “Wait,” you might say. “Old screens had huge pixels! Didn’t this look terrible?” Yes. Yes it did. But there were bigger problems; with huge pixels, complex vector fonts can be very hard to read. To fix this, font renderers adopted a technique we haven’t discussed yet called hinting. This basically amounts to mangling the shape of each glyph so that it aligns better with the pixel edges. The result is sharper, sometimes easier-to-read text. Strong hinting is a key component of ClearType, whereas macOS is famous for doing little or no hinting in order to better preserve glyph shape. For our purposes, all that it’s important to know is that after distorting glyphs to fit the screen pixels, it doesn’t matter so much what the correct pixel offsets are. Getting them right might even be counterproductive if it makes the glyph blurrier.

So for a while, there was little reason to bother with subpixel positioning (as opposed to subpixel rendering, discussed above). But times changed, screens improved, and now it’s important. This is in part because hinting has fallen out of fashion, except on desktop Windows.

If you would like to learn more about these topics, the best introduction I know is the slightly outdated Treatise on Font Rasterisation.

A brief technical history

In 2019, the text rendering library Pango finally released with support for subpixel positioning, enabled by default. This had made many people very angry and has been widely regarded as a bad move.

The issue is that Pango depends on another library called Cairo, and at the point when Pango released this feature, Cairo did not yet support it. The result, if you had the most recent releases of both Pango and Cairo, was “unpleasantly uneven glyph placement” to quote the Pango developer. As a consequence, they reverted the change in Pango 1.44.3. Unfortunately, this decision was never revisited. Programs (and UI toolkits) must still opt in to subpixel positioning, despite the fact that a version of Cairo that supports it was released years ago now.

Recent UI toolkits (like Qt or GTK) either enable subpixel positioning support in Pango (GTK) or implement it themselves (Qt). However, not all software uses the latest versions of these toolkits. How serious is this problem?

For Qt it’s not that bad. Both Qt 5 and 6 support subpixel positioning, which covers the vast majority of applications. However, only GTK 4 enables it. That leaves GTK 2 and GTK 3. Is this an issue in practice?

As of April 2023, on Arch Linux,

86 packages require GTK 4
561 packages require GTK 3
125 packages require GTK 2

Note that this is only a rough estimate, for a variety of reasons. It suggests a bad state of affairs, nonetheless. Only ~11% of packages that require one of GTK 2/3/4 depend on GTK 4. This is surprising, since the last major release of GTK 2 was in January 2011, and it was officially end-of-lifed in 2020, but it’s what we have to deal with. It probably makes sense to plan on supporting GTK 3 applications indefinitely.

I asked the GTK developers about adding support for subpixel positioning to GTK 3, but they said that development focus was entirely on GTK 4 and that GTK 3 wouldn’t receive new features. That puts us at something of an impasse. The Pango developers haven’t made subpixel positioning the default. GTK developers are only working on GTK 4. And the vast majority of GTK applications are using GTK 2 or 3.

Let’s fix it ourselves

Applications using pango are supposed to call

pango_context_set_round_glyph_positions (context, FALSE)

to disable rounding glyph positions to whole pixels. If an application doesn’t set the option, it falls back on Pango’s default (rounded positions, so no subpixel positioning). Hold on, though. The legacy UI frameworks aren’t explicitly disabling subpixel positioning — they’re just not enabling it. So if we change the default in Pango, we’ll immediately get everything using Pango working, for free!

Turns out this is a one line code change.

diff --git a/pango/pango-context.c b/pango/pango-context.c
index cbc4f173..82b44136 100644
--- a/pango/pango-context.c
+++ b/pango/pango-context.c
@@ -69,7 +69,7 @@ pango_context_init (PangoContext *context)
context->set_language = NULL;
context->language = pango_language_get_default ();
context->font_map = NULL;
-  context->round_glyph_positions = TRUE;
+  context->round_glyph_positions = FALSE;

context->font_desc = pango_font_description_new ();
pango_font_description_set_family_static (context->font_desc, "serif");

Starting applications after compiling in this change was nothing short of stunning. I think my jaw actually dropped. (Note: the “large text” images are redrawn from actual screenshots by treating each vertical subpixel like a 3x1 grayscale pixel.

The Quod Libet music player (gif):

The GIMP about screen (before):

The GIMP about screen (after):

A gif close-up. Note: since my blog shrinks images to fit in the margins, the best way to look at this image is by opening it in a new tab.

That’s a huge difference! I’m going to add Pango to the slowly growing list of packages I have to build myself to include important patches. The results, though, are worth it.

They static now? They static now!

2024-02-27T22:42:20+00:00

Origins

When I started this site back in 2014, I only had a couple of years of experience writing Python, but I wanted to try writing a real (meaning “dynamic”) site, so I went with the Flask framework.

The site started as a single Python file and a few static CSS files and Jinja2 templates. When I did the conversion this weekend, it was still a single Python file, a few static CSS files, and the same Jinja templates.

My early inexperience resulted in a site with some odd design decisions, and in the first year or two I added features (like an RSS feed) by adding more functions to my single file. Don’t get me wrong, it’s cool that you can write a real web server so easily, but the result was an unmaintainable mess with some very questionable decisions.

One early decision I made was to avoid using any kind of database. My blog posts were (and are) simple Markdown files. To add a new post to the site, I simply copied it to a directory on my server and the Flask application did the rest. I went with the easiest possible solution to most problems; for example, the URL paths to individual posts were simple numbers counting up from the first post, /post/0 for example.

This led to problems, because the post numbering is just determined by the .md files the Flask application finds. Flask is not an especially friendly framework for, say, running a background task at a specific interval and setting a global variable containing the markup for all the blog posts it finds. The easy solution is to look up the files every time someone visits the page, so that’s what I did.

Even worse, there’s not a one-to-one relationship between the post paths and the Markdown files on disk. If someone sends GET /post/5 then what post does that represent? For all the Flask app knows, a new post has been added with a modification time in between what used to be posts 3 and 4, so the only thing to do is reload every single post on every single page view.

I partially rewrote the application in 2019. Because the code had turned into such an ugly mess, it wasn’t fun to work with, so I once again turned to easy solutions. I put a bunch of caching in front of the heavy calls, so that only the first request for a post would be slow, and then the rest would be fast, inside the cache window. I also finally switched to URL paths based on the title by default. This still didn’t provide a one-to-one relationship between URLs and Markdown files (since characters that aren’t safe get removed), so this didn’t solve the underlying issue of having to fetch all the posts on every page view.

For a long time, I intended to migrate the blog off the old server (running Ubuntu 18.04!) to a new one. Over the weekend, I finally got around to this, and in the process fixed the issues with the site.

Changes

My criteria for the transition were the following:

The site must not visually change.
The site should be static HTML.
The static HTML for existing posts must not be created via scraping the old site and saving the files.
The process for writing new posts should be relatively painless.

The original development of the site, as I have said, was a learning opportunity for me. I’ve made other (and much better) sites since, and these don’t have quite so many problems! I was stuck on the old site primarily because the obvious transition (to static HTML) seemed so difficult given my requirements.

When I finally found the time, I decided to try an interesting approach. Rewrite my existing code once more, to remove bugs, and also remove the dependency on Flask. Instead, I have a main() function that collects changes to the source files and renders them via the same functions called by Flask. The resulting HTML is then written to disk in a directory specified by a config file, and I rsync the result to my server.

Overall, this turned out to be way easier than I expected, and in fact I fixed quite a few issues with the legacy code in the process.

Perhaps most importantly, I decided to just use Jinja2 directly. This allows me to use my existing templates, meaning that the site doesn’t visually change at all. I performed some cleanup, but in other respects this required minimal work.

You can replace Flask’s ubiquitous render_template with a simple shim:

    def render_template(template_name, *args, **kwargs):
        template = JINJA.get_template(template_name)
        return template.render(*args, **kwargs)

JINJA, here, is just a global Jinja environment.

Perhaps most importantly, I’ve developed a workflow for shipping Python apps that I’m pretty happy with. With my deployment scripts, I can recreate my sites on a new server in a matter of minutes.

I’m also no longer relying on file modification times for my “last updated” dates on posts, thanks to the meta extension for Python-Markdown, which I actually added back in 2019.

Anyway, this update isn’t terribly interesting to most people, I’m sure. I’m writing this to reward myself for getting it done. I do think the idea of rewriting existing server code to just render templates is a good one, so if you’re in a similar situation with your own site, perhaps give that a try.

Why Dune II undermines its protagonist's achievements

2024-03-08T21:27:48+00:00

DUNE

I wanna talk about Dune, and this is my blog, so we’re gonna talk about Dune. This post will contain spoilers for the films Dune I and II. Please mind the caveat that I haven’t read the Dune books, and I’m unfamiliar with most writing, academic and otherwise, about Dune.

There is a lot to say about Dune II. The film seems to invite comparison, criticism, and debate. Obvious responses range from the perfunctory — “ambitious” — to the superlative — “the most ambitious fantasy epic since The Lord of the Rings.” True, but doesn’t capture what makes it interesting.

There’s enough ambition in recent cinema. The arthouse and import markets are more widely accessible than ever. Even in the mainstream, Martin Scorsese is making three-and-a-half hour period pieces for Apple TV in English and subtitled Osage. Dune II is long, and it’s slow paced, and that makes it, if not ambitious exactly, then at least proof of what the popcorn munchers are now willing to put up with. (In my case, I also put up with a projector with a dozen dead pixels.)

In reaction to this emphasis on Dune II’s artistic pretensions, one might espouse the notion that the film succeeds primarily as pure entertainment. I mean, yeah, it’s really good? I don’t claim to know what brings people like me back to the theater. Certainly not two hours of inhaling our late arriving neighbor’s perfume. I enjoyed every minute of Dune II, that much is true. Judging the film purely on this axis is overly simplistic, though. The way I enjoyed Dune II is more like the way I enjoyed Ben-Hur than Mad Max: Fury Road.

I want to try out some ways of characterizing the kind of story Dune II tells, and try to work out how this kind of storytelling might be at the root of what makes the film work.

Kings and Queens

Royalty feature heavily in theater. Drama’s love of kings derives not (or not only) from their patronage but more directly from the fact that they are Subjects, writ large. The dramatic monarch is a being with nearly unlimited power to shape the world. They enact the ideal of a human being in that their stories employ a dichotomy of will and world in clean schematic form that comports with how we understand ourselves.

There is a terrific film, Deux jours, une nuit, which illustrates this point by way of contrast. In the film, the bosses of a manufacturing plant manage to fire Sandra, a young mother on medical leave, by bribing her coworkers to vote for her dismissal with a bonus. She rejects the result as illegitimate because they voted publicly, and pressures her manager to redo the vote with secret ballots. Granted this reprieve, she spends the weekend — the remainder of the film — trying to convince her coworkers to vote for her come Monday.

Sandra is a woman with no power in a desperate situation. The film depicts human comradery and the material forces that break it down, and asks how, or whether, people can have a sense of freedom when they lack control over their circumstances.

The characters of classical drama have moments of power and success, weakness and failure, they see the fulfillment of desires and reversals of fortune. Many a royal Macbeth has faced his Birnam Wood. These men and women nevertheless have their wills and desires sharply defined; the world is shaped by them and their contests, like landscapes carved in a war between giants. They are maximally free in that they possess the capacity to determine the conditions they will face.

Sandra resembles most modern protagonists in that her story defines its world without reference to her; she is not active in its construction. Her story is not about her efforts to shape this world. Instead, it forms an unalterable backdrop to her struggle against its effects.

Most modern characters reside somewhere in the middle of the spectrum I have drawn between deific beings who shape the world with their wills and powerless humans buffeted about by an environment they cannot hope to control. That said, we have largely relocated the former to the realm of fantasy.

Perhaps the superhero comes closest to enacting this character type in its original form. They are never seriously threatened by ordinary people, who exist in their stories as a kind of foil. Their worlds are stages for combat, shaped physically by their conflicts. Yet these characters are not free to act in the way of the queens and kings. Their limits are set not by the world itself so much as by metatextual considerations.

A “hero”, for example, is by definition a character possessing goals aligned with the survival and well-being of ordinary humans. Villains possess the opposite set of motivations, and their stories are about their attempts to pound one another into submission with raw force, not to win a contest of wills or political intrigue.

By contrast, the real life King Henry II participated in two major controversies. The first was his effort to define and cement his power as monarch in contradistinction to that of the Church, a struggle that ended in the murder of the Archbishop Thomas Becket. The second was a feud between himself, his wife Eleanor, and his sons over his possessions and the inheritance of his kingdom. This boiled over into military conflict when several of his sons, along with Eleanor, rebelled against him.

With comic book characters, at least on the big screen, “their” interests are never really their interests. They are never petty. They do not engage in decade-long quarrels with the Catholic church over the right to levy taxes. Even anti-heroes, in addition to their propensity to violence and unsavory language, are by definition among those who show up when it’s time to get the gang back together to save the Earth for the umpteenth time. Their interests are still ours, they’re just unreasonably grumpy about it.

No one really believes in this royal character type any more. Power like this is parochial at best, and fleeting. Fictional worlds shaped entirely by their characters are the stuff of the stage, and this stage, today, is a small one. George and Martha dominate the terrain of Who’s Afraid of Virginia Woolf?, but this is possible because they are giants only in the space of their small home.

Even our ideas of history no longer focus on the capacity of individuals to shape their present and future. “Great man” theories have no traction, and this limits our capacity to accept them in fictional contexts as well.

The Lord of the Rings in space

Aragorn possesses the kingly virtues, by right and by nature, and governs the world as he wills. Simultaneously, the Rings books accord him heroic status and rule of Gondor by inheritance, but modern readers will fail to see a connection between these two facts. If Aragorn had been a bad king, as many of the Numenoreans were, Gondor would have had to tough it out, and perhaps even fallen into decay. If he had lacked the capacity for the leadership others entrusted to him, he would never have become king in the first place.

As readers (or viewers of an adaptation), we accept this account not because of any particular theory of kingship but because we understand Aragorn as a traditional protagonist. Aragorn is the only human being who can use the palantíri without corruption. Arthur takes the sword from the stone. Luke is the only Jedi with the power to confront Darth Vader. Thor is the only guy who can lift the big hammer.

If, as I have suggested, we struggle to accept these characters, how does Tolkien avoid making Aragorn’s story risible? In two ways: the inclusion of antagonist characters of equal or greater power, and with a extratextual imposition curtailing the ways Aragorn’s power can be expressed. Gandalf, Galadriel, Aragorn, and Elrond could potentially mount a frontal assault on their enemy, Sauron, but the moral cost would be too great. Because they are restrained, the focus of the story lies instead on the actions of a couple of nobodies who destroy Sauron’s power through an Achilles’ Heel mechanism.

Paul Atreides seems cut from similar cloth. Never, in Dune II, does there seem to be a credible material threat to his aims. He starts the film down and out, of course, but people instinctively flock to his leadership. His family’s military commander, Gurney Halleck, a William Marshal type, turns out to be alive and to know the secret location of his family’s stash of nuclear weapons. He has a willing army of millions in the south, and his opponents are bumbling idiots or smirking capricious newcomers.

Like Aragorn, Paul holds back because he knows what will unfold if he uses these powers wrongly. Something like two thirds of Dune II features him wandering around in the desert trying to escape destiny because he can foresee the violence and suffering that will come as a result of it. He’s the rightful heir of the house of Atreides, but can’t claim it directly. Like Aragorn, his coming is prophesied and awaited by the faithful.

Is Dune just Lord of the Rings in space, then, with guns replacing swords (wait, the Fremen use swords!)? Well, no, obviously not. There is no substitute for the cental Paul plot, no ring to put to the fire and set all aright. Eventually, as his mother predicts, he follows his fate to the south. From there, the narrative moves remarkably quickly. He wins the fundamentalist Fremen sects to his side in mere days with his fulfillment of prophecy, devises a plan to use nuclear weapons on his enemies, leads an assault on the capital, captures the emperor, and kills his one rival for the throne.

We accept this narrative because Dune II, as it tells this story about Paul’s rise to power, is simultaneously deconstructing narratives about rising to power. The prophecy itself is a sham, having been invented by his mother along with a religious sect in order to win him followers. He must undergo trials to prove himself, like drinking poison, but he has been trained to be able to drink poison and live. He promises to lead his followers to “Paradise”, but this is visually undercut by a shot showing them boarding starships to go to war with his enemies. His choices, especially his use of atomic weapons, results in his girlfriend breaking up with him, and she, no Arwen, is a fiercer warrior than he.

Dune II redeems the overpowered nature of its lead character with bitter political cynicism about where that power originates and how it is sustained. When Immortan Joe promises his followers “you will ride eternal, shiny and chrome”, it is not so important whether we believe him. A belief in an afterlife, a metal Valhalla, is culturally appropriate for the War Boys. When Paul promises the Fremen to take them to green places with enormous oceans, it matters that we see cynicism. Belief and its power are the thematic core of the film, and if Paul is not the messiah, the Fremen seem poised to make him so through sheer conviction. But there is a difference, and an important one, between the two.

Much of the dramatic tension of the film comes from Paul’s attempts to prove himself, as when he rides one of the planet’s giant sandworms, but to what end? The effect of these actions is to convince the Fremen that he is their prophesied messiah, and he’s not. Even if he manages to do everything that the messiah is supposed to do, it’s still the enactment of a story told for the purpose of manipulating the Fremen, and gaining power over them, for generations. What good is that to them?

Paul’s fortunes change the moment he embraces this “destiny”. Rather than fighting a war of attrition in the north, in a matter of days he’s reclaimed the place of his family on top of Arrakis, and has his sights set on even greater ambitions. On the scale of a rather plodding film, it’s hard to convey just how fast this seems to happen. There’s little depiction of the difficulty of moving an army of great size hundreds of miles north, nor of the practical concerns with taking the emperor captive. Even the fighting, carefully choreographed in earlier scenes, becomes perfunctory here — two lines of foot soldiers charging headlong into each other, with the outcome already decided.

The Dunepire strikes back

It’s at this point in the film that, as in The Empire Strikes Back, the fate of the galaxy is determined by single combat. This borders on the farcical. The emperor appoints a member of the Harkonnen family, a young man he’s only just met, to be his champion. This poor kid never had a chance. The movie, all too late, tries to set him up as a big-bad worth fearing, but doesn’t (and can’t) succeed, because Paul’s power by this point verges on absolute. He can literally see the future, now thinks in necessities instead of possibilities. Paul’s triumph is the expected victory of brains over brawn, though the film establishes that he has that too — the very first thing we see him doing in this series is practicing combat.

The film bears other comparisons to Empire as well. A central component of Paul’s transformation from insouciant desert vagabond to contender for the throne of the empire is the discovery that he is the grandchild of Baron Harkonnen.

When Luke Skywalker is revealed to be the son of his enemy, Darth Vader, the purpose of the moment is to justify an unexpected choice: Vader offers Luke an alliance with the goal of overthrowing the emperor. Downstream from this immediate impact is Luke coming to have the belief that Vader is not entirely evil, and that their relationship could be the basis of his redemption.

This moment spread outward from Empire to become a cultural touchstone. In Empire, the fact of their relationship gives us an explanation for why Vader does what he does. Rather than simply kill Luke, he comes up with a solution that would unite them while still achieving his aims. Likewise, the moment helps us understand why Luke treats Vader as redeemable, rather than an enemy to be destroyed at all costs. When this took on cultural significance, the meaning changed. To reveal that the protagonist is related to the antagonist is to imply not merely that they can change to become like one another but that they are also, fundamentally, already alike in some respect.

Relying purely on established cultural meaning in this way is lazy storytelling, and unfortunately Dune II doesn’t rise above it. Paul does not take his ties to the Harkonnens as a reason to believe he can reason with them, or to give him a moral obligation to win them away from their violent ways, but as reason to believe of himself that “fighting like Harkonnens” would be appropriate. This makes no sense at all in the context of the story; as audience we only comprehend this because we know the cultural signification of revealing a protagonist’s heritage. Even if we take for granted the racial assumption that Harkonnens are inherently violent, this at best gives us an explanation for why Paul might be prone to violence (though the film seems to have established that he is not). It does not give Paul himself a reason to start behaving more violently than he otherwise would have!

In the context of Paul’s arc, this revelation is the inflection point in a radical transformation. Paul begins the film determined not to engage the fundamentalist Fremen because of his visions, which prognosticate mass suffering and death if he does so. He reverses course only after the Harkonnens destroy the Fremen’s northern stronghold, making it clear that a long painful defeat will result from his current plans. He drinks a potion which radically enhances his powers of foresight, at which point he immediately embraces his apparent destiny as Fremen messiah figure and future ruler of the empire.

Paul defends his change of plans to his Fremen girlfriend, Chani, by stating that the path he takes is the only way. Unfortunately, we have little means by which to judge this claim. We’re shown only fleeting images from his visions, and they not only don’t explain his actions, it isn’t even clear what he means by the claim that no other choice is possible.

No other choice for whom? Whose aims does he prioritize? If Paul takes the path he does because otherwise the Harkonnens will remain in control of Arrakis and his father will remain unavenged, but trades for this the collapse of the empire and a horrific civil war, that’s an interesting thing for him to do. It reflects the private ambitions and unique loyalties that we expect from a royal character. Dune II, instead, muddies the water by giving us no clear sense of what choice, exactly, has been made.

Chani, understandably enough, dislikes this profound change in Paul’s personality and drive. She takes on the role of viewer insert in the film. We’re clearly meant to disapprove of Paul’s exploitation of religious fanaticism, and Chani embodies this. In fact she does so too much. Dune II deftly emphasizes showing over telling, but it’s possible to “show” so much that the audience feels shouted at, and that happens with many of Chani’s scenes. Nearly every time Paul does something we’re meant to feel skeptical of, we’re handed a cutaway to a reaction from Chani, who emotes her strong disapproval. We don’t need this level of confirmation that, in fact, the movie knows it’s a bad thing to seed prophecies about yourself in an isolated community over multiple generations just so you can come along and fulfill them.

Would you still love me if I were a worm?

Chani promises Paul that she will stick by him so long as he “remains true to himself”. Her rejection of him when he changes course is the most overt sign that he’s gone astray.

While this allows her to serve as the film’s moral center, it also frustrates our efforts to see her, like Paul, as a complete person. How would she respond, for example, if Paul told her that his visions revealed that the Fremen would be wiped out should he not take the specific path he does? I think we have to imagine, hewing as she does to righteous means, that her opinion would be the same. Yet the film misses the chance to give her depth by having her struggle with questions like these. Everyone else in the film is granted a point of view, while she is burdened with carrying the point of view of the filmmaker.

Paul and Chani consummate their relationship in a scene immediately after he successfully rides one of the giant sandworms of Arrakis for the first time. This is not just amusing symbolism, but another instance of Chani serving as audience surrogate. Timothée Chalamet has, so to speak, wooed us with his feats, and Dune II consistently succeeds in its efforts to feel like a true adventure film, with Paul a swashbuckling protagonist despite our misgivings.

Villeneuve’s chops as an action director shine through here; there’s none of the grunting and panting that typically conveys effort in blockbusters; here the whole story is told with the camera. Paul’s platoon of believers proclaim him the “Mahdi” (one of several elements in the film clearly inspired by Islam), and the brilliance of the film is such that for a minute we feel that they’re right. We want them to be right.

The subservient attitude of Paul’s Fremen acolytes points up Chani’s insistence on her own independence. “I’d like very much to be equal to you,” Paul tells her early in their relationship. This line sticks in one’s memory precisely because the world of Dune is such a thoroughgoingly unequal one. There is absolutely no pretense that Paul is the equal of anyone else in the story. He is a Duke, the heir of the house of Atreides. He has preternatural gifts of leadership, prodigious fighting skill, and strong tactical capabilities. He is literally born to play the specific role he does, with his future-sight giving him textual awareness of his own position which serves as a kind of plot armor, which the film emphasizes while simultaneously deconstructing. If he has “main character syndrome”, it’s because he is the main character, and he knows it.

Outside of Paul himself, the universe displays overwhelming inequality. With the empire carved into family-run fiefdoms, brutality becomes the necessary condition of survival and success. As the emperor explains to Paul, he killed Paul’s father “because he was weak”, where “weak” serves primarily as proof of incompetence. Villeneuve does not depict the emperor as especially evil, but as a cunning and bureaucratically capable man placed at the top of a world in which violence is the only reliable mechanism of governance.

When Paul declares himself the equal, or (rather) aspiring equal, of Chani, what does this “equality” comprise? He seems to gesture toward liberal notions of political equality that have no place in this world. Paul is, after all, an outsider. He comes among the Fremen as an obvious Lawrence-of-Arabia type, promoting the idea that they will guide themselves to freedom, but always with the implicit premise that he will be at the front as their leader and inspiration.

Perhaps he means merely sexual equality? This seems plausible enough, in light of the relative good treatment of women by the northern, non-fundamentalist Fremen. Yet he cannot mean this in a way that comports with modern feminist notions (a brief joke about mansplaining notwithstanding). Fremen society remains heavily segregated, and emancipation for the women of Arrakis never becomes a serious suggestion.

The women of Dune

Moreover, never does the enslavement of women we see in the world of Dune become a political subject. Paul wants revenge on the Harkonnens, and (at least secondarily) victory for the Fremen, but we have not a shred of evidence that the condition of Baron Harkonnen’s slaves ever crosses his mind.

When women are, like Chani, depicted as political actors in this world, it is in their capacity as individuals. The collective interests of women have no significance. Comparing this to how seriously the film takes the status of the Fremen illustrates the severity of this omission. Their ability to defend themselves and make independent decisions features centrally in the film, and Paul’s status as white savior serves as a clear threat.

Dune II certainly likes to point out to us when women are being mistreated, but in the absence of any serious attempt to make this relevant to the actions and decisions of the characters, these depictions serve crudely as mere illustrations of the violent, despicable natures of the film’s antagonists. Contrast Fury Road: George Miller wants us to ask what women’s freedom looks like under conditions of chaos and collapse, what it means to be unified against misogyny.

Political power for women in Dune comes primarily in the form of membership in a religious order, the Bene Gesserit. The official power structure of the empire flows from the emperor through his vassals, a hierarchy based on authority and brute strength. This contrasts with the subtle methods of the Bene Gesserit, who work by influence, manipulation, and subterfuge.

Their goals remain vague, at best, in Dune I and Dune II. They, in conspiracy with Paul’s mother Jessica, have laid the foundations for his power by spreading the prophecies among the Fremen, while at the same time attempting to bring about the birth of one who could fulfill them.

As their plans for Arrakis materialize, they seek to determine whether Baron Harkonnen’s nephew Feyd-Rautha can be manipulated to their interests. (He is, it turns out, easily taken in by attractive ladies.) This episode serves to illustrate the vast power of the Bene Gesserit. Their schemes span across decades or centuries; they have the ability to directly manipulate the minds of others; their envoys have the ear of those with formal power, including in the halls of the emperor. It’s not clear, in other words, why they’re not already running things. Could it be that they are restrained to passitivity by an authorial intervention?

Paul, oddly, is a feminized character. His mother breaks the rules of the Bene Gesserit (who admit only women) because she wants to give him an edge, teaching him to use the Voice — the power to cause others to instinctively obey through speech. Likewise, Paul becomes initiated in Dune II to their most sacred ritual, drinking the Water of Life, which places him in a profound trance and gives him the ability to see the future. We’re introduced to this act in the film when Jessica does it, in order to become one of the order’s spiritual leaders (a Reverend Mother).

Paul’s decision to drink the liquid takes on more significance from the fact that doing so is fatal to men. The fundamentalists interpret his survival as another proof of his messianic status, but more directly, it codes Paul as female. His potency derives from his ability to utilize the “feminine” powers of the Bene Gesserit in the male spaces of the empire. The profound transformation in his attitude and behavior derives, as we have seen, from the insight wrought by the Water of Life (Dune’s “tree of knowledge”, to which his mother beguiles him). He likewise proceeds to use the Voice to silence his enemies, many of whom believe that his participation in the rites is heretical.

The climactic fight between Paul and Feyd-Rautha involves an opponent who has been decidedly masculinized. The young Harkonnen enters the film as a bare-chested gladiatorial fighter, who haughtily disables the energy shield protecting himself out of sheer machismo. His weakness, naturally, turns out to be sex. In terms of military strategy, he rains terror from above, preferring to annihilate his opponents entirely rather than outmaneuver them.

What is the significance of this all-female order achieving political power, at long last, through a man? This move mirrors the “Lawrence-esque” elements of the plot; as the white Paul enters the Fremen world as messiah, so he enters the female space of the Bene Gesserit as political agent. To its credit, this is once again something that Dune II tries to deconstruct. If Paul does what the Bene Gesserit cannot, it is in some sense as their tool, and the consummation of their plans. The film’s examination of this does not appear particularly thorough, as we’re left to guess at what they hope to achieve.

Jessica, in her dual role as Paul’s mother and Bene Gesserit Reverend Mother, displays more agency than the other women of the film. She serves their aims while also preferentially aiding Paul. Over the course of the film, she changes from cynical promulgator of Paul’s cult to something of a true believer. In Dune, the power of a collective mass of people derives from faith, and so long as the Fremen trust in Paul, he can (perhaps must) become the person they believe him to be.

This fact is suggestive of what the film says, in general, about heroes and actors of the “regal” type more generally. They are, at the end of the day, the crest of a wave built on thousands or millions of people who really believe in something. The Bene Gesserit are an organization that understands this principle and manipulates it to their own advantage. The film makes this point clearly enough, and it is only its functioning in gender political terms that it leaves uncomfortably hazy.

Does the movie need a traditional villain?

The interest the Bene Gesserit take in Feyd-Rautha is never particularly coherent. He swings against their usual methods, and obviously lacks Paul’s abilities and more importantly his foresight. Once the movie actually comes to the point of bringing the two into single combat to decide control of the empire, the outcome is no longer in doubt.

For the most part, their attention only serves to build him up as a rival to Paul. They treat him as a kind of insurance plan; should the Harkonnens retain control of Arrakis, Feyd-Rautha will inherit enormous power and influence, and they want a piece of that. Fair enough. But this doesn’t suffice to make him a credible threat, and scenes emphasizing his effectiveness at violence and love for brutality only lessen him as a plausible opponent.

Paul’s struggle, in classical fashion, is an internal one. Should he stay “true to himself”, as Chani wants, he’ll remain a minor revolutionary figure prosecuting a losing war. If he chooses power, his opponents are nothing; he wins Arrakis and captures the emperor and Baron Harkonnen in a few minutes of screen time.

“Why does he take such risks?” one of his followers asks, after he chooses to fight Feyd-Rautha himself rather than choose a second. This question fails to add any tension. The filmmakers must have felt we needed an exclamation point here, but the duel only serves to undercut the profound shock of Paul’s rapid ascent.

For his own part, Feyd-Rautha never becomes an interesting character. Sheer aggression does not a personality make. The film (and certainly any sequel) needs enemies for Paul who can plausibly contest him , but it is clear that raw firepower will not suffice. By the time we’re cognizant of Feyd-Rautha as a recurring figure, Paul has already surpassed him. He becomes a dumb malevolence of the same sort as Azog, the forgettable orc mini-boss in the failed adaptation of The Hobbit.

More generally, many viewers will want more dramatic tension in Dune II. Paul is, and feels like, a character who is on the way somewhere — and he ends the film still on the way there. The film entertains, and I found the pacing to my taste, but Paul lacks a real buffer to his ambitions.

This feeling is perhaps not helped by the casual way Chalamet plays the character, at times. While on most occasions Paul feels appropriately energetic and enthusiastic, on occasion he’s overly familiar, oddly flippant, too “making chocolate of course”. One doesn’t feel him to be in conflict, even with himself.

That said, Chalamet convinces as an action star and romantic lead. He has palpable chemistry with Zendaya and every scene between the two oozes charm. The speech he gives to the Fremen toward the end of the film compels, as does his sense for the changes to his character caused by the Water of Life.

Paul, as Chalamet plays him in this second film, is a man without rivals. He rides the largest worm, he commands the biggest army, he challenges the most powerful opponents, he meets with every success. If we as viewers still find this story compelling, I believe it to be the result of the film’s persistent efforts to question and undercut these achievements. Dune II allows Paul to do whatever he wants, but reveals his accomplishments to be ultimately the work of others.

I have transitioned ... to a new website!

2025-09-09T04:34:51+00:00

I publicly came out as transgender last year, and more recently, changed my name. I’m now Liliane Fontenot, and I’ve been blogging on blog.liliane.io. I hope that if you followed my work here, you’ll continue following me there.

Eventually, I will likely redirect the whole website to my new one, with static copies of old posts kept around for posterity, but I want to leave this site here for now, and give anyone following me via RSS a clear heads-up to change their URLs.

Here is a direct link to the new feed: https://blog.liliane.io/atom.xml

Thanks for reading!