Statistics-assisted weight loss with pyWeight

11 April 2022 · Adam Fontenot

---

Content note: this article contains a discussion of weight loss and healthy eating, and includes my before and after pictures at the end. You can view a version of this document without these pictures here.

Quick summary

I lost 80 pounds following a diet focused on calorie restriction and satiety by writing software to track my weight and suggest adjustments. The program is PyWeight and it is available as a free download for Windows or Linux.¹

The premise of the diet is that it’s possible to achieve a precise rate of weight loss when restricting your intake without counting the calories in all the food you eat or knowing your daily energy expenditure. A reasonable estimate of how much you’re eating is good enough, when assisted by technology.

Because a calorie deficit and the resulting rate of weight loss are mathematically related, software that measures your average rate of weight loss can accurately determine whether you’re eating too many or too few calories, even if you don’t count them exactly. You can use the information the software provides to hone in on your desired rate of weight loss by making small adjustments to your diet over time.

If this sounds interesting to you, read on.

People who want to follow this diet should talk to a doctor beforehand.

Why everyone is gaining weight and why it’s so hard to lose it

Some people believe that their bodies are genetically programmed to reach and maintain weight at a particular “set point”.

This clearly can’t be the whole story. In the last 40 years, the average weight of the adult population in most western countries has increased significantly. If these weights reflect our natural set points, then we are forced to conclude that just about everyone else who ever lived somehow never reached their body’s natural weight. We’re the lucky few — except that apparently we’re unlucky, since this supposed biological imperative just so happens to be bad for us!

Yet neither can the set point view be completely wrong. It accords too well with several common experiences.

When someone in a country without a fast-food culture moves to one with an urban environment, they frequently gain a moderate amount of weight and then stabilize at a new point from which they find it difficult to gain or lose.

Similarly, people who do manage to lose weight often hit plateaus — low points at which they find further weight loss to be much more difficult. Some people regain lost weight until they’re about where they started. What’s really happening?

I suggest that the truth lies somewhere in between. You do have a distinct set of variables that dictate the weight you will approach in the long term. These include your genetic makeup (with its effects on your metabolism, skeletal structure, and so on), age, and hormone levels. But these variables also include diet and exercise, two variables that are partially under your control.

In the rest of this article, I will refer to the weight you are approaching given your current daily intake (and other variables) as your “equilibrium weight”, to distinguish this from the simplistic “set point” view.

Holding everything else constant, if you are currently maintaining your body weight and you permanently reduce your intake by 100 calories per day, you will eventually stabilize at a new equilibrium point about 10 pounds lighter than your current weight.² A cut of this size is equivalent to drinking one fewer can of Coca-Cola each day at work (five days a week).

Why losing weight is hard, and maintaining weight loss is even harder.

There’s one additional but very important wrinkle: one of the variables that dictates your equilibrium weight is your personal history of weight changes, because of its effects on your metabolism and body composition.

Consider three genetically identical people of my height and age. They all weigh 235 pounds.

• Person 1: has weighed 235 pounds for many years
• Person 2: recently lost weight from 350 pounds
• Person 3: recently gained weight from 150 pounds

According to one model of diet performance³, they will need to eat as follows to maintain their present weight:

• Person 1: 3050 calories
• Person 2: 3000 calories
• Person 3: 3400 calories

In other words, if all three want to lose weight, the person with the thin “history” has an advantage. Because their body has not yet adjusted to carrying this extra weight, they will burn an extra 400 calories a day for free.

A 400 calorie per day difference is a whopping huge one. It’s about the equivalent of eating a pint of your favorite Ben and Jerry’s ice cream every Monday, Wednesday, and Friday. On top of that, the previously 350 pound individual was able to eat about 3800 calories a day at maintenance. So just to maintain their current level of weight loss, they have to get used to eating 800 fewer calories than they are used to.

This may seem unfair. These are people who are genetically identical, the same age, the same height, and the same weight. But one of them will maintain at 400 calories a day higher than the other. It is a little unfair, but remember that these are ultimately temporary metabolic effects. Eventually Person 3, who gained 85 pounds, will become Person 1, who has carried that weight for a long time.⁴ The body doesn’t provide you with a metabolic advantage forever.

This leads to two important principles for maintaining weight loss:

• Until your body becomes similar to that of a typical person who maintains at your new weight, you will face transitory metabolic effects that resist your weight loss. This means that until your body adapts, you will have to eat less than others of the same weight can eat.

• When you lose weight, you become a smaller person. Even once your metabolism stabilizes, you will need fewer calories to maintain your weight than you did at your original body weight. Your hunger systems may have accepted dieting, but when the time comes to maintain they may be slow to adapt to your new maintenance intake, and you may find them in revolt.

Most people ignore one or both of these principles. It’s tempting to treat diets as one-time interventions similar to a medical procedure. This simply will not work, as the foregoing discussion indicates. You can’t eat like an overweight person who has temporarily starved themselves to fit into a smaller pair of jeans. What you must do if you want to keep weight off is to permanently eat like the thin person you now are. You must lower your equilibrium weight through consistent, long term improvements in diet and physical activity.

Most people recognize on some level that when they regain weight after dieting, it’s because they “ate too much again”. What many do not realize, however, is that “too much” means something different after dieting than it did before.

The worst part about all this is that you’ll begin to feel the effects of the two principles while you’re still trying to lose weight.

Most people start a diet expecting to lose weight at a relatively quick rate — and they do their first week or two. Unfortunately, most of this weight loss is fake. Running a calorie deficit means that your body burns through its store of glycogen, which is what causes it to tap your energy reserves (fat). Along with the loss of this weight, you will see a one-time drop that’s just water weight; your body has less food temporarily stored in your stomach and intestines as it’s being digested. Your body will also adapt to reduced intake relatively quickly, and your metabolic rate will drop slightly.

Metabolism is not magic. Your body obeys the law of conservation of energy when your metabolism “slows”, and it doesn’t have a more efficient “starvation mode” that uses fewer calories to perform the same tasks. (If it did, natural selection would surely have dictated that we would always be in that mode, and thereby need to eat less.) Rather, a “slowed” metabolism means that you will either be slightly colder (unlikely to be by enough to make a serious difference) or that you will have less energy available, and therefore perform fewer tasks. Maybe you will feel like exercising less, or maybe you will perform tasks in a lazier fashion, or maybe you’ll simply sleep more. Putting all this together, it means that you can expect to lose some fraction of your caloric deficit to temporary metabolic changes.

If you don’t get discouraged and continue dieting, your weight loss will continue apace, but unfortunately you start to run into the problem discussed above: your reduced size is resulting in a smaller energy expenditure. Research suggests that for most reasonably large deficits, it would take years for this effect to neutralize further weight loss (at which point you would have achieved a new equilibrium), but in the mean time your losses each week will get smaller and smaller. No wonder most people get discouraged! They end up cheating or adding calories to their diets little by little, and eventually they stall out. This is why many people hit plateaus in their diets. A slowing rate of weight loss leads to discouragement, and discouragement leads to a failure to adhere strictly to the diet, and this leads to you hitting a new equilibrium point much earlier than expected.

Because of your body’s remarkable capacity to adapt to changes in food availability, you have only two choices if you want to overcome this problem. You can start out with a lower calorie budget, precisely calibrated to match the metabolic changes you anticipate. Or you can adjust your calorie intake as you go along, to maintain a constant deficit. I strongly suspect that the latter approach is easier to follow and less discouraging, so it is the approach PyWeight is based on.

Where is the obesity epidemic coming from?

This talk of equilibria makes an obesity epidemic rather surprising. Even if people are eating more than before, why aren’t they simply reaching new slightly higher equilibrium points, like our theoretical immigrant? After all, if the way one developed a body that required 4000 calories just to maintain itself was to start eating 4000 calories a day, no one would ever do that. Most people with normal body weights couldn’t even stomach 4000 calories for many consecutive days.

But the evidence for epidemic obesity is clear. Here in the U.S., the state of Mississippi had the highest rate of obesity in 1995, at 19.4%. In 2010, a mere 15 years later, Colorado had the lowest rate of obesity in the country, at 19.8%, and 12 states were over 30%.⁵ Today, more than 40% of American adults have obesity.⁶ This is happening quickly enough that it doesn’t work to posit genetic or behavioral changes in a new generation of adults: the same people who were thin 10-20 years ago have obesity now. So what’s happening?

Of course, when we’re talking about the obesity problem, we’re talking about averages. Some people do maintain an equilibrium point that’s just overweight. Other people don’t become overweight at all. Yet in this country, we seem to have reached the point where the majority of people put on weight year over year, endlessly.

You can square an equilibrium view of body weight with this fact using the suggestion that we’ve reached some kind of tipping point. For some reason, people are driven to eat slightly more than they need to maintain, and for many of them the rate at which they increase their intake has begun to outpace the body’s compensatory mechanisms.

The body is calibrated to give you hunger signals that will result in you eating approximately what you need to eat to maintain your current weight, on the sort of diet most people had in the past. Someone who could maintain at 2000 calories a day and significantly overate⁷ would gain weight, but after doing that their body would still only be giving them hunger signals appropriate to maintain their new body weight — or more precisely slightly less until they adapted. (Effectively, the two principles mentioned above are functioning in reverse here.) So this person would not automatically continue to gain weight. The body’s negative feedback of constantly feeling too full would tend to keep their weight in a healthy range.

Our hunger signals are no longer accurate. The body still says “I need 2000 calories”, but it doesn’t feel full until you’ve eaten 2100, thanks to the low satiety calorie-dense diets of today. When you gain enough weight on such a diet, your body will start to say “I need 2100 calories”, but now it takes 2200 to fill you. And because eating food takes time and costs money, your increased appetite will be drawn towards quicker and cheaper options, further exacerbating the problem. Before, the body gave you you a signal that you should eat less food the day after a big meal. Now, even a big meal isn’t enough to make you feel full.

This suggests we should change our view of personal responsibility. People in the last 30 years are facing a problem that virtually no one in the past faced: their stomachs are lying to them about how much they should eat.

Recall the equilibrium equation: your equilibrium weight is a function of your genetics, body composition, age, hormones, diet, and exercise. If you consider this equation on a societal level, you can replace diet and exercise with just one variable: environment. In the absence of a strong effort to control these two variables, the health environment you are in will dictate them to you, and therefore control your weight.

Faced with the environment that most people in Western countries face, the average person is going to gain weight over time. So on a broad social level, that’s what we see. Blaming individual people for the consequences is not just counterproductive, it’s an inaccurate accounting of what has happened.

Consider the following scientific model of average weight gain⁸:

What this model indicates is that the weight gain of the average American over the last decades can be explained by a relatively tiny gap between energy intake and energy expenditure. In fact, the average gap for the period examined in this study was only about 7 calories a day. The problem doesn’t amount to 7 calories here or there, the epidemic stems from the fact that this gap is so reliably sustained. High calorie junk food is causing many of us to overeat, day after day, with no feedback telling us to cut back for a while. This has gone on for so long that in 2005 the average person needed to cut out more than 200 calories a day to achieve the equilibrium point of the average person in the 1970s. This number, of course, has only continued to rise.

Watch this excellent five minute clip detailing some of the causes of calorie overproduction in the United States:

You might be tempted to decide the takeaway of this analysis is political: corporations are profiting off of human suffering on a massive scale, so some social action is needed to fix the problem. Insert regulation, social democracy, or whatever your preferred solution happens to be. That’s all well and good, but it’s of little use to someone suffering the effects here and now. This country will happily let you die in the street. In all probability, things are not going to get better on a timescale that matters to you. The only option is to take matters into your own hands and exploit the two variables under your control to the maximum extent you can. For some people, that will be enough.

The good news

The one bright spot is that for most people the food environment is not yet bad enough to make it impossible for them to control their weights. If you are not very constrained in your diet and exercise choices, then you have a fairly good chance of overcoming your environment.

As for the other variables in the equilibrium equation that are outside of your control (e.g. genetics and metabolic effects), the good news is that these almost always have a relatively minor impact on your weight. Most people have a basal metabolic rate (the number of calories you burn each day just by being alive) within 10 percent of the rate estimated by the Mifflin-St Jeor equation. “Slow” metabolism is not usually the cause of weight gain.⁹

This means that many people can change their equilibrium point by managing their calorie intake and becoming more physically active.

Addendum: it’s okay to try to lose weight, I think

Some authors, including philosopher Kate Manne, claim that encouraging dieting and even dieting yourself are immoral. I agree that negative self-image and exploitation in the diet industry are both things we should condemn, but for this to have consequences for individual dieting decisions something more is needed.

I think that’s why Manne’s article, though it defends an ethical conclusion, ultimately rests on two empirical claims. She writes that “fat bodies can be not only healthy but also athletic” and “the vast majority of diets fail to make people any thinner or any healthier in the long term”. These claims are both true, and suggestive, but don’t on their own deliver her conclusion.

It is true that some fat bodies are healthy. Similarly, many people who don’t get the recommended 30 minutes of exercise every day are also healthy. Despite this, since exercise and a reduction in overweight tend to result in health improvements, it can make sense to pursue both.

Likewise, the fact that most diets fail cannot deliver the conclusion either. If this claim is to show that it would be a bad idea for me to start a diet, it needs to be true that I can have no expectation of success, so that trying to lose weight is similar to playing the lottery. I think this is false. There are reasons that most diets fail, but dieting is not tantamount to fighting genetic or environmental determinism, as I argued in the previous section. I explain further why I think it is reasonable to expect success in the next section.

I suspect that any versions of Manne’s empirical claims strong enough for her conclusion will be false.

The claim that some fat bodies are healthy suggests a stronger version that would suffice: most fat bodies are healthier than the same bodies would be if they were thin. Dieting would then, in all likelihood, make you less healthy. In this form, the route to Manne’s conclusion that encouraging dieting or dieting yourself “is a morally bad practice” is obvious, but the premise seems clearly false. There certainly is a correlation between body weight and health. As I will discuss later, some (contentious) studies suggest that being slightly overweight does not increase mortality, but no medical authorities will endorse the notion that obesity is healthy.

Likewise, if diets could be shown to be inherently dangerous, unhealthy, or doomed to fail, it would be easy to reach the conclusion. Manne hints at this: she says that in one month early in her diet, she didn’t eat for 17 out of 30 days. This is indisputably a bad approach to dieting. If most people trying to lose weight go for diet fads and ineffective products, or take dangerous or disordered approaches like extreme fasting, that would help explain the failure rate.

However, it seems false that these problems are inherent to dieting. I ate food every day on my diet. Manne herself seems to have ended up with a sensible diet: she lost about 50 pounds in a year, close to the common recommendation of a pound per week. The diet industry is certainly blameworthy for distracting people from methods likely to be successful and promoting fads, but this alone can’t show that dieting is wrong.

How to succeed at dieting

Most diets fail. In a major longitudinal UK study, less than 20% of people with obesity managed to decrease their BMI by at least one category without a subsequent increase, over a 9 year period. 78% of people who lost at least 5% of their body weight gained enough back that after 5 years they were back within 5% of their original weight, or even higher.¹⁰ This indicates that while plenty of diets do succeed (the popular claim that 95% of diets fail is incorrect), successful dieters are by no means the majority.

I don’t believe a 20% success rate should be discounted. It suggests that diet success is possible and that we should be looking at what the successful people do. If we were to discover that a particular intervention to help people quit smoking had a 80% recidivism rate, we would still encourage people to take the program (because 20% is much better than zero).

Part of the issue with assessing the success of a typical diet is that if you lower the bar of what counts as successful weight loss, then the success rate goes up. If you demand studies involving weight loss of a quantity closer to what I was hoping to lose, examples become even rarer. Most studies are testing specific short-term interventions to see what works and what doesn’t, not looking at how to achieve a long-lasting high level of weight loss. It’s hard to get reliable data about the prevalence of successful diets resulting in long term weight loss of 25% of body weight or more. A large portion of attempts likely fail.

The reasons why most diets fail go beyond simple misunderstandings of what it takes to succeed, of course. Food is tempting. Not eating food when you are hungry sucks. The restrictions that a diet imposes can be difficult to implement. The nutritional deficits imposed by Western eating can become even more severe on a calorie restriction, especially for those who can’t afford to make their own food. For many people, a personal trainer and gym membership are unaffordable, and for some even a doctor visit is out of reach. (It was for me when I began my diet! Thanks, America.)

Given the high rate of failure, what I went looking for was a diet that could guarantee success to those who stick to it and and don’t cheat. My hope was that following such a diet would not only bring me the weight loss I was looking for, but that the certainty of success might provide additional motivation to stick with the diet.

As a result of studying physics in college, I was naturally inclined to follow a “calories in, calories out” diet. This comes with something like a success guarantee: if you consume fewer calories than your body uses each day, you will lose weight. Other diets may be successful as well, but only if they end up resulting in calorie reduction anyway. Diets may have some individualized metabolic effects but these appear to be minor in comparison.¹¹¹² Perhaps some other diet is easier to maintain than calorie restriction, but no other diet could provide the guarantee of success.¹³

Counting calories has a reputation for being difficult, which is part of the reason diets that dress up the calorie counting by using “points” systems are so popular. It’s not only easy to underestimate the number of calories that you’re eating, it’s also difficult to determine with accuracy how many calories you’re burning. Moreover, if you weigh yourself once a week or so, your measurements might get swamped by variations in water weight or having an unusually large lunch, so it’s hard to know if you’re making progress. If you weigh yourself once a day, the value on the scale is likely to fluctuate so much that you constantly fly into a panic over an imaginary weight gain.

Furthermore, you might already be disheartened by reading the previous discussion of why it’s hard to lose weight. As you lose weight, the number of calories you need just to stay at maintenance is going to be decreasing. How is it possible to reliably stay ahead of the curve? It turns out that there’s a simple trick that can be summed up in a single paragraph, and this trick obviates most of the complexity of weight loss. This trick is:

Record your weight every day. Every two weeks, calculate your approximate weight loss using your record. If you are losing weight too quickly, eat more the next two weeks. If you are losing weight too slowly, eat less the next two weeks.

That’s it. Weight loss, summarized. By only worrying about whether your rate of weight loss matches your predetermined goal, and not calories or points or what have you, you can hone in on a reliable rate of weight loss over time with relatively little effort. Accurately counting calories or determining your daily expenditure is not required.

You can do this yourself with just pencil and paper if you like. But technology will help you in two ways: you can more accurately calculate your rate of weight loss using techniques like linear regression, and you can estimate how many calories you missed your target by (instead of pounds), to make it easier to adjust your intake in the future.

This idea was inspired by the book The Hacker’s Diet. The book’s version of this scheme goes something like this: if we assume that each pound of fat lost contains about 3500 calories, then the math becomes quite simple. If you need to lose x pounds, and you maintain a deficit of about y calories a day, it will take you about 3500 * x / y days to do that. At a deficit of 500 calories a day, you will lose about a pound a week. If after two weeks you’ve only lost 1 pound, you know you’re missing your target by 250 calories per day.

Unfortunately, this relies on the myth that a pound of fat contains 3500 calories. It turns out the correct number is closer to 4280.¹⁴ The slightly better version of the theory, which says that the average caloric density of the weight you lose is 3500 calories, is an inaccurate approximation. You do lose some fat free mass while dieting, and this contains around 824 calories per pound. So 3500 calories per pound is only accurate if about 3/4 of the pounds you lose are from fat. This is only a reasonable estimate for very overweight people at the beginning of a diet. Later on, as you approach an optimal body weight, you will lose a larger proportion of lean mass. If you were to continue losing weight indefinitely, you would of course eventually have nothing to lose but lean mass. This would be extremely bad for your health.

Because of this complication, PyWeight uses an equation from a real diet model¹⁵ to estimate how many calories were contained in the weight you lost in a given interval, and therefore how many calories you are away from hitting your weight loss rate target. This means it’s possible to target a linear rate of weight loss despite all the non-linear effects. (For what it’s worth, when you’re targeting a constant rate of weight loss rather than a constant, pre-determined calorie budget, the problems caused by inaccurate estimates are greatly lessened. In fact, you’d probably be fine with the old 3500 calories per pound estimate. The upshot of PyWeight’s advice is usually just “try to eat a little less / more next week”, just as it would be if you were using pencil and paper.)

With the Hacker’s Diet, the calculations are all done in a series of complicated spreadsheets which are provided with the book. PyWeight allows entering your measurements directly in a built-in log, and all calculations are done for you automatically. It also provides intake adjustment advice at a user-selectable frequency.

The linear regression method used by the Hacker’s Diet can also be improved upon. Doing a naive linear regression by breaking the data into intervals lasting 2-3 weeks means that only the data from those weeks figures into the calculation of your current rate of weight loss. But this is throwing away data; you can determine a much more accurate starting weight for each interval if you also make use of the data from the previous interval. PyWeight takes exactly this approach with a technique called spline interpolation. A possible alternative would be to use a moving average, but I believe this would give worse results because of the high degree of variability in weight measurements and the fact that the diet is built around attempting to lose weight at a constant rate.

As a side effect of this improvement, PyWeight is able to generate very nice looking graphs of your weight loss. Here is mine from the last ~7 months. (I lost the first 20 pounds using the methods described in this article before I wrote PyWeight.)

The idea behind using a program to track your weight is that despite the fact that our hunger systems are broken because of modern calorie dense diets, we can use technology to replace this body function, letting simple math regulate how much we eat. This method actually works even better than simple calorie counting, because it adapts to your changing energy needs as you lose weight. If you want to keep up a one pound per week deficit, you’ll need to reduce your intake over time, and this approach will help you do so automatically.

Even though you’re measuring your weight every day, you don’t have to worry about what the scale says in the way you do with ordinary calorie counting. An unusually large meal might send your apparent weight up 1-3 pounds for several days, but this is mostly water weight that will soon go away. Looking at a statistical measure of your weight loss over a period of weeks smooths out these misleading bumps in the graph.

Looking at my graph, you can see several of these blips, some of them quite large. My daily scale measurement increased 6.8 pounds over the course of four days in late December, but this did not represent any long term weight gain.

I’m not going to tell anyone that this approach makes dieting a sure thing. A great deal of willpower is still involved. But knowing exactly what you’re doing and trying to keep up a consistent rate of weight loss makes it much easier. Steven Novella helpfully writes:

I would conclude that weight control is difficult, but not impossible. In addition, I believe that the popular weight loss industry is making the problem worse by distracting the public from those strategies for which there is at least some evidence of efficacy and focusing on minute, short term, and probably insignificant effects from manipulating macronutrient proportions.

My goal with this diet is to focus on what we know works (consistently eating fewer calories than you burn) and find a way to make sure I am doing that with small course corrections over time. I began this diet with a high degree of confidence that it would work if I didn’t get burned out and give up, and that confidence was rewarded. As you can see from the graph above, I was able to maintain an extremely consistent rate of weight loss over time; I never deviated from my monthly target by more than a few percent (except around the holidays and a strenuous four day hiking trip in mid-February during which I snacked almost continuously).

It’s not just losing the weight that gives me confidence this method works. It’s the fact that I lost the weight at almost exactly the rate I planned in advance! I think many individuals can see success in dieting if they can follow a similar approach.¹⁶

No method for individual correction can solve the underlying problem. So long as the United States fails to get a handle on its poor nutrition environment, obesity will continue to be a serious issue. Proposing diets as the solution is like thinking that encouraging people to recycle can prevent climate change. However, until the social causes of obesity are resolved, individuals are on their own. Diets will not reverse the long term obesity trend, but they can help dedicated individuals who are fortunate enough in their circumstances.

Setting goals

I started out just above 240 pounds, which at my height was a hair into the obese range (30+) by the BMI metric. BMI is not necessarily a good indicator of individual health, but the truth is I felt bad. I did not feel like exercising, or even going for a walk. Hiking long distances, one of the activities I’ve enjoyed the most over the years, was certainly out of the question. While it’s not impossible to be in good shape at the size I was, it’s certainly much more difficult.

How bad is it, in general, to be overweight? There has been some recent controversy on this point in the medical world. The long-time consensus has been that while individuals with BMIs between 25 and 30 are not necessarily unhealthy, overweight is correlated with heart disease and diabetes, as well as increased all-cause mortality.

One 2005 study suggested that being in the overweight category was associated with reduced all-cause mortality relative to the normal (18.5 - 25) BMI category.¹⁷ A follow up meta-analysis¹⁸ by the same primary author confirmed this finding, and also found the difference between mild obesity and normal body weight to be insignificant. However, a much larger third analysis¹⁹ disagreed, finding that after controlling for smoking, prior chronic disease, and length of follow-up, both overweight and obesity were correlated with increased all-cause mortality risk. Moreover, they were able to reproduce the findings of the second paper by removing those quality controls. (The danger of leaving in smokers is that a disproportionate number of smokers will have healthy body weights because nicotine functions as an appetite suppressant. Smoking is much worse for you than a few extra pounds, so these people make “normal” body weights look less healthy on average.)

I’m not here to say that the third study was right and the first and second were wrong. I don’t have the expertise to do that, to begin with. The primary author of the first two papers wrote a response paper which claimed that the approach taken by the third was biased and suggested that the exclusion of smokers might lead to more bias than leaving them in.²⁰ The result has been an increasingly bitter academic feud over claims of “bias” with no obvious conclusion (to me).

However, a very nice fourth paper²¹, a systematic review of the issue, reached more or less the same conclusions as the third. It was significantly larger than either of the others, containing almost 10 million individuals in the never-smoking category alone, and none of the authors overlapped with those of the third study. This study found that an ideal BMI was approximately 23, with negligible differences between BMIs from 22 to 25. Furthermore, the paper reproduced the most important finding of the third: the conclusion was reversed when filtering was removed. A BMI of 27.5 was associated with a lower mortality risk than one of 23, and a BMI of 32.5 lower than one of 20!

Because this study subdivided “normal” BMI into many categories, the conclusions are a bit more fine-grained. The results may help explain why in some studies the 18.5-25 BMI range compares unfavorably with the 25-30 range. The dangers of being underweight are quite extreme: among never-smokers, the study found that a BMI of 17.5 was associated with a 35% increased mortality risk. Someone with a BMI of less than 20 contributes to the average outcome for the normal BMI range (if their BMI is 18.5 or higher), but they’re exposed to more of the risks of underweight compared to someone with a BMI of 23, which the study found to be optimal.

I believe it’s very important for casual interpreters of medical research to remember to look at papers in context. All this bickering is essentially over a 5-10% difference. If you believe the fourth paper, a BMI of 27.5 is associated with a 7% increased all-cause mortality risk versus a BMI of 23. If you believe the second paper, having a normal BMI might be associated with 6% more all-cause mortality risk versus an overweight BMI. Personally, I’m inclined to believe the former, because the studies are larger, it’s more in line with the medical consensus view, and because the exclusion of smokers in the analysis seems justified. But I see no reason for me to argue with someone who believes the latter. To my mind, the sane middle ground is found by keeping your BMI well out of the obese range, and at a level where you feel able to keep yourself in good shape. My guess is that the difference between the two is swamped by just getting plenty of exercise.

For me, the truth is that I wasn’t able to feel healthy or stay fit at the weight I was, and the BMI number merely served as a painful (though possibly useful) reminder of that.

For the sake of having a goal weight, I picked the arbitrary target of 160 pounds. This happens to correspond nicely to taking 10 points of my BMI, and struck me as unlikely to be unsafely low. It corresponds to a BMI of slightly above 20, and my plan is to put on a few pounds of muscle over the next year. The last time in my life when I was thin was around the age of 10 or 11 (when I had temporarily become a vegetarian), and I wanted to experience that again.

It remained only to choose a rate at which I was going to try losing the weight. The UK’s National Health Service says that a weight loss rate of 0.5 to 1 kg a week is safe. 1 kg a week is equivalent to 9.45 pounds lost every 30 days. I rounded this up to a target of 10 pounds per 30 days. I’m considerably taller than the average person, and so there is a larger gap between my daily energy expenditure and my basal metabolic rate than for the typical dieter. I think this was a safe target for me.

I emphasize that if you’re going to pursue fast weight loss, you should talk to a doctor about that. You may very well have more success and invite fewer risks by targeting a pound per week (for some people, even less). In my case, without access to professional advice, I made the best judgment I could about what would be safe and effective for me.

How to get enough nutrition on a diet that doesn’t suck

It can be helpful to get a rough estimation of how many calories you should budget for yourself. In my case, something like 1800 calories a day at the start of the diet was appropriate. One nice thing about taking a statistical approach to weight loss is that even if you screw up this estimate quite badly, you’re unlikely to put yourself in any danger as long as your target rate is reasonable. If you lose too much weight in the first weeks of the diet, you will receive a sharp corrective directing you to eat several hundred more calories every day. PyWeight is thus self-correcting.²²

Cutting a thousand calories out of your usual intake can make it difficult to get the nutrients your body expects to have available. It’s important to have a plan for getting the necessary vitamins, minerals, and essential amino acids.

This is an area where The Hacker’s Diet is not very helpful. It says

Since we’re efficient food processing machines, it’s possible to reduce all the complexity of food to a single number that gives the total energy the body can extract from it—the calorie.

The extreme form of this is the infamous Twinkie diet. You will lose weight on such a diet; it might also kill you. (It’s interesting that the actual Twinkie diet guy didn’t just eat Twinkies. A full third of his calories came from fruits and vegetables along with a protein shake. Eating no protein at all for months on end would probably not have gone well.)

To be fair, the book also suggests eating a balanced diet. As a vegan, I think there’s some cause for me to be concerned, especially on a diet as restricted as mine. I took a vegan multivitamin (DEVA tiny) every day of the diet, to ensure I wasn’t vitamin deficient. I really like this multivitamin because (a) it provides ~100% RDA of vitamins I was likely to be deficient on, instead of 10000% like some multis with potential side effects, and (b) it was small and easy to swallow. I also have a B-12 supplement that I take about once a week (as do most strict vegans).

So the remaining concern was protein. Getting enough protein isn’t generally a problem for vegans²³ and even on a restricted diet it wasn’t one for me either, at least in terms of quantity. The issue is that there are minimum amounts of specific amino acids that we have to consume for building cells and other important stuff our bodies do. I wanted to make sure I was hitting the WHO minimum recommendations fairly consistently, even though I didn’t think a mild deficiency for a few months was likely to be harmful. (After all, the Twinkie guy didn’t fall apart.)

In order to hit these targets, I ended up on a diet that was pretty protein loaded. I hit the recommended amino acid intakes almost every day, even though I was on a diet with considerably fewer calories. At the same time, I also emphasized eating a lot of fiber and low-density foods. The point of this is that both fiber and foods with a high bulk to calorie ratio will make you feel full. Note that neither of these strategies will directly cause you to lose more weight than otherwise. They’re simply ways to make a low calorie diet easier to maintain by reducing hunger.²⁴

Many vegan foods are complete proteins (that is, they include all of the essential amino acids). Most aren’t perfectly balanced between the amino acids, so you might have to eat more than enough of one to get enough of all the others. (This is one of the reasons why a balanced diet is often suggested, since eating many different protein sources will help balance the ratios out.) Note that this problem isn’t unique to plant-based diets — beef isn’t perfectly balanced either, nor are most other animal-foods. It’s only something to worry about for someone on an intense calorie restriction.

In trying to come up with meal plans, I found WolframAlpha indispensable. You can enter an amount of food, and it will tell you how much of each amino acid it has (assuming it’s in their database).

Lentils and soy turned out to be useful for hitting my high protein and fiber requirements on a low calorie diet. It’s actually possible to hit the WHO amino acid recommendations and much more than 100% of your fiber using mostly these sources on a mere 1200 calories a day.

You don’t have to eat the same thing constantly on this diet. (In fact the whole point is that you don’t need to be precise about what you eat or count calories exactly.) However, if you’re cutting so much that you need to be concerned about nutritional deficiencies, planning your meals may help. It’s probably still best to mix it up some of the time, just to ward off any additional danger of malnutrition.

On many days, I ate whatever kind of food I wanted (or even went out to eat) and just went with a rough estimate of my calories. The point is that as long as you are hitting your nutritional requirements, and can get even a rough estimate of the number of calories you eat, you’ll be fine. The best part is that you don’t even have to write anything down if you don’t want. It’s a calorie “counting” diet, but all the counting is quietly being done automatically for you by enhancing the quality of your estimations over time until they’re about as good as a real count. (Of course, should you struggle making the necessary adjustments to hit your targets, you might find a food diary indispensable, at least for a short time.)

If it is at all feasible for you, I do suggest making as many of your own meals as possible, and making them as highly nutritious as you can. Fast food is far worse than you or I probably realize, and it tends to be extremely calorie dense without satisfying.

Keys to success

I think adopting the following attitudes and practices while dieting helped me achieve success:

1. Commit to a lifestyle change, not a brief period of restriction.

   As discussed above, this is actually mandatory if you want to keep the weight off afterwards. This means that you need to recognize that you’ll be giving something up (you will have to permanently eat less food), and decide that the cost is worth it.

2. Commit to weighing yourself every day.

   Forming any habit can be irritating, but when your body has been sending you dishonest signals for years, this is really the only choice. Many people who are successful at maintaining weight loss continue to weigh themselves regularly for years afterwards.²⁵ You’ll need to do so at least until your hunger system is retrained.

3. Don’t declare any food off limits (other than pre-existing dietary restrictions).

   There’s no good reason to set yourself up for a cycle of failure and self-blame. One of the most common reasons for dieting failure is that people prohibit certain foods entirely, crave them incessantly, and then binge when the pressure proves too much or they have a “cheat” day. The resulting feelings of shame may drive them away from the diet entirely, especially if the number on the scale goes up the next day. (Usually, this is a temporary result of eating a lot of food. The whole point of using a statistical measure of your weight loss rate is that you don’t have to worry about the daily fluctuations!) Throw out the concept of cheat days; every day is a diet day, but every day is also a day that you may have a doughnut if you absolutely must have one. As long as the line on the weight chart keeps going down at the rate it’s supposed to, you have nothing to worry about.

   Try to develop a more healthy decision making process about what you eat. Ask yourself: am I willing to trade being hungry later for eating this sugary thing now? Speaking for myself, I ate almost no added sugar during the diet. It was never prohibited, though. I put a teaspoon of real sweetener in my tea. I had a little square of a chocolate bar once every week or two. I had a slice of pie or a few cookies over Thanksgiving and Christmas. I had a beer every now and then. Nothing off limits.

4. Never give up.

   There will be days when you want to. If you have a tendency towards anxiety or depression, or you use food to cope with problems, you are going to have to face that head on. You’ll have to handle feelings of hunger and reduced energy. It’s even likely that there will be days where you screw it up. The difference between the people who succeed and those who fail is that people who succeed don’t tell themselves that they failed. Failure isn’t what happens when you have one bad day, it’s what happens when you give up.

Conclusions

I lost more than a third of my original body weight. The guarantee that you will lose the weight if you follow the simple instructions output by the program and dutifully weigh yourself every day was enough to push me through the hunger pangs and inclination to give up. Here are the usual before and after pictures.

Is this a good way to lose weight? For me, the dominant consideration in answering this question is “is this a successful way to lose weight?”. That’s because most diets are unsuccessful, and so successfully losing a significant amount of weight is already a high bar to clear.

I think it’s reasonable to say that this can be a safe way to lose weight. If losing the standard recommendation of 1-2 pounds a week is a safe rate of weight loss for you, PyWeight will not tell you to do anything stupid. It is merely a tool to help keep you on track. You can choose a slower rate of weight loss and it will come off just as reliably, but will require more patience.

The other question, of course, is can I keep it off? That remains to be seen, but I’m planning to continue weight tracking for the rest of my life while eating at maintenance (net-zero calories in-out every month). This is something PyWeight can also help you do. If you set your desired deficit to zero, any time you begin to gain weight PyWeight will quickly warn you to reduce your intake.

The fact of the matter is that even if I knew for certain that I’ll gain it all back within five years, I’d still do it all again. I really do feel that much better, just as a result of the weight loss. You can’t assess the quality of someone’s life by what they happen to weigh when they die; even if I eventually regain weight I will have bought myself years of enjoyment in the mean time that can’t be taken away. That said, I understand that most people are looking for long term weight loss. I hereby commit to publish an update to this article with my current weight one year from the date this was posted, and then again a year after that.

Thanks to anyone who took the time to read all the way through this. Feel free to ask me any questions that you have.

Addendum (2023):

It’s now been one year since I published this article. Here’s my scale as of this morning:

I think it’s fair to call this a complete success!