17 Jan 2017

Freakonomics -- the review

I’ve heard of this book for years, but never bothered to read it because I was already an economist and didn’t think I needed to read a popular summary of Steven Levitt’s work.[1] My impression was that the book summarized his work in a popular (non-academic) style that helped people understand what economists do. I though that the book was useful in this respect in helping people understand what I do. Indeed, the most common reaction I get from people when telling them that I am an economist is that they have read Freakonomics, which implies that they have at least seen some work similar to what I do at aguanomics.[2]

It turns out that I was mistaken in my initial beliefs.

I just read this 2004 book 2006 revision, and it's made me think a bit more about how we (economists) communicate with the general public, and I think that some ways are better than others. First, there are textbooks, which describe the tools that economists use to put their theories into practice. Many many people have told me “I didn’t learn anything in economics. All I remember was a lot of math and curves.” This depressing outcome results from lecturers who merely reproduce problems and equations on the blackboard, without helping students understand either why those theories are used or how they came to be so popular with economists.[3] Second, there are books that explain how economists think or how their thinking has evolved as they have tried to understand and summarize the people’s behaviour. These books, in my opinion, are the most interesting — and challenging — because they push people to revisit their assumptions and perspectives.[4] These are the books that I would recommend to people looking to “think like an economist” or, to be blunt, to think more accurately about how they and those around them actually behave. Third, there are books like mine [pdf] that try to explain how to improve failing policies using basic economic insights and incentives. Finally, there are books like Freakonomics that reproduce academic papers in a popular form. These books — like Economic Gangsters — give the public a limited vision of research without explaining the struggles of getting the right data or explaining the limitations of theories that are used (or not) in the final paper.

Freakonomics is therefore NOT the book that I would recommend to anyone interested in (a) learning economic theory, (b) learning about how economists think, or (c) understanding the world or thinking of ways to improve it.

This book with a memorable (but useless) name provides readers with just-so stories that are good for  cocktail conversations but not for understanding economics.[5]

For example,
  • Legalized abortion explains the drop in crime in the US. Not only do Steve and Steve back off from the main claims of the original paper (they add other factors), but this theory has been falsified by others (see this and this). What struck me is their ongoing attempts to hold onto at least some elements in the original claim in later blog posts in what I’d call a “my-ladydoth-protest-too-much” manner.
  • Real estate agents serve themselves better than they serve clients when selling their own homes. As a former real estate agent, I had to agree with their basic premise, but I thought their explanation too simplistic.[6] The most obvious problem is that agents have an entirely different understanding of themselves as sellers as well as of the markets. Surely that different information (and the resulting “patience” that gets them a higher price as sellers) matters?
Looking over their other chapters (on cheating sumo wrestlers, drug dealers who live with their moms, the KKK as a multilevel marketing organization, etc.), I agree that the chapters are interesting and thought provoking, but they do not provide “lessons on the hidden side of everything.” Instead, they read like a series of magazine articles whose quirky “insights” might contribute to your next cocktail conversation.

The authors say that they want you to ask more questions and see the world differently, but what tools have they given to you in this book?[7] I didn’t detect any reliable technique (except perhaps to collect a neat dataset and call Steve Levitt), and that’s where I was disappointed. Freakonomics does not really reveal the hidden side of everything. Indeed, it’s more likely to mislead you into thinking you’ve learned something, when you’ve only learned an interesting angle on a complex topic on which you may lack either the experience or methods needed to put it into a useful context.[8]

Take their example of the “underpaid” drug dealer who they say faces a higher risk of death than someone on Death Row in Texas (and thus must be overestimating the gains from their job). Does this statistical analysis mean that those street dealers are irrational? I don’t think so. As all economics students learn, you need to look at their opportunity cost (i.e., the costs and benefits of their potential other choices). In this case, street dealers are (a) NOT condemned to death, (b) not able to find other work with their experience, and (c) not aware of their statistical mortality as much as their potential wealth. Those street runners are “taking their chances” in the same way as Americans are “living the dream,” i.e., in ignorance of reality.[9]

Bottom Line: I give this book THREE STARS. Dubner and Levitt present interesting puzzles worthy of cocktail conversation, but they overstate their contributions and accuracy (“numbers don’t lie” but theory can be incomplete or just wrong). I suggest that anyone interested in understanding how economics works and applying those lessons to “the hidden side of everything” read Economics in One Lesson by Henry Hazlitt. It’s free to download and provides a really useful perspective that anyone can apply to any topic they care about.[10]

  1. I’ve met Steven Levitt. He's a fine person and excellent economist, but this book is too “pop” in its oversimplification of his work and hagiographic treatment of his insights. Yes, he brings interesting statistical tools to“freaky” questions, but he’s not a “rogue economist exploring the hidden side of everything.” He’s just a guy with a dataset and empirical theory who finds some strong correlations. As I explain later on, he does not deliver the last word on pretty much any topic in this book. (It's interesting to see the two authors pooh-poohing people's objections to their claims in this revised edition. I get the impression that their answer is "bestseller, bitch!" more than "hmmm..., maybe we claimed too much.") Also see note 8.
  2. I wrote on the human right to water and oil and water for their Freakonomics blog.
  3. I published an article [pdf] on how students don’t really understand the “downward sloping demand curve” because its form is based on advanced techniques they won’t see for a few more classes (meaning "never" for those who take one class or drop the major).
  4. Of those I have reviewed, I recommend Small is Beautiful, the Calculus of Consent, the Company of Strangers, Predictably Irrational, etc.
  5. "Freaky" anything sounds bad to me, and "freaky" economics -- unlike most economics -- isn't useful to most people. Even worse, there's nothing freaky to the stories in terms of the economics. I wish the authors had spent more time on the basic economics (making the book a useful learning experience) and less time defending empirical research that is interesting and provocative but not really wise.
  6. I corresponded with Levitt’s co-author on my objections to their working paper back in 2005. The main one was that their analysis missed the most important point: it’s better to have an agent than not to have an agent — an obvious insight that saved me about €10,000 when I bought a flat in Amsterdam. Going further, would an agent work harder for you if their commission was a flat rate rather than a percentage of the sales price? They harp about commissions as detrimental to the client’s interest due to the small share of the additional profits an agent gains from working harder on your behalf — e.g., 3% of another $10,000 is only $300, but why would an agent work any harder on a flat-rate commission? In my experience, agents love on referrals from old clients, which may explain why they work hard "despite the weak incentives."
  7. My definition of an expert is “someone who knows what’s missing.” As an example of this, I give a fourth reason why it’s NOT irrational to vote (they give three weak reasons in the blog post included in their revised edition), i.e., the benefits to an individual from study and engagement in a topic.
  8. My years of experience traveling in 100+ countries leads me to respect the diversity of beliefs and institutions that result in a variety of outcomes. Most academics need to exit the Ivory Tower and hit the road more often.
  9. On page 134, they write “The typical parenting expert, like experts in other fields, is prone to sound exceedingly sure of himself. An expert doesn’t so much argue the various sides of an issue as plant his flag firmly on one side. That’s because an expert whose argument reeks of restraint or nuance often doesn’t get much attention.” They need to apply the same critique to themselves. (They cite themselves in later chapters — p139 on abortion and crime — as if their earlier claims were facts.) As another example, take Dubner on page 199, who writes “that paper [on police officer counts and crime] was later disputed… a gradate student found an obvious mathematical mistake in it — but Levitt’s ingenuity was obvious.” I’m not sure I’d say the same about someone whose claims rested on logic with “obvious mathematical mistakes”!
  10. Hazlitt says "the art of economics consists in looking not merely at the immediate but at the longer effects of any act or policy; it consists in tracing the consequences of that policy not merely for one group but for all groups," which he summarizes as "...and then what?" That's a good question, and it's how I can easily predict that cheap water prices will lead to failing infrastructure or water shortages, why climate change is arriving "too fast" (due to a lack of carbon taxes), and so on. I'd prefer people to ask "and then what?" more often and spend less time showing off their knowledge of cheating sumo wrestlers.

For all my reviews, go here.


Anonymous said...

Sounds like Bastiat's 'What is Seen and Is Not Seen' is a better actual introduction to what economists do? #opportunitycosts http://www.econlib.org/library/Bastiat/basEss1.html

Tim in Albion said...

This review mirrors my disappointment in the book, and I'm not even an economist. At first I was intrigued by the theme and early examples, but often found myself thinking "wait a minute..." as they repeatedly failed to consider other explanations or show how their answer provided the best fit to the data. I couldn't finish the book because my brain kept offering up alternative conclusions that seemed equally plausible but weren't considered. In short, too little humility.

David said...

I remember at UC several years back, the two experimental econ overview classes open to undergrads were by John List and Steve Levitt. These days, Levitt seems to be working with more observational data (?), but at the time, they had just published an overview paper together and it was a hot topic.*

Both classee were more or less a quarter long survey of their papers, but List's class really excelled at weaving the tools he had developed into a bigger picture epistemic case for field experimentation in economics. He would invest significant time into the motivation behind resolving tensions between observational findings (IVs, discontinuities, etc) / theoretical predictions, and soc/psych/behavioral lab experiments, and then teach the statistical tools and applications as refinements to those approaches. In the end, I walked away with a new basic framework for empirics.

I never took the Levitt class, but some friends and I compared syllabi and experiences. From what I understand, Levitt's class was much as how you describe Freakonomics- "Wow, look at this weird data! Let's see what conclusions we can draw! Hey, this is strange, what experiment can we set up to explore it further?" Much more spontaneous, much less methodical, and more willing to wade into the fine details rather than build a worldview of the place for field experiments in the econ toolbox.

Yet, I think you're too quick to dismiss Levitt's approach. If there is a central thrust to the book, it would probably be to look carefully at data before coming to conclusions, even about intuitive topics. It's not a book about how to "think like an economist through models" or even "think like an econometrician through regressions" but a more simple case for empiricism and openness to overturning one's priors (which is why Levitt's tendency to drive a stake into the ground to defend his preferred counter-intuitive results is disappointing, and his praise of others changing their minds through evidence hypocritical; c.f. Emily Oster on Hep B). Regardless of these flaws, I think the message of thinking empirically is sufficiently nuanced for a general audience today- as an environmental economist, you must have a particular appreciation (word choice?) about how often the public blindly ignore data. Perhaps nothing new since Francis Bacon, but still a valuable insight today.

In terms of exposition, the fact that Dunbar uses essentially a series of vignettes to show this rather than to explicitly articulate it is more an aesthetic choice than a pedagogical failure. It relies on the reader to synthesize the big picture conclusion from the "datapoint" chapters, but in a book about empiricism and synthesis, I would argue it is appropriate to present ideas in this way. In a sense, the reader undergoes training in how to interpret patterns (the papers' consistent counter-intuitive results) into a conclusion (be wary of your intuition; rely on data instead).

All in all, any particular vignette might make for a fun cocktail conversation, but one would hope the conversation might lead into new directions from there- "if we misunderstood car seats, maybe our assumptions about other topics are too strong; what do you believe that might actually be false?" We certainly don't have the data about reader comprehension and response to tell either way, but I can certainly imagine a world where some readers pursue such reflection as a second order effect of reading the book. Perhaps that's a "consequence worth tracing".

* http://www.nber.org/papers/w14356

David Zetland said...

Great comment. Thanks! (I agree that it's real important -- central to academic work and integrity -- that we be able to question our data and theories...)

Post a Comment

Note: only a member of this blog may post a comment.