An Important & Badly Designed Survey

A massive survey across three countries claims to act as a bellwether for people’s views on Coronavirus priorities.  I genuinely hope it isn’t influencing any decisions but I know in my heart of hearts that it is.

This survey asks a simple, obvious question, “What matters more: trying to save every life or protecting the economy?”  Survey respondents in every country have, unsurprisingly, said they care more about saving lives than they do about money. 

Is health more important than wealth?

Don’t Ask A Question & Give False Choices

The good thing about the survey result is that people are lovely.  The bad thing is everything about the question that led to the result.  It commits the basic and deceptive logical fallacy of offering a false choice.  Here’s just 3 things wrong with this choice:

1. The question asks us to distinguish between 2 things that aren’t independent.  In fact they’re highly dependent on each other.  A stronger economy saves lives, with studies of wealth versus longevity showing how important the economy is for saving lives and for years of life.  And conversely, because at its foundational level the economy is just people doing valuable things for each other, losing lives is by definition a loss to the economy

2. The survey doesn’t present the choices in a like for like way.  One is extreme, emotive and individual; the other is measured rational and impersonal.  In fact it’s presented in the most non like-for-like way I can think of, other than “would you rather (1) save this kitten from drowning or (2) see an extra 0.1% annual return on the median tax free savings account?”

3. The survey designers have presented just these 2 factors.  They’ve ignored all the other factors that affect the broader underlying question of how cautiously we should open up after lock down.  There are many, many overlapping factors here that might contribute to more or less caution.  One obvious missing factor to understand, to my mind, in an enforced but popularly supported lock down, is freedom.

Is Freedom More Important Than Health? Evidence from Key Workers

I’m not aware of anyone asking directly about the importance of freedom in any recent surveys.  But digging into the data from a couple of them does give us some hints about how freedom stacks up in importance against health and “the economy.”

To avoid philosophical rabbit holes, I’ll just say that freedom is the ability to get out and about and away from home confinement.  One group with this honour is the newly defined key workers, and an ongoing ONS survey compares this group’s behaviour with other workers.  

Key workers are having to work harder while the rest of us are working less.  They’re also much more concerned about their health than the rest of us.

Key workers are working harder and more worried about their health

If we believe that health and safety dominates people’s concerns then this should be reflected in lower life satisfaction.  But key workers say they are more satisfied, feel more worthwhile, are happier and are less anxious than the rest of the working population.

Is freedom more important than health and wealth? Key workers are more satisfied than other workers

Evidence like this can give us no more than hints, but it looks at least like the freer group has higher satisfaction despite much greater health and safety concerns.  

Maybe it’s not just their greater freedom though.  Perhaps there’s something else at play here because key workers report greater job and financial security.  Maybe also the key worker demographic is the sort that would be happier with life anyway.

Is Freedom More Important than Health & Wealth? Evidence from Rebellious Young Men

To address this potential bias, we’ll look at a different group from another survey.  It’s one that enjoyed freedom because it broke the rules.

A King’s College/Ipsos Mori survey used data clustering on survey sample to identify 3 groups in the UK lock down.  An Accepting group happily bought into the lock down and was older and predominantly male.  A Suffering group did not like but put up with the lock down and was predominantly female.  A Resisting group actively revolted against the lock down, and was predictably predominantly young males.  To make comparison as like for like as possible, I’ll just compare the birds not content to be caged: the Suffering and the Resisting.

The Suffering group comply with and even support lock down.  The Resisting are a bit rebellious and have gone out and about, breaking the rules.

The Suffering group are complying, the Resisting group are rebelling

Our freedom loving Resisting group has more health worries and financial worries than the Resisting group.  So they’re similar to the key workers in being worried about their health, but they’re also worried about their financial futures.

The Resisting group are more worried about their health and wealth than the Suffering

But despite being less accepting of the lock down, more certain that they’ve had Coronavirus, and more precarious financially, the freedom seeking rule breaking Resisting group claims to have lower anxiety and helplessness than the more disciplined and compliant Suffering group.

Is freedom more important than health and wealth? The Resisting group are happier than the Suffering

So a legitimately freer group, key workers, says it’s happier despite having much greater health concerns than the rest of us.  An illegitimately freer group, The Resisting, says it’s happier despite thinking it’s got more health and financial concerns.  This is all hinting at freedom being an important ignored issue.

Hints Only, But We Can Be Confident It’s About Much More Than Health & Wealth

Our analysis is far from solid.  We’ve relied on studies that use personal assessments as opposed to solid measures of actual behaviour.  The second study uses a statistically dubious technique called data clustering that commits a basic statistical sin called sampling on the dependent variable.  And we’ve ignored every other measure except for health, finance and freedom.  So we’re guilty of the same lack of exhaustiveness that I criticised in the first chart in this article.

But I think the data shows enough for us to suspect that health and finance may not be the only important issues when making decisions about releasing our self administered restrictions, even if the survey asked the question properly and didn’t offer a false dichotomy of choices.  And maybe freedom should be weighing a bit more heavily.

by Steve Hacking & Jim Powell


YouGov survey of attitudes to Coronavirus in Britain, USA and Germany

ONS survey on Coronavirus behaviours & attitudes of key workers and other groups:

Opinions and Lifestyle Survey (COVID-19 module), 24 April to 3 May 2020 

King’s College and Ipsos-Mori data clustering analysis of the Accepting, the Suffering and the Resisting