Now live Black Friday✓ HSA/FSA eligible² | 30-day money-backShop now
Now live Black Friday Two worlds, one sale. Wellness or Play, pick yours.✓ HSA/FSA eligible² | 30-day money-back

How to read a sound and wellness study: 7 questions that cut through the hype

Written byAnastasia DeripaskaPublished Reading6 min
An open notebook beside a printed research paper with a pen and reading glasses on a cream linen tablecloth, soft daylight

"Study finds music makes exercise easier." Headlines like that are everywhere, and they rarely tell you much. Was it 12 students or 12,000 adults? A feeling or a measurement? One lab or ten?

You do not need a science degree to tell a strong study from a weak one. Knowing how to read a study comes down to seven good questions, shown here with real examples. Trisha Greenhalgh wrote a well-known BMJ series, how to read a paper. Her starting point is ours: ask what the study actually set out to do.

1. Who took part?

Look at the people first. A study of healthy students tells you about healthy students, not necessarily anyone else.

In the 2013 running study from our sister blog, all 19 participants were recreational runners, students aged 19 to 27 from the Faculty of Human Movement Sciences at VU University Amsterdam. That is not a flaw. It is simply the boundary of what the study can say.

2. How many people?

Small studies are fragile. A 2013 paper in Nature Reviews Neuroscience showed that small studies often have low statistical power. That means they can miss real effects, and when they do find one, they tend to make it look bigger than it is.

A study with 20 people is a first look. The music and exercise meta-analysis we covered on our sister blog pools 139 studies, which gives a steadier picture than any one small study. Its authors rated all 139 as low to moderate in quality, so a review is only as strong as the studies inside it.

Two rows of wooden chairs facing each other in a bright empty room

3. Was there a comparison group?

Without a comparison, you cannot tell what caused a change. People's ratings and readings change over time anyway, and a comparison tells you how much of a change is just that.

The 2013 running study handled this well. Each of the 19 runners did all three conditions, silence, a metronome and music, on different days and in shuffled order. So every runner was their own comparison, and because the order was shuffled across runners, a good day or a bad day was unlikely to favor one condition.

There is a subtler trap too, called regression to the mean. A person with an unusually high or low reading on the first day will often land closer to their average the next time, with nothing done at all. A classic paper in the International Journal of Epidemiology explains the effect and shows how study design helps guard against it.

4. Did people know what they got?

If you know you are getting the real thing, you may feel better simply because you expect to.

Some studies can hide what people get by using a look-alike stand-in. Music cannot hide: people always know whether they are hearing it. The authors of the 139-study exercise review note that this rules out double-blind designs in music research. Those studies are still worth reading, but expectations are part of the result.

5. What was actually measured?

A feeling, a body signal and a real-world result are different things. How hard a workout felt is not how long someone kept going.

Read the headline, then find the measure. "Music makes exercise easier" might mean people kept going longer, or that they said it felt lighter, and those do not always move together.

A 2009 treadmill study shows why that matters. Thirty young adults walked longer with motivating music than with neutral music or silence. But their ratings of how hard the walk felt did not differ significantly. A headline that said "music makes walking feel easier" would have reported the wrong half of the result.

A hand holding a magnifying glass over a printed bar chart on a wooden desk

6. How big was the effect?

A result can be statistically significant and still small. Significance says a difference would be unlikely if there were no real effect. It does not say you would notice it.

Good papers report effect sizes, so look for a number you can picture, such as minutes, points on a scale or percent change. In the running study, the number is easy to picture: runners lasted about 10 minutes in silence and about two minutes longer with a beat. In the music and exercise review, music lifted mood with a moderate effect, but its effect on performance was small. Both are real, but they differ in size.

7. Has anyone repeated it?

One study is a starting point, not an answer. John Ioannidis made the case in a much-cited 2005 essay in PLOS Medicine. Findings are less likely to hold up, he argued, when studies are small, effects are small, and designs are flexible. The remedy is replication: another team, another sample, the same result.

When several teams have found the same thing, the picture gets much stronger.

A bonus question: who paid?

Funding does not make a study wrong. Plenty of good research is paid for by companies, and plenty of weak research is not. But it is worth knowing, because the Ioannidis essay lists financial interest among the things that make a finding less likely to hold up. Most journals list the funding and any conflicts of interest near the end of the paper, so checking takes seconds.

What these questions cannot do

These seven questions will not make anyone an expert. Some judgments, like whether the statistics were done well, need training. Taking any single study as proof, in either direction, is the mistake to avoid. So is dismissing a whole field because its first studies were small.

How we use these questions

Every study on this blog is linked so you can check it. We say who took part, how many, what was measured and what the study did not show. You will find that in our guide to vibroacoustic sessions and in vibration plate or vibroacoustic. No study cited in this article tested Woojer products. Woojer products are not medical devices. If we ever fall short of these standards, we want to hear about it.

Putting it together: how to read a study in five minutes

Here is the short version to keep on your phone:

  • Who took part, and how many?
  • Was there a comparison group, and did people know what they got?
  • What was measured, and how big was the change?
  • Has anyone repeated it?

A study that answers all seven well is rare. Most good research answers some of them and is honest about the rest. That honesty is a good sign in itself.

Frequently asked questions

What is a pilot study? A small first study that tests whether a bigger one is worth running. Its results are a hint, not a conclusion.

What does randomized mean? People are assigned to groups by chance, so the groups start out alike and differences later are easier to trust.

This article is for general information only and is not medical advice. Talk to a qualified health professional about any health condition.

Sources

  1. Greenhalgh T. How to read a paper: getting your bearings (deciding what the paper is about). BMJ, 1997.
  2. Bood RJ, Nijssen M, van der Kamp J, Roerdink M. The power of auditory-motor synchronization in sports: enhancing running performance by coupling cadence with the right beats. PLOS ONE, 2013.
  3. Button KS, Ioannidis JPA, Mokrysz C, et al. Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 2013.
  4. Terry PC, Karageorghis CI, Curran ML, Martin OV, Parsons-Smith RL. Effects of music in exercise and sport: a meta-analytic review. Psychological Bulletin, 2020.
  5. Barnett AG, van der Pols JC, Dobson AJ. Regression to the mean: what it is and how to deal with it. International Journal of Epidemiology, 2005.
  6. Karageorghis CI, Mouzourides DA, Priest DL, et al. Psychophysical and ergogenic effects of synchronous music during treadmill walking. Journal of Sport and Exercise Psychology, 2009.
  7. Ioannidis JPA. Why most published research findings are false. PLOS Medicine, 2005.