You spend around a third of your life asleep, with the amount and quality of sleep having a huge impact on the rest of your life.
So it’s natural to be curious about how long you really sleep for, and whether you’re getting enough good quality sleep without any sleep disorders to address.
Until just a few years ago, your options for assessing your sleep were somewhat limited: you could keep a sleep diary if you had the dedication; ask your partner about your sleep; or record yourself sleeping (an unlikely activity!)
The only way to accurately assess your sleep would be to spend a night or two in a specialist sleep clinic. But now there’s another option to improve your accuracy of self-assessment: the personal sleep tracker.
A question of accuracy
How do consumer sleep trackers work though? How do they compare to the equipment and expertise in a sleep lab? And can you actually rely on the information they provide?
These are important questions to consider, especially if you’re thinking of spending a fair bit of money on one.
I’ve been conducting a sleep tracker review for several years, regularly testing new wearable and non-wearable devices. And what I’ve come to realize is that there’s quite a range in terms of what they measure, and how they do it.
So I decided to investigate what researchers have to say about the accuracy of both the devices themselves and the technology behind them.
Hopefully, this article will give you a better understanding of what you can expect from your sleep tracker. And how much you can trust all the information about your sleep they claim to provide.
How sleep specialists measure sleep
To understand the strengths and weaknesses of personal sleep trackers, it’s useful to first take a look at how specialist sleep centers measure sleep.
If your doctor suspects you have a sleep disorder, they might refer you to a sleep clinic for a sleep study, also known as polysomnography.
This typically takes place over a night or two, with different pieces of equipment measuring your sleep stages and cycles. Someone will usually observe you while you sleep as well, and so a lot of information is gathered about your sleep, including:
- Brain waves.
- Heart rate.
- Body movement.
- Leg movement.
- Eye movement.
- Blood oxygen levels.
- The positions you sleep in during the night.
All of this information is then used by sleep specialists to assess your sleep and diagnose any sleep disorders. This detailed analysis and use of brain wave monitoring is why polysomnography is often referred to as the gold standard of sleep assessment.
Sleep trackers Vs polysomnogram
Logically, it’s hard to imagine how a small sleep tracker worn on the wrist, a strip placed under your mattress, or even a smartphone App could do the same as all these high-tech measuring devices. A major difference is that consumer sleep trackers don’t measure brain waves, which is a key way a polysomnogram determines sleep stages.
So I think it’s useful to bear this in mind when it comes to the expectations you might have of your personal sleep tracker. It also leads to an important question: if personal sleep trackers can’t measure all of those factors, what exactly can they reliably do?
As you’ll see later, some of them are reasonably good at measuring basic sleep information, such as the total time you spend asleep and how often you wake up in the night. To understand what they can actually do though, let’s first take a look at the technology inside the trackers.
The science behind wearable sleep trackers
One of the key measuring tools of wearable sleep monitors is called actigraphy. Actigraphy essentially involves recording movement through a measuring device called an accelerometer.
The idea is that a certain amount of movement corresponds with being awake, and periods of being still indicates that you’re asleep.
As Fitbit says on their help page:
When you haven’t moved for about an hour, your device assumes that you’re asleep. Additional data—such as the length of time your movements are indicative of sleep behavior (such as rolling over)—help confirm that you’re asleep.
Why actigraphy can be useful
Actigraphy has been used by sleep clinicians for decades to measure sleep. Even though polysomnography is the gold standard for sleep measurement, actigraphy also plays an important role, especially as polysomnography isn’t without its own issues.
When a patient is hooked up to various machines with multiple electrodes stuck to their head, and in a lab rather than their own bed, they understandably might not sleep normally.
So not only is actigraphy less expensive than polysomnography, but people can wear a device in their own home for a week or two, giving a more natural insight into their sleep than just one or two nights in a clinic.
But how accurate is wrist actigraphy, and what can it accurately tell you about your sleep?
Fortunately, there’s been a fair bit of research into those two questions.
How accurate is wrist actigraphy?
In 2011, Martin and Hakim published some fascinating research into the usefulness of clinical wrist actigraphy. Importantly, they looked at how actigraphy compares to key forms of sleep assessment:
- Clinical interviews and sleep questionnaires.
- Daily sleep diaries.
- Laboratory Polysomnography.
- Videosomnography in children.
They state that wrist actigraphy is useful for assessing sleep in a natural environment, rather than in a laboratory setting. However, they do advise caution against relying on it solely:
Although actigraphy should not be viewed as a substitute for clinical interviews, sleep diaries, or overnight polysomnography when indicated, it can provide useful information about sleep in the natural sleep environment and/or when extended monitoring is clinically indicated.
They claim that wrist actigraphy can be quite accurate when it comes to estimating information such as total time asleep, sleep percentage, and how long after sleep waking occurs.
However, they also state that the main limitation is mistaking being awake and motionless for being asleep. This could then make it difficult for those who wake many times, or with severe insomnia, and who tend to lie still in bed awake rather than move about.
This is also one of the key points which consumers complain about with commercial sleep trackers. Looking at online customer reviews, for example, will turn up many complaints that time lying in bed watching television or reading a book was recorded as being asleep.
Comparing two types of assessment
In 2013, researchers in the United States also looked into the accuracy of wrist actigraphy compared to polysomnography. They found very similar results to the 2011 research:
…we conclude that wrist actigraphy with current algorithms is of value for individual-level estimates of both sleep duration and wakefulness after sleep onset
They also mention the idea that results from actigraphy might help people get an objective measure of their sleep, especially when having treatment for insomnia, for example. Considering many people tend to underestimate how long they sleep for, this could help them see that the treatment is in fact helping.
The limitations of actigraphy
A point I’ve seen made in many online articles where sleep experts have been contacted by the writer is that sleep trackers perform better when you sleep reasonably well.
It’s when you have a disrupted night’s sleep that errors become more apparent. And it seems it’s not just the skeptical owners of sleep labs that think this – there’s research to back it up.
A study by Terri Blackwell in 2008 compared actigraphy with polysomnography in older women. She cautions that the more disrupted the sleep, the less accurate the sleep tracking. However, she did find that actigraphy is reasonably accurate for people to use in their homes:
Actigraphy does not replace polysomnography in sleep estimation, but was a convenient, affordable and accurate method of collecting measurements of sleep in a large epidemiologic study of older women…Sleep parameters from actigraphy corresponded reasonably well to PSG in this population…Those with poor sleep quality had the largest measurement error between the 2 procedures.
A team of Korean researchers in 2018 compared the sleep diaries of 78 people over 14 nights with several sleep trackers:
- ActiGraph GT9X Link
- SenseWear Mini armband
- Basis Peak Fitbit Charge HR
- Jawbone UP3
- Garmin Vivosmart
Even though some of these are now old models or discontinued, some of their findings are still interesting:
Overall, we found that the SenseWear, Fitbit Charge HR, Jawbone UP3, and Garmin Vivosmart can be valid measures of TST (Total Sleep Timre) and TIB (Time In Bed) when compared with a sleep diary in a healthy adult population in a free-living setting. However, these trackers cannot be considered valid regarding wake times during a night of sleep…our study also found greater discrepancies between reported variables during nights that appeared to consist of more disrupted sleep (i.e., the sleep diary reported 7 h of sleep and each band would report between 2 and 7 h of sleep for that night).
So it seems that although wrist actigraphy can be useful for some measurements, it struggles in certain circumstances. And frustratingly, it seems they struggle most when you’re not sleeping well – which is in theory when many people would appreciate some accuracy!
Manufacturers aware of the issue
Manufacturers of sleep trackers are aware of these limitations. In response, some devices allow you to manually override the automatic sleep tracking – either to say exactly when you’re trying to sleep rather than relax in bed, or to correct errors in the morning.
But let’s face it – if you have to keep doing this, your faith in your device is going to plummet.
Beyond actigraphy: heart rate and respiration in consumer sleep trackers
When I first started reviewing sleep trackers, actigraphy was all there was. But now, some include other measures, such as heart rate, heart rate variability, and respiration to help improve their estimations of your sleep.
To make matters more complicated, sleep trackers seem to have branched off into two major types, within which there are subtypes (as I call them):
- Wearables: most are worn on your wrist, and also function as activity trackers and/or smartwatches. Some recent devices are worn on the head.
- Non-wearables: Measuring strips that sit under your mattress and work by ballistocardiography, tracking small movements made as your heart pumps blood and you breathe. Or a device that sits on your nightstand and measures points like breathing, movement, light and noise. One or two that clip onto a pillow. Some smartphone Apps also try to track sleep.
Going back to the Fitbit help page, which also explains how they now track sleep stages, they summarize how they ‘estimate’ them.
And this is a key point: sleep trackers don’t exactly measure your sleep, but they estimate it based on data they collect and algorithms the manufacturer creates:
While you’re sleeping, your device tracks the beat-to-beat changes in your heart rate, known as heart rate variability (HRV), which fluctuate as you transition between light sleep, deep sleep, and REM sleep stages. When you sync your device in the morning, we use your movement and heart rate patterns to estimate your sleep cycles from the previous night.
Sleep stage tracking: the unmeasurable elephant in the room
Estimating your sleep based on a combination of actigraphy, heart rate or breathing is great (when it works), and my feeling is that it will get better in time as technology improves.
However, along with the inclusion of new measuring tools arrived a giant elephant in the room: sleep stages.
Whether sleep tracker manufacturers decided around the same time that the technology was good enough to offer sleep stage data to users, or they were worried about not selling if they don’t offer what competitors do isn’t for me to say.
But here’s the problem: unless you have access to a sleep lab and brain wave measuring, you have absolutely no way of knowing if your sleep stage data is accurate.
So we have a situation where some consumers don’t even worry about it and trust the way their expensive new toy works, while others have suspicions but no way to confirm them.
And neither of those is ideal.
Sleep researchers compare sleep trackers with polysomnograms
I suspect many people would like to know how big the elephant in the room is, but we just don’t own elephant measuring tools. And there’s no authority that regulates sleep trackers to help us out.
However, there have been a handful of studies in recent years where sleep researchers have compared sleep trackers with polysomnograms, EEG or sleep diaries.
The problem, as I see it, is that manufacturers regularly release new models or update the firmware. So most available published research involves sleep trackers that are either older models or entire brands that have since discontinued (such as Jawbone).
Still, they are interesting to consider as there are some common findings that crop up more than once.
1. Portable EEG + Fitbit Versa (older model)
A study in 2019 particularly interested me as it involved one of my favorite wearables – the Fitbit Versa (even though the Versa 2 is now out).
20 people used the Versa for 14 nights, as well as a portable EEG. They found the Versa was similar to the EEG for total sleep time, time in bed and calculated sleep efficiency. However, it differed when it came to sleep stages, time to fall asleep and time of first waking. They concluded:
The consumer sleep tracker could be a useful tool for measuring sleep duration in longitudinal epidemiologic naturalistic studies albeit with some limitations in specificity.
2. Polysomnogram + Jawbone UP3 (now discontinued)
Even though the Jawbone UP3 is now discontinued, this 2017 study is interesting because 17 patients wore the UP3 while undergoing a polysomnogram in a sleep lab.
The results weren’t the best for Jawbone though. They found it significantly overestimated total sleep time and sleep efficiency. And it was found to overestimate REM latency, total REM sleep time and deep sleep time. And it underestimated the time to fall asleep and time of first waking.
The grim conclusion was:
The JB3 inaccurately estimated sleep duration and efficiency relative to PSG. Further investigation to determine JB3 specificity, sensitivity, and accuracy for sleep staging, as well as the evaluation of the JB3 to measure sleep during naps, is warranted.
And considering the lack of studies like this done for any sleep trackers, it didn’t inspire much confidence in general!
3. Polysomnogram + Jawbone
Just to be fair to Jawbone, especially since we’re all hoping for a resurgence in the future, there was an interesting study in 2015 of the Jawbone UP before sleep stages came along. The results were more positive than the previous study, as the authors say:
Jawbone UP shows good agreement with polysomnography in measures of total sleep time and wake after sleep onset in adolescent boys and girls.
4. Polysomnogram + Bodymedia SenseWear Pro Armband (medical actigraphy) + Withings pulse 02 + Jawbone Up
This study compared two commercial trackers – the Withings Pulse 02 and Jawbone UP with PSG with medical lab sleep tests, specifically with 36 obstructive sleep apnea patients.
They again found mixed results with the only positives being that they correctly assessed time in bed and light sleep. But no other measures were deemed to be accurate.
5. Polysomnogram + Actiwatch-64 (medical actigraphy) + Fitbit
This study, done in 2012, is now pretty old and Fitbit has made significant changes to its sleep tracking. However, since highlighting Jawbone’s problems, it seems only fair to bring this one up.
Interestingly, despite being caught out by the polysomnogram, it was at least no worse than the medical grade actigraphy, as noted by the authors:
Fitbit has the same specificity limitations as actigraphy; both devices consistently misidentify wake as sleep and thus overestimate both sleep time and quality
6. EEG + Fitbit Charge 2 (older model)
If heart rate changes are used when estimating sleep, surely that’s an easier one to check – especially during exercise.
The Charge 3 is now the latest version, but nevertheless, a study in 2018 found some worrying results while recording the heart rate of 15 participants riding a stationary bike.
Compared to an electrocardiograph, they found that while it wasn’t too far off in some cases, there were incidences where the error was large:
Whilst there is only moderate bias on average, precision is poor for individual measurements, which could plausibly be underestimated by as much as 30 bpm.
To be fair, the authors do note that the way the device is worn on the wrist can lead to errors. And this, presumably, can also happen during sleep:
…instability and improper positioning of the device may potentially explain different results and poor-quality HR data.
7. My own tests in 2018
In 2018, I tested 3 different sleep trackers at the same time, comparing them to my manual sleep diary. I didn’t compare them with an objective polysomnogram, but I thought the results were interesting nonetheless.
One standout result was that the sleep stage reporting of all three was wildly different. So that ruled out the admittedly small chance that they were all accurate!
And with other points I could measure myself, like estimated time of falling asleep and time of waking, I found the Versa and Beautyrest were ‘reasonably’ accurate, while the Emfit struggled.
And that raises another point: where do we draw the line where accuracy is concerned? One person’s definition of ‘reasonable’ could be another’s motive to log into eBay.
The results of the studies above aren’t exactly reassuring – to put it mildly.
As I said though, technology advances quickly, and to my knowledge, there are no published independent studies of the latest devices made by the companies which have been put to the sword. And there are a few brave new companies entering this lucrative, yet vulnerable to self-destruction, market.
So what can you take from all of this?
Well, my personal take remains the same as it did when I first wrote about sleep trackers six years ago: take the results with a pinch of salt, and don’t get too caught up in the finer details of the sleep tracking data – especially where sleep stages are concerned.
I think it’s good to have realistic expectations of what you can get from your sleep tracker. They are potentially a useful tool for giving you an overview of your sleep. Just be aware that they aren’t perfect yet, and will make mistakes from time to time.
It’s still a developing technology, and it will be very interesting to see just how accurate they get in the coming years.
For now though, if you have sleep problems, remember to speak to your doctor/physician about it.
At the end of the day, they and the sleep specialists they can refer you to are still the most reliable way to find out about your sleep and diagnose any sleep disorders.
What do you hope to find out from using a sleep tracker? Which device have you tried before, and did you find it to be accurate, or provide you with useful information?