If you think you can chow down guilt-free on a 540-calorie Big Mac because the smart watch on your wrist says you just burned that many calories at the gym, guess again.
A new study found that while devices such as the Fitbit Surge and the Apple Watch provided accurate heart rate measurements, they were way off as far as how many calories people burned when exercising. Sometimes the results were too high, sometimes they were too low, but they were never just right, or even close to it.
The Fitbit Surge had the lowest error margin for energy expenditure of the seven devices tested, although it was still off by an average of 27%. The PulseOn had the highest error margin for energy expenditure, off by an average of 93%.
On the other hand, coauthor Dr. Euan Ashley told me, “I was surprised at how accurate they were for heart rate. It made me feel all warm inside. As a physician, I would really love it if these devices were 100% correct.”
Ashley, a professor of cardiovascular medicine, of genetics and of biomedical data science at the Stanford University School of Medicine, said he and his collaborators were curious about the accuracy of popular wrist-worn fitness trackers, used by millions of people, because consumers don’t have access to the results of tests conducted by the manufacturers.
The researchers enlisted the help of 60 volunteers, 31 women and 29 men, who were at least 18 years old. The participants were a diverse group, a deliberate move on the researchers’ part so they could examine whether factors such as age, fitness level, weight or skin color might affect the devices’ accuracy. Besides the Fitbit Surge and the PulseOn, the volunteers tested the Apple Watch, the Samsung Gear S2, the Basis Peak (all of which were recalled last August, after the study was conducted, because they overheated), Microsoft Band and Mio Alpha 2 while walking or running on treadmills or using stationary bicycles. The researchers had planned to test an eighth fitness tracker, but multiple ePulse 2 devices had technical problems and were excluded.
While the study subjects exercised, the researchers measured their heart rate with an electrocardiograph and estimated their metabolic rate and energy expenditure based on the oxygen and carbon dioxide in their breath. They then compared those findings with the numbers generated by their fitness trackers.
“Is it possible that some people are just very hard to measure?” wondered Ashley and his collaborators, whose research was published online Wednesday in the Journal of Personalized Medicine . But no study participants ended up with consistently inaccurate measurements across all activities, he said, although device error on average was higher for men, greater body mass index (BMI), darker skin tone and walking (cycling had the lowest error rate of all the types of exercise studied).
So why was the calorie-counting so inaccurate? No one besides the manufacturers can know for sure, because the algorithms they developed to measures things like energy expenditure are proprietary. “We think each of the devices would use a different equation,” Ashley said. Considering that the devices measured heart rate accurately, he speculated that the companies’ algorithms for energy expenditure might not take heart rate into account.
I sought a comment from Fitbit, which held the largest share of the global wearables market until Apple overtook it in the first quarter of this year, according to research by Strategy Analytics.
“Fitbit trackers show an estimated total number of calories burned based on users’ BMR (basal metabolic rate) and activity energy expenditure (AEE),” the company told me in a prepared statement. “Fitbit uses a scientifically validated estimate of BMR based on height, weight, age and gender information that users provide when setting up their Fitbit account.