Comparing the iWatch, Misfit and iPhone as pedometers

Overview

I have three devices that measure my daily steps. All three feed data directly into the Apple health app - and the Apple health app has a nice feature to easily export all this data as an xml file.

This got me thinking - what would happen if I compare the iWatch, iPhone and Misfit to see how my daily step count compares depending on the device used.

The sources

The iWatch are iPhone are two devices that are essential to my daily life. I mainly have the Misfit to track my sleep - but the Misfit is also the device that is permanently on my wrist, as the Misfit battery lasts for months.

As always, the code is in my github repo (minus the data!).

Raw steps

Below are the raw step counts from the three devices. I’ve highlighted when I was in the US (for work), and NZ (for holiday).

Difference

The following plot compares the iWatch and iPhone to the Misfit (as it’s the only device, I’ll treat the Misfit as the closest I have to the true number of steps). Here you can see the iPhone tends to have less steps than the Misfit, which makes sense - as the Misfit is always on my wrist. What doesn’t make sense is that the iWatch has more steps than the Misfit. I don’t know which is more accurate… but as I wear the Misfit more, and on the exact same wrist as the iWatch - there is clearly a difference in the way these two devices count the same number of ‘true’ steps.

As time in the US and NZ probably doesn’t represent my normal patterns of when I use different devices, I exclude that data from here on.

I could do a boxplot or histogram to compare central estimates - but I think a ternery plot makes sense here, as it keeps the dependent nature intact. This means I can easily look closer at where the plots are most discordant. The following plot shows the three devices, in a single graph.

Statistically significant differences

I played a little with modeling - but for simplicity, I thought the Tukey’s honestly significant difference test was the simplest to interpret. It tests all pairwise differences while controlling the increased probability of false positives that comes with multiple testing.

Using this method we see that the Misfit tends to record 787 more daily steps than the iPhone, while the iWatch tends to record 865 more than the Misfit. These two estimates are not that robust - and statistical significance (based on my a priori threshold of 95%CI) was not achieved.

But - when comparing the device that measured the most steps on average (the iWatch), to the one that measured the least (the iPhone), we see that the mean difference was 1,652 steps, with a 95%CI of 278 to 3,025 steps.

I was expecting the iPhone to undercount, but I was a little shocked at the difference between the iWatch and Misfit. It just doesn’t make sense for the iWatch to count more - when both devices are worn on the same wrist and the iWatch is taken off every night, while the Misfit never leaves my wrist.

I still don’t know which is closer to the true number of steps - but I can say that if you are are trying to maximise your daily step count - an iWatch will give you more steps than a Misfit….

Added bonus

The misfit also tracks sleep. I’m going to hold off on looking deep into this data - as I’m interested to look at the effect my daughter will have on the sleep patterns. Below are just some quick plots I made with the data as a first touch.