Trackers - recommendations

[mccoy]

 

[mccoy]

The video above, part of a series where Thequantified scientist tests in a rather rigorous way many devices is not encouraging. A few points:

• Apple watches are always good or very good with HR and sleep monitoring
• Some other watches are pretty good with HR and sleep monitoring
• Most non-apple are not very good with sleep monitoring
• Apparently no or very few non-apple is good at either heart tracking and sleep tracking. 

 

 Out of the Apple ecosystem, if we are interested only in at-rest heart tracking, then we may choose the models with the best sleep-tracking capabilities.

Here the Fitbit charge 5 would seem promising, but there remain the issues with HRV some users complain about.

For example, this is a most useful graph showing Applewatches are the best for both functions, whereas huawei watches are very good at HR but pretty bad at sleep monitoring.

 

Edited July 15, 2023 by mccoy

[mccoy]

[mccoy]

[mccoy]

For those who are (still) interested, I've gone thru a few hours of watching and studying the videos (the 'quantified scientist' series and others), plus reading the comments from the users on Amazon and in various Reddit fora, plus consulting the characteristics declared by the producers.

So, I developed a sort of decision tree. One caveat, the suggestions below are valid for those, like me, who would like mainly a health tracker, not to be used in conjunction with more than a mild exercise. So, main parameters I considered are recounted reliability of the HR sensor, the reliability of dream architecture algorithms, and other health sensors like spO2. I did not consider at all GPS, step count, and the plethora of smartwatch tech gadgets that are not related to health (notifications, music and so on).

 

DECISION TREE FOR HEALTH TRACKING

1. Do you already possess an Apple phone? if yes, goto step #2. If no and not willing to change from Android to Apple, then goto step #4
2. If already an owner of iPhones, or willing to go thru the change and cost of switching from android to Apple, the best option in absolute, according to the tests of the Quantified scientist, is an Apple watch. The later models yield constantly better results in terms of HR tracking AND sleep tracking with respect to all other brands.
3. The overwhelming reliability of apple Watches come at a monetary cost, plus at an energy cost: the battery seems not to be able to go beyond 24 hours, maybe less. Still, if I had an iPhone, I would get an Apple watch, recharge it every evening before going to bed, and enjoy the most reliable tracking these kinds of devices can offer. All the above of course assuming the reviews of the quantified scientist are correct and unbiased, and I saw no apparent reason why it should not be so.
4. IF the choice is a device operating on Android, then we must decide the feature(s) we desire most.
5. If we desire the best HR tracker within the Android devices, to be used with diversified exercises, including weight lifting, then there are a few options which are to all practical effects as reliable as Apple watches, the best one presently apparently is the Huawei GT3 Pro.
6. If we desire the best HR tracker with mild exercise, then many other models are reliable, even the Fitbits and some moderate-cost devices.
7. If we desire the best sleep tracker device, then Huawei lags behind and the Fitbit devices and the Pixel watches, both owned by Google which evidently developed a decent sleep architecture algorithm, are almost on par with the Apple watches. Fitbit inspire3 got a very good score, but based on just two trials, other Fitbit devices also being good, reflecting the goodness of the same algorithm.
8. If we desire a good compromise between sleep tracking and HR, then the Fitbit/pixel devices are still the best choice, since they are all reasonably good at rest, in daily activity and with nothing more than activity on a stationary bike. 

All the above are just general, non-specific guidelines. Some of us may privilege some activities, like running on the road for example, then the Garmin Forerunner would be one of the best choices, using something else for sleep tracking.

Combinations are also possible, like wearing 2 non expensive trackers like the Inspire 3 for sleep and the Huawei BAnd6 or Ban7, very inexpensive for HR, or even The Inspire 3 for decent sleep tracking and the Huawei GT3 Pro (more expensive) for excellent HR tracking.

 

 

Edited July 16, 2023 by mccoy

[mccoy]

After all of that time, luckily I was able to decide. I ordered a Fitbit Inspire3 for US$ 90 at the present exchange rate. I'll have hopefully a good sleep tracker and a decent HR tracker at rest and during mild life activities. I also appreciate the unobtrusive size, the lack of a GPS I don't need, the lack of other Hi-tech gadgets I don't need. 

When exercising more strenuously, I'll wear a Pola H10 strap. I'm also going to use the strap to check The watch HR sensor capabilities.

If the experiment fails, maybe I'm going to purchase a Huawei GT3pro and separately a Dreem2 headband, which is presently the most reliable sleep tracking device aside from medical grade equipment. Or weare both the Huawei and the Inspire3, possibly wearing the Inspire3 only at nights.

The possibilities of optimization are really infinite and out there there is a wide choice for all pockets and requirements.

I'm going to add  to this post if something new comes up, plus I'm going to relate my experience with the Inspire3, which I ordered yesterday.

 

[Ron Put]

2 hours ago, mccoy said:

For those who are (still) interested, I've gone thru a few hours of watching and studying the videos (the 'quantified scientist' series

McCoy, I've been traveling, or I would have responded sooner. My personal opinion is that the "quantified scientist" should largely be ignored, at least as far as Garmin Plus 2 and to an extent Huawei.On the Venu Plus 2, he is absolutely wrong, especially on Sleep. I know, because I own it, and bought it right after his review came out. It's because half the people I know own either Apples or Garmins, and a couple of them had already jumped on the Venu Plus 2. He may be sincere, but he may have had an early or a problematic device. Garmin has also dramatically improved its sleep tracking over the last year and when I worked it with my Versa a few months ago, it tracked better than the Versa over a three or so days.

A few years ago, Fitbit was the king of wrist-based sleep tracking (but had issues with HR accuracy under higher loads). Fitbit was also too optimistic about VO2Max, to the tune of 5+ points. Fitbit has improved its HR tracking a bit, but still seems to lag behind Apple, Garmin or Suunto. Huawei seems to test similarly to Fitbit, if not a little worse, but it's still the most accurate among the Chinese brands (it's also incomparably with any other platforms).

You will be happy with the Fitbit/chest strap combo.

[barrymilels]

Garmin devices are popular among fitness enthusiasts. The vívosmart 4 offers features like heart rate monitoring, sleep tracking, and stress tracking. It also provides advanced metrics like blood oxygen saturation levels and body battery energy monitoring. It has a slim and comfortable design with a touchscreen display. MLSDev https://mlsdev.com/services/custom-software-development can integrate custom software solutions with existing systems and third-party services using APIs (Application Programming Interfaces) to ensure seamless data exchange and functionality. Quality Assurance and Testing: MLSDev follows rigorous testing procedures to ensure the reliability, security, and performance of the custom software solution.

Edited July 24, 2023 by barrymilels

[mccoy]

19 hours ago, Ron Put said:

On the Venu Plus 2, he is absolutely wrong, especially on Sleep. I know, because I own it, and bought it right after his review came out. It's because half the people I know own either Apples or Garmins, and a couple of them had already jumped on the Venu Plus 2.

Ron, it may well be that the quantified scientist is wrong on some models, it would be strange if he got all it right (and sometimes he himself underlines his limitations). I was surprised to see the Oura ring lagging behind some watches in sleep and HR tracking, given that the Oura is specialized in health tracking without the other gadgets, and ditto the whoop strap. I still have some doubts about that.

How did you determine the goodness of sleep tracking of the Venu plus 2? Comparisons with some Apple watches models?

I'm curious about my test with the Inspire 3 and do not exclude negative surprises, for one the convenience of being small comes with the inconvenience of hard reading...

Edited July 17, 2023 by mccoy

[Ron Put]

On 7/17/2023 at 4:14 AM, mccoy said:

Ron, it may well be that the quantified scientist is wrong on some models, it would be strange if he got all it right (and sometimes he himself underlines his limitations).

I base my opinion of his tests primarily on my own experience, and also on the experience of those whom I know that have some of these devices, as well as on other reviews from reputable testers that make sense to me.

As I said, Garmin has not been the greatest for sleep tracking (neither is Apple). But a few years ago Garmin acquired Firstbeat and while my previous Garmins were not great at sleep tracking, although by no means bad, literally within a month after I bought the Venu 2 Plus a firmware update made the whole thing far more accurate. I wore my old Fitbit Sense on for several nights, and I tracked with the chest strap for close to a month, while comparing to the V2+.

The Garmin's greatest weak spot was correctly detecting the time I went to sleep, or woke up, if I lingered in bed (which I often do). Fitbit was better at it (Samsung also sucked). But after the update the V2+ was suddenly spot on, and tracking my short wake times during the night also perfectly, while the Fitbit was not as accurate.

Also, see for example this (DC RM is one of the more reliable testers, IMO):

Garmin Venu 2 Plus In-Depth Review | DC Rainmaker
 

Edited July 19, 2023 by Ron Put

[mccoy]

Ron, I'm very new to the world of tracking devices but, as I understood it, the quantified scientist graded the sleep trackers by their similarity with the sleeping pattern or architecture provided by the Dreemband2, which is probably the most accurate proxy of medical grade polysomnography.

Whereas, tracking effective sleep duration (not apparent sleep in bed) is probably simpler, although not without its importance.

[Saul]

On 7/7/2023 at 1:09 PM, corybroo said:

I just received a Fitbit by volunteering to be a subject for the All of Us project.  (Considerations while using Fitbit Data in the All of Us Research Program)

Here's what I've found so far:

It's new, so I need to learn about the Fitbit to maximize its usage.  

My son mentioned one benefit of wearing a tracker of any sort was when his wife was rushed to the hospital a year ago.  At arrival, when the staff saw her smart watch, the first question was, "May we download the data?"  So even if you don't use the charts and data, if something occurs and there is a need to see what was the state of your body leading into the situation, that data would be available for medical professionals to help in diagnosis and treatment.   

Hi Corybroo!

I also was offered a free Fitbit from AllOfUs -- I will be playing around with ti when it arrives in about a week.

  --  Saul

[Ron Put]

15 hours ago, mccoy said:

the quantified scientist graded the sleep trackers by their similarity with the sleeping pattern or architecture provided by the Dreemband2

Yeah... His review of the V2+ (and of Huawei, and of Apple Watch, to an extent) is an outlier, and it doesn't comport with experience, at least mine and many others'. I saw that he has been repeatedly asked to rerun his test, but he never did, and since the Venu 3 is about to launch, I doubt that he will. Good credentials, but IMO a very flawed review, for whatever reason.

All current devices have various strengths and weaknesses, but most are perfectly fit for casual use.

[mccoy]

12 hours ago, Ron Put said:

All current devices have various strengths and weaknesses, but most are perfectly fit for casual use.

Maybe I'm going to build up a good arsenal of tracking devices, with all their pros and cons. For example:

1. Dreem 2 headband for sleep architecture (the reason why I don't already have it is that my sleep is not governed by myself presently, but rather by the patterns of my autistic son and I have very little room to intervene).
2. Aktiaa blood pressure bracelet (I already have it)
3. PolarH10 chest strap for very reliable HR and HRV (I already have it)
4. Fitbit or smartwatch for reasonablee HR and HRV , maybe reliable Sleep architecture (to be chosen after careful research and to be checked by the above more reliable devices).

If I'm lucky, the minimalist inspire3 is going perhaps to be a good start with #4, maybe, maybe not

I'm still trying to figure out why the quantified scientist tests rated the Oura ring not so high, considering it's a dedicated, arguably well-designed health tracking device... I was considering its purchase but it cast some doubts, Maybe the limited dimensions come with a drawback.

Edited July 19, 2023 by mccoy

[mccoy]

12 hours ago, Ron Put said:

eah... His review of the V2+ (and of Huawei, and of Apple Watch, to an extent) is an outlier, and it doesn't comport with experience, at least mine and many others'. I saw that he has been repeatedly asked to rerun his test, but he never did, and since the Venu 3 is about to launch, I doubt that he will. Good credentials, but IMO a very flawed review, for whatever reason.

Something is strange with the quantified scientist's reviews, for example, he recommends the whoop bracelet for health tracking even though it's costly and it doesn't rate very high. 

What do you think about his reviews of Huawei and Apple watches? Too optimistic? It looks strange that all apple watches overwhelm other devices so much, considering they are not just health tracking devices, although the hi quality could be justified by the extra high price. All speculations, and yes, he goes on with the basic database he published with little updates and why he didn't publish HR tracking goodness at rest and why he said nothing about HRV algorithms....

[Ron Put]

I think the Inspire 3 will be a good device to start with, although I seriously resent the monthly charge, and would never buy a Fitbit/Google tracker because of it. Here is a pretty reliable review of it, if you haven't seen it before:
Fitbit Inspire 3 review - Wareable

The quantified scientist's reviews methodology sounds good, but some of his results are outliers. Like those for Garmin Venu 2+. Huawei is the best of the Chinese brands in terms of accuracy, but it's middling compared to Western brands and it does not compete well. Apple does have stellar sensors and algorithms, on par with the best, and sleep tracking has improved, but rating it at the top for sleep is probably a stretch, based on other reviews and also on anecdotal evidence from the people I know who bitch about it 🙂

BTW, I tested sleep on my Venu 2+ against a Fitbit Sense, not Versa (I'll correct above). I keep forgetting the Sense, since I used it for a relatively short time before I switched to Garmin.

[Ron Put]

I should add that any reputable tracker is better than none. IMO, having information is almost always better than not having it, and tracking data has caused me to change various aspects of my lifestyle, including drinking alcohol, eating deserts and how I exercise.

I would again strongly recommend the Whithings Body Cardio scale, which provides key body composition information that on most points jives with my DEXA scans, plus it provides Pulse Wave Velocity measurement that seems fairly accurate too.

The Pulse Wave Velocity bit is a big deal in a consumer device and it seems to make sense as far as validity goes: it measures arterial stiffness and it appears to be predictive of cardiovascular health. I've been trying to optimize my PWV with diet and lifestyle and today it was 6.3 m/s, not bad for a 60 year old 🙂

 

Edited July 19, 2023 by Ron Put

[mccoy]

In the meanwhile, I have the first measurements from the Inspire 3. I too am not too happy about the premium fee, which runs at about 90 US$ per year. I'm going to cancel the membership before the trial 6 months expire and see if what I have is enough. Fitbit Premium has many features which are not extremely useful.

Sleep. Of course I cannot say much about sleep stages, except that probably the first deep sleep period has been detected correctly. This based on triangulation with the Aktiia bracelet, which in the same time range detected a very low systolic BP of 95 mm Hg. From previous experience, I have these lows in BP while sleeping and the most reasonable explanation is the effect of deep sleep.

About effective sleeping time, it's close and about REM sleep, maybe it read some REM sleep as light sleep, since I dream a lot and would suspect REM stages are longer. This is the graph produced by the algorithm. It's in Italian but it is intuitive, the troughs are deep (profondo) sleep stages, the baseline is light (leggero) sleep, the peaks are REM and the red spikes are times of awakening. I recognize the wake period after 02 hours, I actually stood up and go check the main power switch of the house which went off. The other 2 awakenings have been perhaps overestimated.

 

Edited July 20, 2023 by mccoy

[mccoy]

I also checked again the table provided by the quantified scientist, which shows that, compared to the Dreem2 headband, the Inspire3 is pretty accurate in detecting wake stages and deep sleep, less so in detecting REM sleep. These data, personal memories and triangulation by other devices may help in interpreting the sleep stages graphs provided.

Edited July 20, 2023 by mccoy

[mccoy]

One disappointment was tonight's HRV. I read quite a few complaints about apparently too low values of HRV in reddit fora. So far, I would agree with the complaints, since the value detected by the Inspire3 was 22 ms, whereas the value detected by the Polar H10 strap (in the morning resting at bed before breakfast) was 45, practically twice as much. 

I'm going to see if the divergence remains, in the next measurements

SpO2 during sleep was measured as 94%, which may be good since my wake baseline is usually about 96% and by night this parameter is slightly lower than during the day

[Ron Put]

McCoy, I really think you place way too much stock in the qualified scientist's videos. Some of his results are way out of whack with those of others, which I guess is what helps him surge and go viral. He does sound convincing, but his results do not always make sense.

The Dream 2 is no longer sold, and no longer supported. If I remember, it had decent accuracy but still had about 15% error margins even in the amount of sleep, with most likely greater margins for sleep stages. Combined with aberrations due to factors such as placement, sweat, tightness on wrist and so on, I'd take his videos with a grain of salt, especially when they show results that are dramatically different than those seen in other reviews.

I just searched for a "real" study I'd seen in the past, and I think this is it:

"Compared with polysomnography, agreement (and Cohen's kappa) for two-state categorisation of sleep periods (as sleep or wake) was 88% (κ = 0.30) for Apple Watch; 89% (κ = 0.35) for Garmin; 87% (κ = 0.44) for Polar; 89% (κ = 0.51) for Oura; 86% (κ = 0.44) for WHOOP and 87% (κ = 0.48) for Somfit. Compared with polysomnography, agreement (and Cohen's kappa) for multi-state categorisation of sleep periods (as a specific sleep stage or wake) was 53% (κ = 0.20) for Apple Watch; 50% (κ = 0.25) for Garmin; 51% (κ = 0.28) for Polar; 61% (κ = 0.43) for Oura; 60% (κ = 0.44) for WHOOP and 65% (κ = 0.52) for Somfit. Analyses regarding the two-state categorisation of sleep indicate that all six devices are valid for the field-based assessment of the timing and duration of sleep. However, analyses regarding the multi-state categorisation of sleep indicate that all six devices require improvement for the assessment of specific sleep stages."

 

[mccoy]

22 hours ago, Ron Put said:

The Dream 2 is no longer sold, and no longer supported.

I didn't know, Dreem has been acquired by Beacon and now the headband (Dreem3) will be used for clinical trials only. Too bad.

I couldn't find other reviews focused mainly on health tracking, I agree that the quantified scientist produced results that should be validated by other sources.

Thanks for the article link, yes, these peer-reviewed papers are much more reliable and some aspects do not agree at all with the quantified scientis's data, like the sleep-stages accuracy of apple watches (classified as non-reliable). Also, the paper tends to suggest that sleep-stages tracking in watches is probably always more or less crappy.

I was amazed by the total accuracy of the whoop strap at HRV. Mike Lustgarten must have known it beforehand, LOL

 

[Ron Put]

15 minutes ago, mccoy said:

I was amazed by the total accuracy of the whoop strap at HRV

Hah, straps are accurate, more so than wrist-based opticals. But, I find it a pain to strap it for every walk, and after comparing it to the Garmin Venu 2+ and found it tracking mostly within literally 2-3 beats, even at higher rates. like 170+, I've kind of abandoned the strap.

HRV fluctuates a lot, so I wish that the V2+ automatically measured it periodically (when at rest). But it's also close enough for me compared to the strap.what your experience is with it.

The Inspire 3 gets good reviews, keep us posted on your experience with it.

[mccoy]

Inspire3 pros experienced so far:

1. Very unobtrusive, minimalist, easy to wear while working out, long battery life duration, so it is good for those who mainly need to see the time and to track health parameters and other basic parameters like steps number and so on and forget about it when not needed.
2. Interesting sleep tracking and related parameters, time awake and sleeping are sufficiently accurate, sleep stages are probably only a very rough estimate, there are other good features like a graph of the nightly trend of HR, with % above and below the resting HR, taken as reference. My resting HR was taken as 60 bpm (sufficiently accurate), whereas the lowest was 53 tonight, with 70% of the time below resting HR. many of these graphs make up a useful sleep analysis.

Cons experienced so far:

1. The small dial makes it difficult at times to see the smaller numbers, like bpm, although you can choose the dial face which minimizes this issue (there are not so many though available).
2. HRV measurements turned out to be inaccurate so far, the algorithm underestimated HRV for all the 3 nights it was measured. Tonight it indicated 29 ms, whereas the morning measurement I took with the Polar H10 strap was 53 ms. This issue has been discussed in users fora.

[Ron Put]

I just did another quick search.

I was close on the Dream 2's sleep detection accuracy, which according to a study led by the company provides 83.5 accuracy compared to polysomnography measurements. So, about 16.5% error for sleep detection. To me this is the by far more significant measurement and the Dreem 2 doesn't seem to perform better than some of the wrist-based trackers in the study posted earlier.

And here is another study comparing 5 different trackers, including Dream 2 and the Fitbit Versa 3:

Comparing consumer grade sleep trackers for research purposes: A field study