When we review gear at OutdoorGearLab, we do everything we can to be as informed as possible about the lay of the field, so we're abreast of all the new trends and developments. We spend forever researching so we can understand how things work and how they're performing for users, but that's only part of the process. The rest is actually getting our hands on this stuff and trying to break it. Testing to see whether claims can be substantiated. Testing to see if we'd recommend this stuff to our closest friends. Are there pro tips we would vouchsafe to someone (you all, in this case)?
To get there, we break down each review along a handful of performance measures and work hard to iron out all the subjectivity we can. We're not exactly clinical researchers running triple-blind longitudinal studies, but we're still diligent, disciplined and we can pull from our expertise to deliver fair, thorough analysis with a few quirky tips here and there.
We'll look at each of the measures we use in our reviews for the remainder of this page. We briefly describe how we use each of our performance measures to test and analyze each bike trainer.
Connectivity and Power Accuracy
When testing connectivity we make every effort to test the smart trainers in the review with both IOS and Android smartphones, as well as Mac and Windows-based computers. In this process, we assess the ease-of-use of the in-house applications like Wahoo Fitness and the Tacx training app, but we also test all of the trainers on two of the most popular training programs, Zwift and Trainer Road. both on Mac and PC platforms. Most of the trainers use both ANT+FEC and Bluetooth Smart communication protocols so we make sure to break out the old' ANT+ dongle to test both communication protocols.
Lots of users prefer to or prefer to use their phones or tablets so we're sure to check things out on that level too. Bluetooth connections are tested using IOS and Android smartphones as well as Mac computers with native Bluetooth Smart capabilities. The process can be complex and every effort is made to work with customer support from the various brands to ensure that connection issues we note are indeed valid and not a result of out-of-date firmware or user-error.
We compare power accuracy using a Garmin pedal-based power meter as a baseline in conjunction with a Garmin head unit to compare power from the smart trainers with the data coming from the Garmin unit. We disable smoothing on the readings and look at the most frequent measurement interval to get the most accurate readings possible. We do a static power comparison at 200 watts on the pedals using the native applications for power readings, and we also make comparisons over the course of training sessions over Zwift. These longer comparisons reveal the power measurement drift we experience on the tire drive smart trainers and aide in illustrating the advantages of direct drive trainers for accurate power measurement.
This is an area that we feel deserves a lot of attention. The heaviness of a trainer's flywheel determines its inertia and significantly contributes to its momentum and therefore the smoothness of the riding experience. Out on the road, resistance is primarily determined by our body weight pressing against the road and its vertical movements as well as the wind and any other oppressive elements weighing down our morale. Trainers have the tough job of mimicking that while remaining (mostly) stationary. That's why the flywheel is such an important component - really, the most important performance component. That's because there's a sweet balance somewhere between resistance and momentum that a rider looks for and needs to feel for a trainer to be a worthwhile home endeavor instead of just going to a spin class down at the gym.
The balance is what most riders consider good road feel. Easy on downhills, hard from a standstill. If you coast for a bit, you should lose some steam and need to crank a bit to get back up to speed, just like you would out on the road. For trainers to get it right, they typically equip heavy flywheels to ensure there's enough momentum once you get up to speed, and a brake of some sort to provide simulated resistance.
When you ride on the road there is inherent inertia at play as the wheels rotate. On a trainer, much of that is lost, because instead of pushing against rolling resistance, wind, and gravity the resistance is created by some sort of brake or fluid medium. Thus there is nothing to carry the pedal stroke through the dead spot Quite a few of the advanced trainers do a great job of roughly replicating the road, but those at the entry- and mid-level can be a bit less responsive and perhaps inattentive as you go through your ride and change resistance requirements. The basic (meaning not smart) trainers are almost invariably susceptible to this dead spot issue when you get into the high cadence workouts and aren't actively increasing output. That's not to say they're all terrible, of course, just that you'll notice it happening more on most of them as compared to smart trainers, which will adjust power to match output.
We test the going larger-is-better theory by doing a roll-out test where the bikes are pedaled up to 20 MPH in the lowest resistance setting possible and then allow it to spin to a stop. The time it takes them to spin to a stop is recorded. Over the history of this review, we've found that the larger-is-better theory tends to hold up. Trainers that take the longest to come to a stop tend to have larger (or at least heavier) flywheels and better road feel.
As important as flywheel mass is, it's not the entire story, particularly when it comes to smart trainers. There's dynamic feedback happening with a lot of smart trainers, so they're responding to both simulated terrain and your output (depending on your settings). That partially altered the theory, but even so, the smart trainers still tend to have pretty heavy flywheels.
We also spend a lot of time assessing how the trainers react to changes in grade or resistance increases in ERG mode. This is all detailed in the individual product reviews.
In addition, we measured the noise in decibels that each trainer put out at 20 mph at about an 85 cadence with the decibel meter reading measured from the center of the handlebars.
When scoring the products on their design, we assess the construction of the product and materials used. We make note of what we like, what we don't like, and generally look for weak points or design areas that could lead to problems with performance or longevity. We examine compatibility with different types of bikes and a variety of rear hub and axle standards.
Over the years, many bike trainer manufacturers have done a lot to make sure their trainers are compatible with just about any bike out there. Sometimes it's by adding close-to-universal hubs, but it's typically by throwing in a few washers and adapters to match the right axle width and hub diameter.
We also consider the accessories that are included, and whether or not they added to the functionality of the product or were just sales fluff. For quite a while, lots of direct drive trainers were coming with pre-installed cassettes. That was fairly convenient if you had a drivetrain that matched, but if you didn't run that gear, then it 1.) Added extra steps to your setup, and 2.) meant you were essentially paying for a useless cassette to pin up on your wall in some sort of abstract utilitarian art piece.
On a related note is the compatibility. Shimano and SRAM comfortably rule the groupset market, but a few of us fancy folks still run Campagnolo, which has its own design standard that makes it less compatible with the other two. A lot of times that means that you'll need to run different cassettes on your direct drive trainer, requiring a different hub body, which you'll need to buy - sometimes different hub bodies even between different versions of the same line…which can be infuriating if you don't thoroughly research the specs and notice that minor detail.
So we make those mistakes for you and tell you about them so you don't have to sit there in your bib shorts, carbo-loaded, fan on, music rocking, dongle broadcasting, all cued up to ride as soon as you get this damn hub on…only to find that your hub is 2mm off because you ordered the widely available version whose description seems to say it's compatible with your brand new trainer. We grade on that stuff in proportion to the degree of lividness in the case of experiencing the above completely hypothetical scenario that in no way occurred while testing one of the high-end trainers.
Product research is a big consideration in this metric. If a product has negative reviews, we look at those spots - if they had tons of negative reviews, we make note of it and do even more research and testing in those areas to see if we can bring those issues out and understand why they're happening if we're seeing them too.
Our setup assessment evaluated all of the trainers on how difficult we found day to day use as well as the initial setup before the first ride. We evaluated the need for tools to assemble the trainer, as well as how much effort was involved in folding-unfolding and attaching a bike to the trainer. Another area that quickly became front and center was calibration before use. We found that the tire drive smart trainers required a significant amount of time to perform calibration prior to each use, which was ultimately a detriment to their score. We set up and took down all of these trainers more times in our testing period than most people will in a lifetime. We became intimately familiar with the nuances of all of the trainers and our findings are outlined in this rating metric.
Over the course of testing, we moved these trainers around a lot. Transferring between different test locations and swapping trainers amongst our testers. We discussed in detail the packing, carrying, and storage issues we encountered over months of testing. We also used all of the trainers without electrical connections with varying results. Some trainers are designed to be self-powering, so you can still get power readings, for example, without being plugged in, but you lose that function after you stop pedaling for a few seconds. Tire drive trainers typically aren't powered, so that's not a real concern there. In addition, all of the trainers were weighed and measured in house to give you an idea of the size in both the folded and unfolded space considerations.