When we review gear, we do everything we can to be as informed as possible about the lay of the land, so we make a big effort to remain abreast of all the new trends and developments. We spend forever researching so we can understand how things work and how they're performing for users, but that's only part of the process. The rest is actually getting our hands on this stuff and trying to break it. Testing to see whether claims can be substantiated. To see if there are pro tips we would vouchsafe to someone (you all, in this case). We research everything we can find and use that insight to target, question, and revise our thoughts.
To get there, we break down each review along a handful of performance measures and work hard to iron out all the subjectivity we can. We're not exactly clinical researchers running triple-blind longitudinal studies or running our operations out of a squash court at the University of Chicago in the '40s, but we're still diligent, and disciplined, and we can pull from our expertise to deliver fair, thorough analysis with a few quirky tips here and there.
The rest of this page will be spent looking over our performance measures and how we test each bike trainer.
Connectivity and Power Accuracy
When testing connectivity, we try to test the smart trainers in the review with both IOS and Android smartphones and Mac and Windows-based computers. In this process, we assess the ease of use of the in-house applications like Wahoo Fitness and the TacX training app, but we also test all of the trainers on two of the most popular training programs, Zwift and Trainer Road, both on Mac and PC platforms. Most trainers use both ANT+FEC and Bluetooth Smart communication protocols, so we make sure to break out the old ANT+ dongle to test both.
Many users prefer to use their phones or tablets, so we check things out on that level too. Bluetooth connections are tested using IOS and Android smartphones and Mac computers with native Bluetooth Smart capabilities. The process can be complex, and every effort is made to work with customer support from the various brands to ensure that connection issues we note are indeed valid and not a result of out-of-date firmware or doltish operators (us).
We compare power accuracy using pedal-based power meters (Garmin and Favero) as a baseline in conjunction with a Garmin head unit to compare power from the smart trainers with the data from the Garmin unit. We disable smoothing on the readings and look at the most frequent measurement interval to get the most accurate readings possible. We do a static power comparison at 200 watts on the pedals using the native applications for power readings. We also make comparisons throughout training sessions over Zwift checking for spikes and drops. These longer comparisons reveal the power measurement drift we experience on the tire drive smart trainers and illustrate the advantages of direct drive trainers for accurate power measurement.
Many of us feel at least a little reticence at the idea of sitting on an indoor trainer for hours for the same reason it's harrowing even to hear the word "treadmill." Many of us must remove ourselves from the inherent monotony, including the unnatural feel you get when riding stationary bikes at the gym or on lower-end trainers. If you're going to be inside, nothing beats your bike on a good trainer with a good training program and kick-ass tunes. For that reason, we direct a lot of focus on this measure. If it doesn't feel right, you won't want to be on the bike, and the point is to want to be on the bike - or at least not have any good excuses for taking another rest day you don't need…
The heaviness of a trainer's flywheel determines its inertia and significantly contributes to its momentum and, therefore, the smoothness of the riding experience. On the road, resistance is primarily determined by our body weight pressing the tires against the road and its vertical movements, as well as the wind and other oppressive elements weighing down our morale. Trainers have the tough job of mimicking those dynamics while remaining (mostly) stationary. That's why the flywheel is such an important component - the most important performance component. There's a sweet balance between resistance and momentum that a rider looks for and needs to feel for a trainer to be a worthwhile home endeavor instead of just going to a spin class down at the gym.
The balance is what most riders consider a good road feel. Easy on downhills, hard from a standstill. If you coast for a bit, you should lose some steam and need to crank a bit to get back up to speed, just like you would out on the road. To get it right, trainers typically equip heavy flywheels to ensure enough momentum at cruising speed and a brake to provide simulated resistance.
Out on the road, you're used to feeling all of the inertia at play as the wheels rotate. On a trainer, much of that is lost. Instead of pushing against rolling resistance, wind, and gravity, the resistance is created by some brake or fluid medium. The lower-end trainers are especially vulnerable to this, often resulting in dead spots in the pedal stroke and quickly spinning out at higher speeds. They're super convenient for getting in perfunctory or maintenance spins, but serious workouts with lots of hard climbing or sprints can be more difficult to maintain.
As you move up the range to smart trainers, the gap between indoor and outdoor narrows as they start to use bigger flywheels and mechanical or magnetic brakes to simulate resistance and road feel. Suppose you're using something like Zwift to simulate a real mountain course. In that case, the high-end trainers can simulate freewheeling down one side of a climb and hitting the resistance wall on the other side as you go from a -10% downhill through the trough and back up at 5 or 15%. This transition is actually where we spend a lot of time comparing smart trainers to each other.
For all of the smart trainers we test, we take them out on virtual routes or workouts with lots of extreme changes in elevation or resistance to get a good feel for how well they respond. We also put them into ERG mode to see how quickly or smoothly they transition between power output requirements and actual outputs. If you're unfamiliar with ERG, it's shorthand for ergasia, meaning something like work or gains from work. You'll understand why the term's chosen if you use ERG mode. It's one of the best ways to do hard-structured training; if the performance is poor here, it can throw your training session. Certainly, it throws your excitement and demoralizes you.
We test the bigger-is-better theory on flywheels by doing a roll-out test where the bikes are pedaled up to 20 miles per hour in the lowest resistance setting possible and then allowed to spin to a stop. The time it takes them to come to a stop is recorded. Over the history of this review, we've found that the bigger-is-better theory tends to hold up. Trainers that take the longest to come to a stop tend to have larger (or at least heavier) flywheels and a better road feel.
As important as flywheel mass is, it's only part of the story, particularly regarding smart trainers. Dynamic feedback is happening with many smart trainers, so they're responding to both simulated terrain and your output (depending on your settings). While that is a slight divergence from the bigger-is-better theory, it still generally supports it because the best smart trainers also tended to have big, heavy flywheels to do much of the heavy lifting. In addition, we measured the noise in decibels that each trainer put out at 20 mph at about an 85 cadence, with the decibel meter reading measured from the center of the handlebars.
When scoring the products on design, we assess the construction of the product and materials used. We do a substantial amount of research, becoming amateur materials scientists, engineers, and test dummies, putting in hours trying to make stuff break or work as advertised. We examine compatibility with different types of bikes and various rear hub and axle standards. We note what we like and don't like and generally look for weak points or design areas that could lead to problems with performance or longevity. Of course, we also find claims from other users and do what we can to dispute or validate them.
Over the years, many bike trainer manufacturers have done a lot to ensure their trainers are compatible with any bike. Sometimes it's by adding close-to-universal hubs, but it's typically by throwing in a few washers and adapters to match the right axle width and hub diameter.
We also consider the accessories included and whether or not they added to the product's functionality or were just sales fluff. Many direct drive trainers were coming with pre-installed cassettes for quite a while. That was pretty convenient if you had a drivetrain that matched, but if you didn't run that gear, then it 1) Added extra steps to your setup (any good process analyst would find this repugnant), and 2) Meant you were essentially paying for a useless cassette to pin up on your wall in some sort of abstract utilitarian art piece.
On a related note is the compatibility. Shimano and SRAM comfortably rule the groupset market, but some fancy folks still run Campagnolo, which has a design standard that makes it less compatible with the other two. A lot of times that means that you'll need to run different cassettes on your direct drive trainer, requiring a different hub body, which you'll need to buy - sometimes different hub bodies, even between different versions of the same line…which can be infuriating if you don't thoroughly research the specs and notice that minor detail.
So we make those mistakes for you and tell you about them so you don't have to sit there in your bib shorts, carbo-loaded, fan on, music rocking, dongle broadcasting, all cued up to ride as soon as you get this damn hub on…only to find that your hub is 2mm off because you ordered the widely available version whose description seems to say it's compatible with your brand new trainer. We grade that stuff in proportion to the degree of lividness experienced in the case of experiencing the above completely hypothetical scenario that never occurred while testing one of the high-end trainers.
Product research is a big consideration in this metric. If a product has negative reviews, we look at those spots - if they have tons of negative reviews, we make a note of it and do even more research and testing in those areas to see if we can bring those issues out and understand why they're happening if we're seeing them too.
Our setup assessment evaluated all of the trainers on how difficult we found day-to-day use and the initial setup before the first ride. We assessed the need for tools to assemble the trainer and how much effort was involved in folding-unfolding and attaching a bike to the trainer. Another area that quickly became front and center was calibration before use. We found that the tire drive smart trainers required a significant amount of time to perform calibration before each use, ultimately a detriment to their score. We set up and took down all of these trainers more times in our testing period than most people will in a lifetime. We became intimately familiar with the nuances of all of the trainers, and our findings are outlined in this rating metric.
Throughout testing, we moved these trainers around a lot: transferring between different test locations, swapping trainers among our testers, and putting them in closets or back into the box. We discussed the packing, carrying, and storage issues over months of testing. We also used all trainers without electrical connections, with varying results. Some trainers are self-powering, so you can still get power readings, for example, without being plugged in, but you lose that function after you stop pedaling for a few seconds. On the other hand, tire drive trainers typically aren't powered, so that's not a real concern there. In addition, all trainers were weighed and measured in-house to give you an idea of the size in both the folded and unfolded space considerations.