The world's most in-depth and scientific reviews of gear

How We Tested Bike Trainers

Tuesday January 12, 2021

When we review gear, we do everything we can to be as informed as possible about the lay of the land, so we make a big effort to remain abreast of all the new trends and developments. We spend forever researching so we can understand how things work and how they're performing for users, but that's only part of the process. The rest is actually getting our hands on this stuff and trying to break it. Testing to see whether claims can be substantiated. Testing to see if we'd recommend this stuff to our closest friends. To see if there are pro tips we would vouchsafe to someone (you all, in this case). We research the hell out of everything we can find and use that insight to target, question, and revise our thoughts.

It all starts with the unboxing.
It all starts with the unboxing.
Photo: Ryan Baham

To get there, we break down each review along a handful of performance measures and work hard to iron out all the subjectivity we can. We're not exactly clinical researchers running triple-blind longitudinal studies or running our operations out of a squash court at the University of Chicago in the '40s, but we're still diligent, disciplined, and we can pull from our expertise to deliver fair, thorough analysis with a few quirky tips here and there.

The rest of this page will be spent looking over our performance measures and how we test each bike trainer.

Connectivity and Power Accuracy


Connectivity
When testing connectivity we make every effort to test the smart trainers in the review with both IOS and Android smartphones, as well as Mac and Windows based computers. In this process we assess the ease-of-use of the in-house applications like Wahoo Fitness and the Tacx training app, but we also test all of the trainers on two of the most popular training programs, Zwift and Trainer Road. both on Mac and PC platforms. Most of the trainers use both ANT+FEC and Bluetooth Smart communication protocols so we make sure to break out the ol' ANT+ dongle to test both communication protocols.
Chances are, you'll need an ANT+ dongle to get full functionality...
Chances are, you'll need an ANT+ dongle to get full functionality. Make sure you get one and keep it in a safe place near your trainer!
Photo: Tara Kharrazian

Lots of users prefer to use their phones or tablets so we're sure to check things out on that level too. Bluetooth connections are tested using IOS and Android smartphones as well as Mac computers with native Bluetooth Smart capabilities. The process can be complex and every effort is made to work with customer support from the various brands to ensure that connection issues we note are indeed valid and not a result of out-of-date firmware or doltish operators (us).

We tested the connectivity of all of the trainers with applications...
We tested the connectivity of all of the trainers with applications like Zwift, and used both Mac and Windows computers
Photo: Curtis Smith

Power Accuracy

We compare power accuracy using a Garmin pedal-based power meter as a baseline in conjunction with a Garmin head unit to compare power from the smart trainers with the data coming from the Garmin unit. We disable smoothing on the readings and look at the most frequent measurement interval to get the most accurate readings possible. We do a static power comparison at 200 watts on the pedals using the native applications for power readings, and we also make comparisons over the course of training sessions over Zwift checking for spikes and drops. These longer comparisons reveal the power measurement drift we experience on the tire drive smart trainers and aide in illustrating the advantages of direct drive trainers for accurate power measurement.

We looked at power accuracy over short, intense rides with lots of...
We looked at power accuracy over short, intense rides with lots of random efforts to look for variation as well as longer, slower, steady rides to look for drift.
Photo: Tara Kharrazian

Road Feel


Many of us feel at least a little reticence at the idea of sitting on an indoor trainer for hours for the same reason it's harrowing to even hear the word "treadmill." It's important for a lot of us to be able to remove ourselves from the inherent monotony , which includes the unnatural feel you get when riding stationary bikes at the gym or on lower-end trainers. If you're going to be inside, nothing beats your bike on a good trainer with a good training program and kick ass tunes. For that reason, a lot of focus is directed to this measure. If it doesn't feel right, you're not going to want to be on the bike and the point is to want to be on the bike - or at least not have any good excuses for taking another rest day you don't need…

The heaviness of a trainer's flywheel determines its inertia and significantly contributes to its momentum and therefore the smoothness of the riding experience. Out on the road, resistance is primarily determined by our body weight pressing the tires against the road and its vertical movements as well as the wind and any other oppressive elements weighing down our morale. Trainers have the tough job of mimicking those dynamics while remaining (mostly) stationary. That's why the flywheel is such an important component - really, the most important performance component. There's a sweet balance somewhere between resistance and momentum that a rider looks for and needs to feel for a trainer to be a worthwhile home endeavor instead of just going to a spin class down at the gym.

The balance is what most riders consider good road feel. Easy on downhills, hard from a standstill. If you coast for a bit, you should lose some steam and need to crank a bit to get back up to speed, just like you would out on the road. For trainers to get it right, they typically equip heavy flywheels to ensure there's enough momentum at cruising speed, and a brake of some sort to provide simulated resistance.

Out on the road you're used to feeling all of the inertia at play as the wheels rotate. On a trainer, much of that is lost. Instead of pushing against rolling resistance, wind, and gravity, the resistance is created by some sort of brake or fluid medium. The lower-end trainers are especially vulnerable to this, often resulting in dead spots in the pedal stroke and quickly spinning out at higher speeds. They're super convenience for getting in perfunctory or maintenance spins, but serious workouts with lots of hard climbing or sprints can be more difficult to maintain.

As you move up the range through to smart trainers the gap between indoor and outdoor narrows as they start to use bigger flywheels and mechanical or magnetic brakes to simulate resistance and road feel. If you're using something like Zwift to simulate a real mountain course, the high-end trainers can actually simulate freewheeling down one side of a climb and hitting the resistance wall on the other side as you go from a -10% downhill through the trough and back up at 5 or 15%. This transition is actually where we spend a lot of time comparing smart trainers to each other.

Riding in SIM mode in Zwift gave us a good way to test road feel, by...
Riding in SIM mode in Zwift gave us a good way to test road feel, by assesing resistance changes as the slope increased or decreased.
Photo: Curtis Smith

For all of the smart trainers we test, we take them out on virtual routes or workouts with lots of extreme changes in elevation or resistance to get a good feel for how well they respond. We also make sure to put them them into ERG mode to see how quickly or smoothly they transition between power output requirements and actual outputs. If you're not familiar with ERG, it's shorthand for ergasia, meaning something like work or gains from work. You'll understand why the term's chosen if you ever use ERG mode. It's one of the best ways to do hard structured training and if the performance is crap here it can throw your training session. Certainly it throws your psyche and demoralizes you.

We test the bigger-is-better theory on flywheels by doing a roll-out test where the bikes are pedaled up to 20 mph in the lowest resistance setting possible and then allow it to spin to a stop. The time it takes them to spin to a stop is recorded. Over the history of this review, we've found that the bigger-is-better theory tends to hold up. Trainers that take the longest to come to a stop tend to have larger (or at least heavier) flywheels and better road feel.

As important as flywheel mass is, it's not the entire story, particularly when it comes to smart trainers. There's a dynamic feedback happening with a lot of smart trainers, so they're responding to both simulated terrain and your output (depending on your settings). While that is a slight divergence from the bigger-is-better theory, it still generally supports it because the best smart trainers also tended to have big, heavy flywheels to do a lot of the heavy lifting.

Our decibel meter is seen here at the rear of the bike, but we took...
Our decibel meter is seen here at the rear of the bike, but we took our measurements from the handlebars to control for measurement error/variation.
Photo: Crystal Huffman

In addition, we measured the noise in decibels that each trainer put out at 20 mph at about an 85 cadence with the decibel meter reading measured from the center of the handlebars.

Not all trainers perform the same in ERG mode. Constantly fighting...
Not all trainers perform the same in ERG mode. Constantly fighting your machine to maintain cadence and resistance is a real pain for ERG training sessions.
Photo: Ryan Baham

Design


When scoring the products on design we assess the construction of the product and materials used we do a substantial amount of resource, becoming amateur materials scientists and engineers and well as test dummies, putting in hours trying to make stuff break or work as advertised. We examine compatibility with different types of bikes and a variety of rear hub and axle standards. We make note of what we like, what we don't like, and generally look for weak points or design areas that could lead to problems with performance or longevity. Of course we also find claims from other users and do what we can to dispute or validate them.

Over the years, many bike trainer manufacturers have done a lot to make sure their trainers are compatible with just about any bike out there. Sometimes it's by adding close-to-universal hubs, but it's typically by throwing in a few washers and adapters to match the right axle width and hub diameter.

We also consider the accessories that are included, and whether or not they added to the functionality of the product or were just sales fluff. For quite a while, lots of direct drive trainers were coming with pre-installed cassettes. That was fairly convenient if you had a drivetrain that matched, but if you didn't run that gear, then it 1.) Added extra steps to your setup (any good process analyst would find this to be repugnant), and 2.) meant you were essentially paying for a useless cassette to pin up on your wall in some sort of abstract utilitarian art piece.

On a related note is the compatibility. Shimano and SRAM comfortably rule the groupset market, but a few of us fancy folks still run Campagnolo, which has its own design standard that makes it less compatible with the other two. A lot of times that means that you'll need to run different cassettes on your direct drive trainer, requiring a different hub body, which you'll need to buy - sometimes different hub bodies even between different versions of the same line…which can be infuriating if you don't thoroughly research the specs and notice that minor detail.
Discovering that your special-ordered, 2-week-delivery Campy freehub...
Discovering that your special-ordered, 2-week-delivery Campy freehub doesn't bloody fit is not the most exciting thing on the planet. Make sure to research parts and compatibility thoroughly.
Photo: Ryan Baham

So we make those mistakes for you and tell you about them so you don't have to sit there in your bib shorts, carbo-loaded, fan on, music rocking, dongle broadcasting, all cued up to ride as soon as you get this damn hub on…only to find that your hub is 2mm off because you ordered the widely available version whose description seems to say it's compatible with your brand new trainer. We grade on that stuff in proportion to the degree of lividness experienced in the case of experiencing the above completely hypothetical scenario that in no way occurred while testing one of the high-end trainers.

Product research is a big consideration in this metric. If a product has negative reviews, we look at those spots - if they had tons of negative reviews, we make note of it and do even more research and testing in those areas to see if we can bring those issues out and understand why they're happening if we're seeing them too.
We're not above enjoying the frills. The Tacx Neo 2T has cool lights...
We're not above enjoying the frills. The Tacx Neo 2T has cool lights for those of us who like riding in the dark.
Photo: Ryan Baham

Setup


Our setup assessment evaluated all of the trainers on how difficult we found day to day use as well as the initial setup before the first ride. We evaluated the need for tools to assemble the trainer, as well as how much effort was involved in folding-unfolding and attaching a bike to the trainer. Another area that quickly became front and center was calibration before use. We found that the tire drive smart trainers required a significant amount of time to perform calibration prior to each use, which was ultimately a detriment to their score. We set up and took down all of these trainers more times in our testing period than most people will in a lifetime. We became intimately familiar with the nuances of all of the trainers and our findings are outlined in this rating metric.

Even the simplest trainers will require a bit of tweaking here and...
Even the simplest trainers will require a bit of tweaking here and there. That's especially true of the tire-drive trainers, like the Blackburn Tech Mag 1.
Photo: Ryan Baham

Portability


Over the course of testing, we moved these trainers around a lot! Transferring between different test locations and swapping trainers amongst our testers. We discussed in detail the packing, carrying, and storage issues we encountered over months of testing. We also used all of the trainers without electrical connections with varying results. Some trainers are designed to be self-powering, so you can still get power readings, for example, without being plugged in, but you lose that function after you stop pedaling for a few seconds. Tire drive trainers typically aren't powered, so that's not a real concern there. In addition, all of the trainers were weighed and measured in house to give you an idea of the size in both the folded and unfolded space considerations.

Try doing this with a big direct-drive trainer! The 15 pound...
Try doing this with a big direct-drive trainer! The 15 pound Blackburn Tech Mag 1 is an ideal traveling companion for road-tripping with your bike and taking out to races.
Photo: Ryan Baham