The world's most in-depth and scientific reviews of gear

How We Tested Bike Panniers

Thursday December 12, 2019
The Vario stood up to miles and miles of trail use. We think it's the most robust offering in the backpack pannier lineup.
The Vario stood up to miles and miles of trail use. We think it's the most robust offering in the backpack pannier lineup.

How do we know which pannier is best for the job? Because we were ruthless in our testing of these 11 different models. Each pannier was subject to a series of tests and trials that were designed to push the limits of their performance. They endured day-to-day use on the streets and bikes paths around Santa Barbara, and a month of extensive off-road (there were no paved surfaces for 60 miles) testing in the Wrangell Mountains of Alaska. We invite you to read through our categorical breakdown of each test below.

Related: Buying Advice for Bike Panniers

Mounting System


Each pannier is unique in how it attaches to your bike rack. Even if they are utilizing similar mounting systems, each bag has its own nuances that can add to or take away from overall sturdiness and usability. First, we tested the ease of mounting and dismounting each pannier. After removing it from the box, we timed how long it took to get each pannier securely fastened to a bike frame, including the time it took to assemble or adjust hardware before use. Naturally, we gave lesser scores to models who had a steep learning curve or weren't ready for use right out-of-the-box.

The Ortlieb Gravel Pack is one of the easiest models to secure to the rack.
The Ortlieb Gravel Pack is one of the easiest models to secure to the rack.

Next, we tested how quickly we could secure each pannier to a different rack to determine if certain models have any compatibility issues with certain types of racks. Some models, like the Ortlieb Back Roller Classic, come with additional hardware pieces to accommodate racks of different sizes, while others, like the Timbuk2 Tandem, are universally sized. We gave higher scores to panniers that work on various rigs without fussing to change mounting hardware.

The Thule Shield's unique mounting systems with magnets attached to embedded metal plates within the side of each bag.
The Thule Shield's unique mounting systems with magnets attached to embedded metal plates within the side of each bag.

Finally, we tested how secure each bag felt once attached to the bike. This was accomplished by securing the pannier with a full load and then riding down a steep, bumpy decline at full speed. As the bike bounced over rocks and ruts, we listened for vibrations and rattles and checked to see that the pannier remained attached. Once the test was complete, we inspected the mounting hardware to see if there was any shifting, jarring, or detachment.

Storage


This metric is a bit more subjective than the rest. While storage ability is a primary concern when selecting a pannier, we had different expectations for the various categories of touring, commuting, and urban panniers. Each pannier was subject to the same storage tests, though we adjusted the parameters slightly to reflect each model's intended use.

The most critical storage test was measuring overall capacity or volume. We accomplished this by selecting an assortment of category-specific gear to fill each pannier to its functional limits. We took note of how much total gear we could store, as well as how easy it was to pack and unpack each model; higher scores didn't necessarily go to higher-volume bags. In addition to total capacity, we placed value on how easily the panniers could be loaded up to the brim.

The Suffolk has one of the largest single-bag capacities of any commuting pannier in our lineup  and was able to fit all the gear pictured above with room to spare.
The Suffolk has one of the largest single-bag capacities of any commuting pannier in our lineup, and was able to fit all the gear pictured above with room to spare.

We were also sure to test each pannier's load security. To achieve this, we monitored whether or not gear had shifted or jostled after having ridden on bumpy terrain. When possible, we utilized compression straps or load-securing implements to test whether these features were useful in keeping gear secure. Furthermore, we paid close attention to how well closures held during these rough rides.

Durability


If you're anything like us, you take good care of your gear but inevitably wind up being hard on it after repeated use. Considering this, we thought that durability should be a primary consideration when selecting a pannier, so you know it's willing to go the distance time and time again. We conducted much of our testing in Southeast Alaska, where the conditions are extremely unforgiving. There are no paved roads or sidewalks and very few flat surfaces in general. Life on a glacial moraine is very tough on equipment, and there were no exceptions for the panniers we tested. We routinely exposed each model to thick undergrowth, coarse rocks, and dusty silt that is capable of working into the smallest crannies.

With sturdy webbing handles and boxy storage  the Market Pannier was even capable of hauling power tools  decking screws  and scooby-snacks. Though  the poorly distributed weight was tough to ride with.
With sturdy webbing handles and boxy storage, the Market Pannier was even capable of hauling power tools, decking screws, and scooby-snacks. Though, the poorly distributed weight was tough to ride with.

To keep tabs on each pannier's overall durability, we monitored how well each bag would stand up to the harsh conditions of life on the frontier. We kept our eyes peeled for not just cosmetic scuffs but also critical failures in hardware and materials that would be detrimental to the panniers' performance.

Weatherproofing


The thin but critical margin between waterproof and merely water-resistant demands your attention when selecting the proper bike bag. Wouldn't you like to know that your gear, possibly electronics, is more than just part-way protected from road spray or downpours? Furthermore, it's good to know it will stand up to other elements like the beating sun, and muddy trails. To more wholly capture the spectrum of elemental-security, we chose to test for comprehensive weatherproofing and not just waterproofing. To test for resilience in wet conditions, we had a two-tier waterproofing test. First, we packed each pannier with newspaper wads—in both main compartments and extra pockets—and sealed them up using all the available water protection features. After they were secure, we left them outside overnight in the rain. In the morning, we checked to see how much moisture, if any, had been let into the bags by checking the relative dampness of the newspaper inside.

We were so confident in the Ortlieb's waterproofing that we sent it for a ride down the creek.
We were so confident in the Ortlieb's waterproofing that we sent it for a ride down the creek.

The second phase of this waterproofing test was more intensive. We sent each bag for a swim in the creek while attached to a rope (also a good measure of durability). Afterward, we fully submerged each pannier and held them underwater for a total of 15 seconds. Similar to the rain test; we used wads of newspaper to monitor how much moisture was let into each bag. In addition to simulated conditions, we also used the panniers during full-on sideways rain in the Wrangell Mountains. During these rides, we encountered a good deal of mud and slop, in addition to rain. Beyond performance in wet conditions, we also took note of the effects on performance during dry, dusty days on the trail. How easily could fine-particle sand or trail debris enter pockets and openings? We paid close attention to how well each bag operated once it had gotten thoroughly soaked and covered in silt. Specifically, we checked to see whether or not mud had any effect on the function of closures, straps, and mounting hardware.

With superior waterproofing and storage volume  the Back-Roller Classic is a great choice for longer tours.
With superior waterproofing and storage volume, the Back-Roller Classic is a great choice for longer tours.

Ease of Use


This metric was the least weighted in our testing plan, but that doesn't mean it wasn't important. To us, ease of use translates to ergonomics, user-friendliness, and overall satisfaction with the product. We also used this metric to provide scores for additional features that didn't fall into the other four categorical distinctions. Details like reflective materials, shoulder straps, hidden pockets, or other features that either add to or take away from the user experience were factored into ease of use.

The "Y"-shaped cinch strap of the Shield Pannier is easy to use and very effective.
The "Y"-shaped cinch strap of the Shield Pannier is easy to use and very effective.