How do we know which pannier is best for the job? Because we were ruthless in our testing of these ten different models. Each pannier was subject to a series of tests and trials that were designed to push the limits of their performance. They endured a month of extensive off-road (there were no paved surfaces for 60 miles) testing in the Wrangell Mountains of Alaska to prove their salt. We invite you to read through our categorical breakdown of each test below.
Each pannier is unique in how it attaches to your rack. Even if they are utilizing similar mechanics, each bag has its own nuances that can add to or take away from overall sturdiness and usability. Firstly, we would test the ease of mounting and dismounting for each pannier. After removing it from the box, we timed how long it took to get each pannier securely fastened to a bike frame—including the time it took to assemble or tweak hardware before use. Naturally, we gave lesser scores to models who had a steep learning curve or weren't ready for use out-of-the-box.
Secondly, we tested how quickly we could secure it to a different rack. This test was necessary to ascertain whether or not the pannier was compatible with a wide variety of racks. Some models, like the Ortlieb Back-Roller Classic, require additional hardware to accommodate racks of different sizes while others, like the Timbuk2 Tandem, are universally sized. We gave higher scores to panniers that work on various rigs without fussing to change mounting hardware.
Lastly, we tested how secure each bags mounting system was once attached to the bike. This was accomplished by securing the pannier with a full load and then riding down a steep, bumpy decline at full speed. As the bike bounced over rocks and ruts, we listened for vibrations and rattles and checked to see that the pannier remained attached. Once the test was complete, we inspected the mounting hardware to see if there was any shifting, jarring, or detachment.
This metric was a bit more subjective than the rest. While storage ability is a primary concern when selecting a pannier, we had different expectations for the various categories of touring, commuting, and townie panniers. Each pannier was subject to the same storage tests, though we adjusted the parameters slightly to reflect each model's intended use.
The most critical storage test was measuring capacity or volume. We accomplished this by selecting an assortment of category-specific gear to fill each pannier to its functional limits. We took note of how much total gear we could store, as well as how easy it was to pack and unpack each model; higher scores didn't necessarily go to higher-volume bags. In addition to total capacity, we placed value on how easily the panniers could be loaded up to the brim.
We were also sure to test each panniers load security. To achieve this, we monitored whether or not gear had shifted or jostled after having ridden on bumpy terrain. When possible, we utilized compression straps or load-securing implements to test whether these features were useful in keeping gear secure. Furthermore, we paid close attention to how well closures held during these rough rides.
If you're anything like us, you take good care of your gear but inevitably wind up being hard on it after repeated use. Considering this, we thought that durability should be a primary consideration when selecting a pannier, so you know it's willing to go the distance time and time again. We conducted our tests in Southeast Alaska where the conditions are extremely unforgiving. There are no paved roads or sidewalks and very few flat surfaces in general. Life on a glacial moraine is very tough on equipment, and there were no exceptions for the panniers we tested. We routinely exposed each model to thick undergrowth, coarse rocks, and dusty silt that is capable of working into the smallest crannies.
To keep tabs on each panniers overall durability, we monitored how well each bag would stand up to the harsh conditions of life on the frontier. We kept our eyes peeled for not just cosmetic scuffs but also critical failures in hardware and materials that would be detrimental to the panniers performance.
The thin but critical margin between waterproof and merely water resistant demands your attention when selecting the proper bike bag. Wouldn't you like to know that your gear, possibly electronics, is more than just part-way protected from road spray or downpours? Furthermore, it's good to know it will stand up to other elements like the beating sun, and muddy trails. To more wholly capture the spectrum of elemental-security, we chose to test for comprehensive weatherproofing and not just waterproofing. To test for resilience in wet conditions, we had a two-tier waterproofing test. First, we packed each pannier with newspaper wads—in both main compartments and extra pockets—and sealed them up using all the available water protection features. After they were secure, we left them outside overnight in the rain. In the morning, we checked to see how much moisture, if any, had been let into the bags by checking the relative dampness of the newspaper inside.
The second phase of this waterproofing test was more intensive. We sent each bag for a swim in the creek while attached to a rope (also a good measure of durability). Afterward, we fully submerged each pannier and held them underwater for a total of 15 seconds. Similar to the rain test, we used wads of newspaper to monitor how much moisture was let into each bag.
In addition to simulated conditions, we also used the panniers during full-on sideways rain in the Wrangell Mountains. During these rides, we encountered a good deal of mud and slop, in addition to rain. Beyond performance in wet conditions, we also took note of the effects on performance during dry, dusty days on the trail. How easily could fine-particle sand or trail debris enter pockets and openings? We paid close attention to how well each bag operated once it had gotten thoroughly soaked and covered in silt. Specifically, we checked to see whether or not mud had any effect on the function of closures, straps, and mounting hardware.
Ease of Use
This metric was the least weighted in our testing plan, but that doesn't mean it wasn't important. To us, Ease of Use translates to ergonomics, user-friendliness, and overall satisfaction with the product. We also used this metric to provide scores for additional features that didn't fall into the other four categorical distinctions. Details like reflective materials, shoulder straps, hidden pockets, or other features that either add to or take away from the user experience were factored into Ease of Use.