Our testing broke down into three phases:
First, we talked to all our friends and drew on our collective experience to see what were the best pads currently available. We did an informal poll to decide what features were most important to people and what to look for and scrutinize in our tests.
Next, we acquired the crash pads. Some we already had. We wanted to buy the rest, but after buying a couple, we realized this would be way too expensive. So we asked some companies to donate pads and got a few that way.
For about two months we used all the pads as much as possible in real-world bouldering.
Finally, we lined all the pads up for a few days of rigorous testing. This is where we saw how the pads compared to each other. We took the same fall from the same height and put the same "hidden" rock under the same spot on each pad. We debated what features were most important and why. We made our own judgments but also asked for feedback from friends/testers. We used feedback from other testers to point us toward features we might not have noticed or might not have put enough importance on. But ultimately, we made our own judgments.