When I began my career as a magazine ski tester 20 years ago, the landscape of the ski market looked very different than it does today. I was fresh from the halls of Salomon, where I learned the ski test methodology then in vogue among the world’s leading ski manufacturers. (Salomon was conducting ski tests of competitors’ models three years before they would introduce a ski, part of a development culture that was at the top of its game, having introduced in the previous decade alpine boots and cross-country boot-binding systems that shook their respective sectors to their roots and soon captured obscene market shares.) When Snow Country magazine asked me to assemble an elite test team and concoct a methodology that would identify star products I felt well trained for the challenge.
As soon as the magazine switched course from patronizing pabulum meant not to offend (nor inform) anyone, to hard test results that ranked skis in their order of finish, skiers took notice and circulation soared. Skiers were not the only ones doing the noticing. Manufacturers whose results did not measure up to their self-esteem (meaning they didn’t finish first) raised a caterwauling stink, bellowing at the publisher, demanding re-tests, threatening to pull ads and asking, none to politely, for my head on a platter. I defended my methodology against countless charges, including, but not limited to:
• We tested at the wrong time of year, April. Never mind that that’s when the best athletes were available, it was also the only time next year’s skis were available. It’s hard to run a ski test without skis.
• The snow isn’t right in the spring. Yet we tested for close to a month, always beginning at 8:30 and off the hill before noon, to reduce the influence of evolving snow conditions. And we changed test categories to match snow conditions as best we could. At the time, no ski tests were conducted mid-winter, but that didn’t stop the bellyachers from singling out my test for this transgression.
• Our testers all skied alike. This was one of the more patently idiotic assessments, as no two skiers on our 20+ person test team skied anything remotely alike. How anyone could conflate the techniques of Hansi Standteiner, Cameron Boyle, Kim Reichhelm and Wayne Wong, just to name a few, is beyond comprehension. When one is looking for excuses for one’s own shortcomings, any port in a storm will do.
• We didn’t tune all the submitted skis identically. This neatly misses the point, as every manufacturer had ample opportunity to tune their skis exactly as they liked before they touched snow. Some did the job, some assigned it to others, some abdicated completely. Untuned skis had absolutely no chance, but that was the ski maker’s duty to address before the fact, not lament afterwards.
• The exercise lacked statistical merit, so rankings of any kind were based solely on subjective appraisals. Exactly, just the way ski manufacturers, then as now, test their prototypes to determine which iteration to make. For years, the mighty Rossignol, who bitched about my test more than any other brand, employed four skiers as testers while we used over twenty. While neither method had a firm statistical footing, at least I wasn’t just testing with my immediate family.
• It was wrong to rank skis in order of finish, not just because the test was subjective, but because all skis were basically good (ha!) and whatever the poor consumer bought would probably make them happy, even it was utter crap. Guiding skiers to the best skis in a category so they wouldn’t piss their money away on garbage was just wrong. And the ski industry wonders why new ski sales have shrunk by over 50% since then…
But my favorite inane protest was that we did not employ intermediate skiers as testers. This idea has remarkable appeal for the unformed mind. Skis that will be bought by intermediate skiers should be tested by intermediate skiers! How obvious! The core problem can perhaps be illuminated by this comparison: it doesn’t matter how many tens of thousands of chimpanzees you plunk down in front of a computer, not one will produce a single line of Shakespeare. It should be clear to anyone who has watched their fellow man ski that most have only a tenuous idea of how to use their gear and couldn’t possibly render coherent feedback, as the supreme struggle for existence preoccupies the mind that ought to be reading nuances in the equipment.
The most strident proponent of this position is no longer with us, one I. William Berry, who produced a newsletter that served as his bully pulpit for change. Mr. Berry was a chain smoker with a jaundiced complexion, an Arafat-style, mottled beard, thick glasses, the physique of the chronically emaciated and the ski talent of a wombat. To avoid his opprobrium, ski makers routinely invited Mr. Berry to new product introductions where he unfailingly demonstrated that he hadn’t the least idea what was going on under his feet. He was led down every mountain by a patient-as-a-saint product manager whose duty was to complement this homunculus on his talent even as he opined negatively about equipment he was incapable of operating and utterly unqualified to judge.
It might surprise Mr. Berry to know that a few intermediates and even more skilled amateurs have occasionally infiltrated a ski test or two. At Snow Country we ran a contest among the readership the winner of which was awarded the right to participate in our annual on-snow adventure. He showed up almost on time, was delighted to meet the crew, given his first ski to test, a handful of test cards and brief instructions on how to proceed. That was the last we saw of him until the end of our session, when he reappeared, smiling beatifically despite otherwise looking like he had been mugged. His one test card had no scores on the front as I suspect every test criterion was Greek to him, and a pitiful scrawl on the back that began, “This ski…” and ended on this same, somewhat enigmatic note. He was overwhelmed by the prospect of evaluating one ski, much less the dozen the rest of us worked through that morning.
Eventually Snow Country’s brilliant editor, John Fry, elected to retire and management chose to replace him with a turnip, with predictable results. Luckily for me, my ski test endeavors did not end there as I landed on SKI’s test roster in the role of a rank-and-file tester. This afforded me the opportunity to witness another contest winner take a swing at the ski test piñata. This fellow was more capable but his test cards were no more coherent than the Snow Country guinea pig’s were. The highlight of his test experience came not on the slopes but at our annual sushi dinner, where, after warning him that dining with us could be dangerous, I rather ungraciously spewed shrimp heads and beer in his general direction. It was an accident caused by ill-timed hilarity, but whatever the cause it helped to indelibly imprint a day he claimed was the most memorable of his ski career.
So it’s not that intermediates won’t enjoy ski testing, they just won’t know what they are doing. Their ability to communicate, which may function at a high level in other environs, is reduced to a series of grunts and a few pencil scratchings that look more like Arabic than English. But the idea that unskilled skiers should be at the heart of the enterprise persists to this day. I can’t imagine a publication devoted to, say, violins, asking non-musicians to rate fine instruments, but for some reason ski publishers think this is a dandy idea for ski rags. The latest incarnation of this inanity is the invitation by SKI’s publishers to invite well-heeled skiers, at a steep tariff, to test with their professional crew this spring. One would think that the current economic crisis would have permanently disabused people of the notion that money and talent go hand in hand, but this initiative suggests otherwise. Buying one’s way in, with all the entitlements that implies, strikes me as a prescription for disaster. I can hardly wait.
I’d like for a moment to pull back the veil on the ski tests to which I have been privileged to participate. The skills of the testers can scarcely be underestimated (which should be evident by their bios) and is underscored with every run they take alongside their civilian brethren. What is less evident to the readership is the dedication and focus they bring to the task. Every ski, every run, is given every opportunity to shine. Each tester strives to imagine the target skier’s needs and desires and adapts his or her efforts accordingly. In the daily debriefing one hears heartfelt descriptions of ski behaviors that make one ski a star and another a dud. Is ski testing fun? Sure, but those who engage in it year after year take it very seriously. Most testers on SKI’s roster have tested hundreds, if not thousands, of skis in the course of their careers. They are, in a word, professionals.
One last behind-the-curtain insight: the raw results are far more coherent than one might expect of a subjective experiment in which the lab is constantly changing. Yes, there are some anomalous results, but for the most part the consensus best skis are a tight group, not at all as scattershot as testing’s critics would like to believe. At the end of the day, the reader is entitled to sound reporting – particularly at the price of skis today - which begins with a solid methodology and a dedicated crew trained to perform the assigned task. It is in service to the reader that amateurs and posers are excluded. Let us pray, dear reader, that this continues to be the case.
To see all of the Musings from the Pontiff of Powder, Click Here.