There are basically four types of field testing, with some overlap between them.
1. Evaluation of conceptual and product prototypes in connection with research and development engineering, working with engineering department. 2. Evaluation of products during the process of product release, working primarily with engineering department. 3. Evaluation of products during the process of product release, working primarily with marketing department. 4. Evaluation of products during or after introduction, primarily for the purpose of publishing a report in a magazine or other public media.
Evaluation of prototypes requires a high level of knowledge and experience with metal detectors, the ability to communicate findings to the engineers, and a keen appreciation for the need for confidentiality. In general, those who do such evaluation receive neither money nor "freebies"-- they do it out of love for the hobby. The people who are best at this often do field testing for more than one company, which places a high premium on their professional reputation for protecting of trade secrets. Persons without brand loyalty are usually preferred for this kind of field testing because this normally results in more objective reporting of findings to engineering department.
Evaluation of products for engineering feedback during the process of product release requires knowledge and experience with metal detectors, particularly with those which are somewhat similar to the product being released. Ability to communicate findings to engineers, and the ability to refrain from leaking information without explicit permission to do so, are important. The person doing the evaluation is not normally paid anything but is often allowed to keep the unit they are testing, with the factory providing upgrades to production specifications as may become necessary. Usually, after the product is fully released, the field tester is given permission to openly discuss what they know about the product as released, but is encouraged to refrain from discussing issues which arose during field testing which are not representative of the actual released product. The person doing the testing is preferably without brand loyalty; or has some loyalty to the company on behalf of whom the field testing is done, but not so much that objectivity in reporting to engineering department might be compromised.
Evaluation of products during the process of product release on behalf of marketing department requires knowledge and experience with metal detectors, especially of competitors’ units in the same general category, and the ability to communicate clearly in public settings such as Internet forums and competition hunts the desirable characteristics of the product. The field tester must use good judgment in releasing information about the product which is representative of what the product is or will be, while refraining from comment on issues which are still not fully determined. Because this activity is a marketing activity, some brand loyalty on the part of the person doing the field testing is usually expected, and said person will often have their own business interests involved such as a dealership or metal detecting website. It is customary for the person doing the field testing to be allowed to keep the unit they test, the factory updating the unit at factory expense if necessary to bring it up to full product release specifications. Since marketing departments like wheeling and dealing, sometimes there is also other compensation.
Evaluation of products during or after introduction primarily for the purpose of publishing a “field test” for instance in a magazine, requires someone who is knowledgeable enough to do the kind of evaluation appropriate for that product, is skilled in the art of the written or video medium where the field test will be published, and has the necessary connections to get the work published. This kind of testing is arranged through marketing department.
PERCEPTION OF REPORTING BIAS ON THE PART OF FIELD TESTERS
There is an inherent bias in choosing people for field testing, in that someone who doesn’t like the sort of product in question isn’t going to be asked to evaluate it—they lack the competency required to do the job. On the other hand, publicity is best coming from people who can express themselves knowledgeably and honestly, because that is necessary for publicity to be taken seriously. Back in the bad old days, all that most people knew about a new metal detector release was what they read in advertising or were told by their dealer. Nowadays, Internet forums and other usergroups have changed all that. Once a product is released and paying customers who have no connection with the company get their hands on it, they will be frank what they do and don’t like about it and the whole world can click and read it. A field tester who stretches the truth too much or pretends that a product has no limitations gets found out and loses credibility. Worse yet, a field tester without good judgment can early on convince people that a product is something different from what it actually is, and the factory is deluged with customer complaints that the product was not what the customer expected.
Sometimes people will believe that a particular person speaks of a particular manufacturer’s products glowingly to the point of dishonesty, just because they get freebie field test units. This is rarely true. To do field testing with an unfamiliar metal detector and then to communicate with the company and later to the public requires many hours of work on the part of a knowledgeable and highly skilled person, and in most cases the person already has a whole stable of metal detectors and isn’t in desperate need of another one. The factory comes out ‘way ahead on the deal, and for the person doing field testing it’s mostly that they just happen to love doing field testing.
Summarizing: the people who report the results of factory-sponsored field testing will for the most part be favorable toward the product for the simple reason that people who are predisposed not to like that kind of product are the wrong people to be using for field testing. It often happens that a field tester will outright fall in love with the new product: after all, that’s the reaction the new product was supposed to provoke in customers. Call it bias if you like, but don’t expect sympathy. ……Outright dishonesty in field test reporting usually works to the factory’s long term disadvantage, and most companies avoid that. Finally, remember that a machine one person likes can be one that someone else hates, due to different personal preferences. Such disagreements don’t necessarily indicate either ignorance or dishonesty on anyone’s part.
THERE IS NO ONE “RIGHT WAY” TO DO FIELD TESTING
If you follow the Internet forums you may be aware that over the last several years we have been using two different individuals in particular to evaluate several different products. Neither individual would be regarded as in the top echelon of beeperists, although both have used a number of different machines under various conditions.
The first fellow generally reports how much he likes a machine, and has little negative to say about it. He is very good at describing the overall character of a machine and identifying the reasons why a person who would like that kind of machine would like it. Some folks out there in beeperlandia probably think we give this guy freebies so he’ll say good stuff about ‘em, but he is primarily an engineering dept. contact and we would regard him as a valuable field tester if he never said a word in public.
The second fellow is usually generous with criticism. I would describe him as looking for the Holy Grail of metal detectors and of course nothing ever measures up, but he sure is thorough in going over the details! He is also primarily an engineering dept. contact. Most marketing people would scratch a fellow like that off the list, but fortunately our marketing dept. sees the value he adds to our efforts. Those who follow his posts know that when criticizing he’s not grinding an axe, it’s just his way of describing the gap between what he’s got in his hands and his perception of what would make a perfect metal detector. And when he says he likes something, nobody doubts for a minute that he really does like it.
We have also used a number of other people, some well known and others not well known, for field testing. I mention the two fellows above in particular because their approach is so different and yet equally as valuable.
FIELD TESTING BY DEALERS
Field testing by dealers is a touchy subject. It often happens that dealers are highly qualified for this kind of work. But if the company involves a dealer in field testing, we often hear complaints from other dealers of "favoritism" (as though field testing were a perk to dealers, not a piece of hard work to be done on behalf of the manufacturer). Well, sometimes a dealer is so well qualified that putting up with the bellyachers is worth it.
A QUICK REALITY CHECK
What I have described above is not “official policy”, since each new product and each individual who might be used for field testing represents a special circumstance. It is however a description of the way things generally tend to happen most of the time in the American metal detector industry, including “the El Paso company”.
HOW DO WE CHOOSE FIELD TESTERS?
We often receive inquiries from people volunteering to do field testing for us. Because of the level of professionalism needed for this task, the rule of thumb is "don't call us, we'll call you". Marketing department develops contacts in the ordinary course of their business: engineering department develops contacts either through marketing department or through discovering knowledgeable people through independent channels, primarily Internet forums.
However, there have been instances where people with the necessary qualifications and understanding of the task "called us" and things worked out well. If after having read this, you believe you're a good candidate for field testing, you are invited to contact either the marketing manager or the engineering manager (which ever is more appropriate for the kind of field testing you're interested in) explaining why you're interested, what your qualifications are, and how we can verify the information you supply. Please use either "snail mail" or company email: telephone doesn’t count.
Dave Johnson
Chief Designer, Fisher Research Labs & First Texas Products
