You spent years building the plane of your dreams, and now that it is finished it is time to bring your beauty to AirVenture to be judged by a jury of your peers. A lot of questions are floating around in your head: Exactly what are the judges looking for? How will my simple airplane be judged when compared to a more complex aircraft? And what details will improve my score with the judges?
Scoring
Bob Reece is the Chairman of the EAA Judging Standards Committee and the Chief Judge for EAA AirVenture. He and the Judging Standards Committee have created a simple method for scoring aircraft. A basic decision tree is used to score each of the 10 criteria on a scale from zero to 10, with 10 being "flawless in all respects" and zero representing "major deficiency."
The judging reference guide, EAA’s Aircraft Judging Manual, can be downloaded from the EAA web site at www.airventure.org/awards/judging_manual10.pdf. This document is updated yearly, so be sure to download the current version before heading off to the big show.
Safety of flight issues are covered first in the scoring decision tree. Should the judges determine that an item is not airworthy and "mandatory rework" is required, they will score a zero for the category in which the deficiency was noted. What constitutes "mandatory rework" runs the gamut from poorly secured wires to solid fuel lines where flex lines should be used, to important controls such as a fuel valve being placed out of reach.
Homebuilt aircraft are judged in the following categories: Plansbuilt, Kitbuilt or Classic Homebuilt. Regardless of what category your dream machine fits in, it must meet the Flight and Owner Construction Requirements as outlined in the judging manual. Other aircraft such as warbirds and antiques are judged separately using the appropriate criteria.
Zero to 10 points are given for each of the following items: 1) Overall Appearance, 2) Fuselage, 3) Main Lifting Surfaces, 4) Empennage/Pitch and Yaw Surfaces, 5) Landing Gear and Brakes, 6) Cockpit or Cabin, 7) Powerplant and Prop, 8) Finish, 9) Execution and Innovation, 10) Document of Compliance with FARs.
Aircraft judges scoring tree. (Courtesy EAA.)
Overall appearance is simple in concept: Does the aircraft look good? Is it clean and neat? And, finally, is the level of fit and finish consistent from one section of the airplane to the next? Get all that right, and the aircraft will score a perfect 10 in this category.
Quality of workmanship is the top judging criteria for items two through five, as listed above. The judges are looking for sound structures and good aircraft building practices such as safety wire on appropriate fasteners, sound-looking welds and innovation or improvements to the standard aircraft.
The cockpit or cabin is scrutinized for the builder’s utilization of instruments and controls. Are they arranged neatly and in a safe location? Controls should be within reach of the pilot, and the instrument panel should not look as though it were designed with a shotgun and built with a chainsaw.
The heart and soul of most aircraft is the engine. Judges will not open inspection panels or remove cowlings to inspect the engine. However, they may ask the owner to do so. Neatness and adherence to aircraft construction techniques that produce a safe installation are paramount. The amount of droop or sag of the engine is also taken into account.
Composite aircraft, by their very nature, provide opportunities for smooth, uninterrupted surfaces that can woo judges. Smart builders also work to keep the aircraft as light as possible.
The quality of the finish is more important than the paint scheme or type of finish, paint or polish. The judges recognize that there is a difference in taste, so they do not take into account paint color or design. However, they are looking for paint drips and uniform application of the finish. Of course, underlying blemishes such as pin holes in fiberglass parts are negatives.
Building an airplane is supposed to be an educational process, so the judges appreciate and reward execution and innovation. They are looking for something new, such as new safety features or a modification to an existing feature that makes for an aesthetically pleasing or safer aircraft.
Finally, if the aircraft does not carry with it the required FAA documentation or does not comply with the FARs, don’t even attempt to have your pride and joy judged.
It Has to Fly
Each aircraft eligible for consideration must meet the following flight requirements, as outlined in the EAA’s Aircraft Judging Manual:
1. The aircraft must meet the FAA minimum requirements for the Experimental/Amateur-Built category.
2. The FAA operating limitations restricting flight to a test flight area must have been removed.
3. The aircraft shall have flown to EAA AirVenture or it must be flown during EAA AirVenture.
4. The FAA certification documents must be in order and available for inspection by the judges.
This means that the FAA has blessed your aircraft with an airworthiness certificate emblazoned with Experimental/Amateur-Built, the aircraft has flown off the required Phase I test period as described in its operating limitations, the logbooks have been appropriately endorsed, placing the aircraft in Phase II, it must either be flown to AirVenture or flown during the show, and finally the builder must have the appropriate paperwork and make it available for inspection, should the judges ask.
Finally, the EAA judges require the aircraft to meet the same 51% amateur-built requirement as the FAA. The purpose of the EAA is to promote education through homebuilding, so aircraft built by hired guns need not apply.
Homebuilt judging should be about more than the paint, though Chris Wills’ fiery red GlaStar Sportsman is without question a handsome workhorse, not a hangar queen with a thirst for Meguiar’s.
Yes, Your Honor
When talking with Reece it became evident that the quality of the build is more important than most anythin g else. When asked if the judges give additional points for a complex IFR-equipped instrument panel versus a simple VFR panel, he replied, emphatically, "No! It is important that the panel is well executed. What is in it is not important."
Reece went on to discuss what he and the rest of the judges look for. Initial impressions are important. If a plane does not catch their eye as they approach it, chances are it will not look good upon closer inspection.
Paint quality is judged, not paint schemes (color and design). So while having your aircraft adorned by the best airbrush artist you can afford looks impressive, it will not gain you any more favor with the judges than a simple but well-executed solid color.
While talking about aircraft finishes, Reece was asked how an aluminum aircraft would be judged if all of the rivet heads were filled and the aircraft skins were perfectly smooth. His reply was immediate: "Aluminum aircraft should look like aluminum aircraft. If you want smooth wings, build a composite aircraft." He went on to say that if they are looking at a line of rivets on a structure and one or two rivets are missing, they assume there is a dent hiding under filler and paint. This would only hurt the builder’s chances if the other top aircraft being judged had no such imperfections.
In short, a plane such as an RV with no visible rivet heads would not score well. Perfectly smooth wings and fuselages belong on fiberglass airplanes, and these should have no imperfections such as pinholes or glass weave showing through the paint. Reece added that fabric airplanes should look like fabric planes, and the cloth weave should be visible through the paint (or dope).
Nice touches are techniques such as taking the time to blend fiberglass and aluminum parts together (e.g., fiberglass wingtips on an aluminum wing). However, aesthetically pleasing does not necessarily increase the final number on the aircraft’s scorecard. What is important is that when attaching parts to one another, their joints should be straight and even, with constant and equal gaps and no puckers.
Flight controls always attract the judge’s eye. If they do not line up properly when in the neutral position, such as one drooping aileron, points will be deducted. Large trimtabs or wedges attached to control surfaces are telltale signs that the aircraft does not fly straight, and they may show up on a judge’s scorecard.
Custom touches and modifications to the aircraft should not detract from safety and appearance. However, they can only improve the aircraft’s score if the judges know about them. This is where staying with the aircraft to answer the judge’s questions, removing the cowl upon request or having a storyboard pointing out select features can help. The judges are always impressed by innovation, but even if you improved your aircraft by selecting someone else’s innovations that are not standard on your model aircraft, make sure to point them out and explain why these changes are important.
When the judge approaches your aircraft let him do his job, but be prepared to answer any questions and point out the innovations mentioned earlier. Listen to the judge’s comments, and he will tell you exactly what he and the other judges have observed and what you need to do to improve your score.
Winning an award is the culmination of tremendous effort by the builder.
This Old Bird?
Aircraft that are submitted for consideration do not have to be brand new with only post Phase I fairy time. The judges expect these airplanes to be flown and show some signs of use, so a homebuilt that has been cared for and maintained could still show and score well.
Once an aircraft is awarded Grand Champion, it is no longer eligible for judging at AirVenture. If the aircraft is recognized as a runner-up (Reserve Grand Champion, Champion Homebuilt or Outstanding Workmanship awards), the owner may present the aircraft for judging in subsequent years. However, each year the bar is raised a little higher, and an aircraft that won Grand Champion five years ago would perhaps not get a second look today.
How Good is Good Enough?
There is a significant difference in terms of time and money between airworthy and Grand Champion, and it is up to you, the builder, to determine what type of aircraft you want to build. In the end, it is your airplane, and only you can decide to build something that you will be proud to own and fly.
Bill Repucci was handed his Private Pilots certificate back in the days when the written test was taken with a Number 2 pencil. At the time, Bill was told that he now had a License to Learn. And learn he did, mostly that there was humor buried in the quirky ways of those of us who call ourselves aviators.
As homebuilts evolve, shouldn’t judging standards evolve as well?
Back in the early days of homebuilding, just getting a simple homebuilt into the air was its own reward, and craftsmanship was an appropriate recognition of the best aircraft. These days, homebuilding has evolved to where most homebuilts are constructed from kits, and they are often faster, enormously more complex and carry more. But the most significant difference is that after Phase I flight testing, most modern homebuilts are expected by both the original builders and by second and third owners to be used as and to have reliability comparable to a factory-built aircraft. Virtually all planes built from kits (and all ELSAs) are built with this expectation. So as homebuilts have evolved and owners have new expectations of their aircraft, shouldn’t judging standards evolve similarly?
Best Practices
It seems to me that while current judging standards reward artistry in craftsmanship, they don’t necessarily reward those attributes that make an airplane practical, usable and safe. Those attributes can be considered best practices, characteristics of excellent aircraft that are not measured by craftsmanship alone. When these best practices are ignored or diminished in judging standards, the message sent is that they really don’t matter. Worse, the message is that there is no loss of safety in putting cosmetics before best practices, because an airplane can score highly without them.
One best practice ignored in current judging standards is flight testing. Performing a full flight test, recording the data, and then organizing the data into a manual for the aircraft is not that difficult to do. In fact, builders have long since been required to perform a full envelope Phase I flight test. There’s even an FAA Advisory Circular, AC90-89A, that describes a full flight-test program. Complete flight testing is obviously a best practice, for it tells you what kind of airplane you actually have with your equipment and how you rigged the airplane. Best practice for an aircraft manual means that it will include a wiring diagram plus a list of all the equipment on board so that future maintainers will know what they’re working on.
When an airplane receives Grand Champion and has flown off its 25 or 40 hours without having had its full flight envelope tested and documented, this signals that workmanship, not flight testing per the regulations, is more important.
Homebuilder Values
A number of homebuilder values could be recognized in judging, and these are also ways to recognize meaningful innovation. These homebuilder values, all of them best practices in aircraft design and construction, include reducing cost, build time, weight, drag, stall speed, takeoff and landing distance, and especially aircraft noise, while increasing safety, comfort, cruise speed, payload, rate of climb, engine cooling, crashworthiness and handling. For example, I am much more interested in a really nice airplane built in 800 hours than in a nearly perfect airplane built in 8000 hours. And aircraft noise is a meaningful and largely unrecognized opportunity for innovation, for there’s nothing like a squadron of noisy aircraft to draw the wrong kind of attention to an endangered airport.
More Best Practices
Human factors is another area where present-day judging does not appear to consider best practices. In fact, bad cockpit design was the apparent cause of a recent homebuilt hull-loss accident when a "dangerous" switch (high boost pump) was located next to a "safe" switch (a light), and the wrong switch was actuated in flight. Judging should give meaningful weight in the scoring to best practices in human factors, even if those best practices come from certified aircraft and are not legally required for homebuilts. Again, under the present scheme, I believe that judging standards reward neatness and cosmetic execution over safety.
Maintainability is an area ripe for innovation. All of us who have worked on airplanes have stories of scrapes, burns, irretrievable dropped hardware, inaccessible components and fittings, unreachable screws and the accompanying profanities. In one Grand Champion Glasair III, the instruments were installed with conventional nuts and bolts while the wing was off and the owner stood in the opening in the bottom of the fuselage. Changing an instrument was extremely difficult without removing the wing. I’d like to see maintainability rewarded in judging, even for little things like being able to change the oil filter without making a mess. We’ve all become so used to doing things the hard way that we never seem to think about improving our lot.
Last on my list is structural safety. When an airplane is flown with a higher gross weight penciled in on the paperwork or with much more horsepower than the designer intended (or tested for), that’s not a best practice, and that’s certainly not innovation. Instead, that’s ignoring accepted best practices in engineering and taking an uncalculated risk, hoping that at least a little safety margin still exists. While some call this practice "experimental," you never hear what the experiment is and what defines success. It’s certainly not a best practice, and such airplanes should not be award winners. Again, what safety message is sent when an overweight, over-powered airplane is recognized with an award?
Consistency is key. Safety messages, no matter how well intentioned or how well done, are undermined by recognition given to aircraft that shortcut best practices, whether this recognition is in the form of awards or in media publicity.
Side Benefits
There is a half-truth in the Experimental/Amateur-Built aircraft community, and it goes something like this: "It’s a homebuilt, and you can do whatever you want." While some latitude may be permitted in homebuilt aircraft construction, there are laws that don’t care what the pilot thinks, or even if the pilot thinks. Those laws include the laws of aerodynamics, physics, safety and, of course, FAR 91 and 61. Violations of these laws seem to recur in homebuilt accidents and fatalities. If judging standards were more circumspect and less apologetic, the more cautious attitudes of improved judging standards might transfer into how homebuilts are flown, and the homebuilt safety record would improve.
Summary
Craftsmanship alone is not enough. It’s time for Grand Champions to be the best of the best in every way. All of these proposed changes are doable, and once the updated standards are in place, the most obvious question to be answered will be: Why did it take so long?
Ed Wischmeyer is a newly minted RV-8A owner and a vocal proponent of good handling qualities. He has written a number of flight reviews for this magazine and come away fairly impressed with many of those aircraft.
