A standard proficiency test for small drone pilots

27 Oct.,2022


RC Thrust Test


Alan Frazier |

October 27, 2020

Alan Frazier |

Estimated reading time 12 minutes, 3 seconds.

The use of small unmanned aircraft systems (sUAS) by public safety agencies is increasing at an exponential rate. 

It is estimated that over 1,700 U.S. public safety agencies have acquired sUAS. The relatively low cost of sUAS, combined with the simple process required to obtain an FAA Remote Pilot Certificate, have influenced law enforcement, fire, and search-and-rescue agencies to adopt the technology. 

More than 1,700 U.S. public safety agencies have acquired and are operating small drones for various missions. Drone Life Photo

The vast majority of public safety agencies operating sUAS are exercising Federal Aviation Regulation 14 CFR Part 107, “The Small UAS Rule”.  Enacted in 2016, Part 107 was eight years in the making as the regulations started with recommendations made by the Small Unmanned Aircraft Systems (sUAS) Aviation Rulemaking Committee (ARC) that was formed in 2008. Part 107 addresses a wide variety of sUAS regulatory issues including maximum altitude, maximum velocity, cloud clearances and, most applicable to this discussion, remote pilot certification. 

Part 107 created a new FAA pilot certification, the Remote Pilot Certificate. Applicants for a Remote Pilot Certificate must pass an FAA examination consisting of 60 questions covering 12 aeronautical subject matter areas, apply for the certificate in-person at an FAA Flight Standards District Office (FSDO) or utilize the FAA’s online Integrated Airman Certificate and Rating Application (IACRA), and be vetted by the Transportation Security Administration (TSA). 

The FAA Remote Pilot Examination covers a broad spectrum of topics including airspace, aeronautical charts, meteorology, aeronautical decision making, and the specifics of Part 107 regulations. Interestingly, and some would say disturbingly, the FAA’s remote pilot certification process has no practical examination. Consequently, we have no national standard for remote pilot flying skills. 

While the FAA’s Remote Pilot Examination is a good evaluation of remote pilot knowledge, the lack of a practical examination leaves a void that potentially increases liability exposure of individuals and agencies utilizing sUAS. Enter the National Institute of Standards and Technology, or NIST, a non-regulatory agency within the United States Department of Commerce. Its mission is to promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve quality of life. 

NIST employs approximately 3,400 talented scientists, engineers, technicians and administrative personnel at two campuses: Gaithersburg, Maryland, and Boulder, Colorado. NIST has created a set of sUAS standard test methods that are available for adoption by organizations that wish to credential sUAS pilots and/or evaluate sUAS equipment.

“The first step toward credentialing remote pilot skills is to get everybody onto the same measuring stick. That’s where standard test methods can play a key role,” said Adam Jacoff, project leader for Emergency Response Robots Section at NIST. “Especially across public safety, industrial, commercial and even recreational pilots. All need to demonstrate essential maneuvers to maintain positive aircraft control while performing whatever payload functionality is necessary to successfully perform the intended tasks.” 

Jacoff is leading an international effort to develop standard test methods for small unmanned aircraft systems. The test methods can be used to evaluate various system capabilities as well as remote pilot proficiency. The test methods are standardized through the ASTM International Standards Committee on Homeland Security Applications; Response Robots (ASTM E54.09). They are also referenced as job performance requirements in the National Fire Protection Association Standard for Small Unmanned Aircraft Systems Used For Public Safety Operations (NFPA 2400) and the ASTM Standard Guide for Training for Remote Pilot in Command of Unmanned Aircraft Systems Endorsement (ASTM F38.03 F3266-18). The U.S. Department of Homeland Security’s Science and Technology Directorate supports the development of these standard test methods.

The NIST sUAS standard test methods encompass four different “test lane” protocols: basic proficiency evaluation for remote pilots (Part 107 qualification), open test lane, obstructed test lane and confined test lane.

The test methods may be utilized to evaluate sUAS airframes and sensor systems as well as remote pilot evaluation and credentialing. They are easily performed with test apparatus that can be assembled inexpensively with materials readily available at any large hardware store. NIST has done an admirable job of creating a comprehensive user guide, scoring sheets, and printable targets. 

“Suites of standard test methods provide common measures with quantitative scores,” Jacoff said. “They can be conducted individually, in sequences, or embedded into operational training scenarios to provide reproducible scores that augment typically qualitative assessments. Organizations using these tests set their own thresholds of acceptable system and pilot performance to align with their airspace, environment, and mission complexities. Those decisions are easier to make and trust when they are based on quantitative performance data.”

Participants in NIST’s test method validation exercises learn how to fabricate apparatuses, conduct trials, and embed them into their own training and credentialing programs. 

I first realized the potential of these test methods while participating in one of NIST’s exercises. I was greatly impressed by the potential for agencies that wish to internally credential sUAS pilots or serve as a credentialing resource for others.  

One possible layout for the NIST’s standardized sUAS drone pilot test. NIST Photo

The NIST test methods are already being used as the basis for state-wide credentialing of emergency responders in Colorado and Texas. Many other state and local emergency response organizations are also adopting the test methods. Canada is moving quickly to implement these tests as the basis for credentialing their emergency responders nationwide. Others will certainly follow. The Airborne Public Safety Accreditation Commission (APSAC) is strongly considering adoption, as is the Civil Air Patrol, an auxiliary of the U.S. Force, as they seek to standardize their pilot credentialing across 52 wings consisting of 1,200 sUAS pilots.

Ben Miller, director of the Colorado Center of Excellence for Advanced Technology Aerial Firefighting, has followed NIST’s sUAS standard test methods project from the inception. 

“NIST was one of the very first evaluation groups to show interest during the early days of UAS in public safety,” Miller said. “The rigor that today’s standard test methods show is a direct result of their years of work into the project. The applicability of the method supports acquisition decisions as well as employment considerations. The . . . [tests] produce data that can be used to answer the questions of ‘What system do I buy?’ and ‘What system do I use for which mission?’” 

The “Basic Proficiency Evaluation for Remote Pilots (BPERP) can be administered in 10 minutes using three standard omni bucket stands, a 50-foot (15-meter) tape measure, and a stop watch.

Miller is convinced that the NIST sUAS tests have great applicability to public safety sUAS operations. He has backed up that opinion by being an early adopter of the NIST test methods. 

“The Colorado Department of Public Safety has adopted the NIST sUAS Standard Test Methods within our UAS certification process,” Miller said. “Managed by the Center of Excellence for Advanced Technology Aerial Firefighting (CoE) within the Division of Fire Prevention and Control (DFPC), the CoE provides this certification process to stakeholder public safety agencies within the State of Colorado. To date, 16 agencies and 42 UAS operators have gone through the process.”

The “Basic Proficiency Evaluation for Remote Pilots (BPERP) can be administered in 10 minutes using three standard omni bucket stands, a 50-foot (15-meter) tape measure, and a stop watch. The BPERP requires a test area of 50 feet (15m) by 20 feet (6m) so it can easily be administered indoors or outdoors. 

During the test, the remote pilot conducts three takeoffs and landings from and to a 12-inch (30 centimeter) radius circle, climb to specified altitudes of 10 feet (3m) and 20 (6m), conduct yawing turns, and forward, reverse, and transverse flight maneuvers. 

The goal is to capture still images of 36 targets standard lumber. The bucket stands are easy to disassemble and can be transported in a couple of nylon golf club bags or simply stacked and placed in a vehicle. The test consists of one maneuvering phase and two transverse flight phases. Pilots earn one point for each accurately captured target image, two points for an accurate first landing, and one point each for accurate second and third landings. Scoring sheets are available from NIST. Agencies set their own benchmarks scores for passing the test. Ten minutes is the NIST recommended time limit but agencies are free to set a different limit.

The easy setup, scalability, and indoor/outdoor flexibility of the standard test methods make them a great match for public safety organizations. NIST Photo

I have had the opportunity to administer the BPERP to novice and experienced members of the Grand Forks Northeast Regional sUAS Team, sponsored by the Grand Forks County Sheriff’s Office, and remote pilots assigned to the Civil Air Patrol (CAP) North Dakota Wing. The test methods were unanimously endorsed by every pilot that I have run through the course. 

The easy setup, scalability, and indoor/outdoor flexibility of the standard test methods make them a great match for public safety organizations. The well organized system NIST has created, including a comprehensive users guide to educate and assist adopters, make implementation of the standard test methods an almost turn-key operation. Beyond the BPERP are three more advanced test protocols using the open, obstructed and confined test lanes that NIST has created. Agencies can use these more advanced test methods to continuously raise the bar for their remote pilots by creating scenario-based test lanes including advanced outdoor and indoor mission sets.

If I sound enthusiastic about the NIST sUAS standard test methods it is because I believe they represent an excellent way for organizations to raise the bar for remote pilot credentialing as well as exercise the standard test methods to evaluate sUAS equipment. Adoption of NIST’s sUAS Standard Test Methods promises to raise sUAS pilot proficiency, fill the void created by the absence of an FAA remote pilot practical examination, and ultimately provide agencies with further risk mitigation in the area of sUAS accidents and civil liability defense. 

Throughout the next year the Airborne Public Safety Association (APSA) will be presenting several NIST sUAS “Train-the-Trainer” Workshops. These three-day courses are appropriate for experienced sUAS pilots who serve as trainers, supervisors and managers within sUAS Units. Check the APSA Website (www.publicsafetyaviation.org) for an upcoming workshop.

Alan Frazier currently serves as a senior fellow at Georgetown University and is assigned full-time to the National Institute of Standards and Technology where he works on development of sUAS standard test methods. He previously served as an associate professor of aviation at the University of North Dakota. He is a 40 year law enforcement professional having served as a sworn officer and supervisor with city, county, state and federal law enforcement agencies. He founded, and served 10 years as the officer-in-charge of, the Grand Forks County Sheriff’s Office Northeast Regional sUAS Team in Grand Forks, North Dakota. He is an FAA Airline Transport Pilot rated to fly single and multi-engine airplanes, helicopters, gliders, and small unmanned aircraft systems.