How I Developed a Drone Testing Methodology

27 Oct.,2022

 

Drone Test Equipment

This guest post was written by Recursion Labs

Selecting motors and propellers for drone builds can be a confusing and stressful process if you like to make well informed decisions in each component in the build. If you get it wrong, it can significantly reduce the performance and efficiency of the aircraft. Picking the correct combination is not easy, as there are a lot of choices you need to make when selecting components.

Choosing Drone Components

Choosing a Motor

Figure 1: A small selection of motor options

For motors you need to worry about the weight of the motor, as you will need to provide additional lift for a heavier motor. You need to determine what stator size you would like, and for that you need to decide on both the height, and the width of the motor stator. You then need to decide which motor Kv would be most suitable for your build, and propeller size. Once you’ve determined what you’d like for your motor, you then need to pick a brand, and there are many options at different prices. 

Choosing a Propeller

Figure 2: Various propeller options 

The motor you purchase will spin propellers to create thrust, but which propeller you use with your new motor is also a very important decision. You need to determine the length of the propeller (size), the pitch (the slope of the propeller), and how many blades you would like on the propeller. Brand, weight, price, and materials are also considerations on choosing propellers. 

Choosing a Battery

Figure 3: A small selection of battery options

Once these choices are made, you can now choose the appropriate battery to power the motors. The voltage of the battery should be appropriate to match the Kv and propeller combination. The battery must be sized correctly to support the nominal power required to run the motors in flight (which may be higher or lower depending on how the aircraft will be flown), but also be capable of handling bursting without sagging too much. The thrust output and efficiency (watts of power per gram of thrust) will determine which battery size is appropriate for the motors. If the battery you need exceeds the size or weight that would be suitable for the build that you’re making, you need to go back to the start and start thinking about other components. All three choices impact one another, and changing one may require changing the other two.

Drone Design Considerations


Figure 4: Various drone designs and builds

The majority of people who build drones reference what other people are building, and pick the same or similar parts. While this should work reasonably well for the majority of popular builds out there, this goes on the assumption that the individual they are referencing has the same flight requirements, are unbiased, and they have themselves made informed decisions while picking the parts themselves. If you are making a build that isn’t well established, there may be nothing to reference. 

When I was looking to redesign one of my 3” (propeller diameter) builds, with the goal of creating a high definition FPV drone with the highest performance (thrust to weight ratio), while being as light as possible, there was almost no quantitative data available to help choose components. There were so many unanswered questions for part and battery selection, that I decided to dedicate a portion of my free time to answering them for not just my builds, but also to help provide data to the general community to help with theirs. For this I chose to use the Tyto Robotics 1520 thrust stand. My results are published in the videos on my YouTube channel, Recursion Labs.

Creating a Variable Controlled Test Methodology

Coming up with a methodology I was happy with was more challenging than I anticipated. My requirements for testing were to ensure that testing different motors with the same propeller, or different propellers on the same motor would provide accurate and reliable comparative data, where I could run the same test at any time and get the same result within a reasonable margin of accuracy. 

I also needed each test to take as little hands-on time as possible, to make it practical to run a large number of accurate tests with only needing to spend time setting up each test. This meant that the testing procedure needed to be fully automated, which was possible to do since the Tyto Robotics software supports ECMAScript as a direct programming interface to the thrust stand. 

Input Voltage for Drone Battery

The input voltage of a motor will determine how fast it will spin. A motor powered by 16 volts at 50% throttle will spin faster than the same motor spinning at 15.8v at 50% throttle. Traditionally when motors are tested, the person testing fully charges a large Lithium Polymer battery, connects it to the Motor/ESC, and then ramps the motor from 0-100% while recording measurements. 

The problem with this method is that it introduces a very large variable in the results, even if the battery is fully charged between tests. As the motor draws current from the battery, the voltage of the battery will sag. There is no guarantee that the sag will be the same for each test, and as the battery is discharged, the resting voltage also lowers. 


Figure 5: Clean voltage regulation between 30 and 80 amps of current

To remove this variable, I’ve opted for using regulated bench power supplies configured to output a constant voltage. This means that if the power supply is configured to output 16.00v, 16.00v will be provided even if the drawn current is 1 amp, or 30 amps. The voltage sag at the motor itself is not eliminated entirely, because the sag produced by the resistance of the wires to the electronic speed controller (ESC), as well as the resistance generated from the motor itself will create a measurable voltage sag while spinning. With ensuring the wiring used is consistent for every test, the voltage sag should also be consistent between tests, as long as heat is managed.  

Heat Factors, Heat Sink and Test Resolution

With the input voltage variable removed, the next important variable is heat. When a motor is under load, it generates heat inside of the motor windings. The heat that is generated acts like a resistor, causing the voltage to drop, along with the motor performance. 

Looking back at the traditional test methodology, if you slowly spin a motor from 0-100%, the temperature of the motor slowly rises, increasing the resistance, and dropping the voltage. Measurements taken will become more and more inaccurate as the throttle increases since the compounding heat increase will cause the voltage drop to increase, reducing motor RPMs. 


Figure 6: RPM drop over 5 seconds of sustained testing due to heat at a constant 16.00V

With some earlier prototypes of automated testing, I was looking to test the differences of 4s and 6s voltages (16.0v vs 20.0v) on the same kV to determine the impact of running 6s voltages on 4s motors at the same RPMs. The code would spin the motors up until it hit a target RPM for each motor, so the measurements at each RPM could be captured and compared. The automation would quickly spin up the motors near the target RPM, then slowly increase the throttle until it hit the target RPM. 

On the larger motors used in the test, it worked reasonably well since the test was fast enough that the metal on the motor was a big enough heat sink, that it could soak up the heat before the test was completed. When doing a similar test later on smaller motors, the test results were completely unusable since once it got near the RPM and slowed down, the RPM would lower almost as fast as the throttle was rising due to the voltage sag from the generated heat. Static thrust testing will generate significantly more heat than it would with a motor pushing a propeller through the atmosphere, so this convinced me that I had to completely rebuild how testing was done, which treated heat as a variable that needed to be eliminated. 


Figure 7: Thermal image of motor accumulating heat under sustained load

The solution to eliminate heat as a variable ended up being quite simple. The motor is idled at roughly 5% motor utilization when the testing begins. The motor is bursted to a specific throttle point for 2 seconds, where measurements are recorded every 5-6ms for the duration of the test. Once the test is completed, the motor is brought back down to idle speed until the motor has enough time to cool. At lower RPMs, the motor doesn’t need to idle for long, but as the throttle point increases the idle time is increased until it is idling for up to 2 minutes between tests.

The idle period was confirmed by a thermal camera to ensure it was long enough, and then more than doubled to add a margin of safety for more inefficient tests. Each test will increment the throttle point by 0.5% for the entire throttle range, allowing for a very high resolution of tests for the entire throttle spectrum. This method does make the time for tests to run much longer, but with it being fully automated, it is more than worth it.

Result Consolidation

The results generated from the automation will create a dataset for each throttle point tested. Each dataset will contain all of the samples gathered during each test. For analysis where I may be comparing a large number of motors or propellers against one another, there could be over 1500 datasets, each with around 250 captured samples. Reconciling the data manually wouldn’t be practical, so further automation is written to go through each dataset capture an appropriate number of samples at the point after the motor would have reached the setpoint RPM, and average them out to create the result metrics for each datapoint used for analysis. Heat generated by the motor during the setpoint test may impact the overall result average, but given the limited burst time, this was seen as fair to ensure that efficiency was not overstated, as the heat variable is now dependent on the efficiency of the motor for that specific test, and should be measurably consistent for each test.

Figure 8: Sampling many data points of a single test

Once all the results are processed, the automation creates a new file with all of the consolidated data by throttle point for each specified motor/propeller/voltage. This data can now easily be used for individual, or comparative analysis. 

Analysis of Motor / Propeller Performance Data

The best way to comparatively look at motor and propeller data is using graphs, so that you can get a high-level picture of how the motor and propeller performs across the entire throttle range. After the combined results are made available, I spend time generating a number of standard graphs that provide essential information required to properly analyze the motor and propellers. To demonstrate this, I will provide examples of charts used in analysis between 8 different 3” propellers on the same motor, published in an early test video which can be found here.

Thrust 

Figure 9: Grams of Thrust vs. Motor Utilization Percentage

The starting point is the thrust generated across the entire throttle range. Here you can quickly determine if the motor propeller and motor combinations will provide sufficient thrust to meet the basic thrust to weight expectations of your aircraft. What is important here is that you have a general idea of what your target all up weight (AUW) is, so that you can see at which throttle point the aircraft would have enough thrust to hover.

You can do this by taking the AUW of the aircraft, and dividing it by 4 (for a quadcopter) to get the hover thrust required per motor multiplied by 4. Once you have the hover throttle position (which you will need for subsequent graphs), you can then get a general idea of the thrust to weight ratio that is possible on the charted motors/propellers, by dividing the maximum grams of thrust by the hover motor weight. 

In the above example if your AUW is 200g, you can draw a line at the 50-gram mark to see the hover throttle point for all of the propellers. For thrust to weight, you can determine that every propeller can deliver at least a 5:1 thrust to weight ratio, and the top two can deliver an impressive 7:1 ratio.

So why not just pick the highest thrust propellers, they deliver the most power, so they have to be the best right? Well, it’s not that simple. The motor and propeller combination is capable of that thrust, but at what cost? If generating 350g per motor of thrust requires a larger 200g battery to deliver it, making your AUW 320g, then you’ve now significantly lowered your thrust to weight ratio to about 4:1, while also sacrificing the maneuverability and impact resistance benefits of lighter aircraft.

Current 

Figure 10: Current (Amps) vs. Motor Utilization Percentage

Now that we have the thrust data, you can see how many amps of current are needed to generate the thrust. The above graph shows how many amps are needed through the entire throttle range. You could use the above two graphs together to determine how much current is needed to generate a specific amount of thrust. 

This is actually a very unintuitive way to do this, not only because it requires referencing two graphs instead of one, but also because current is almost never a good way to measure power requirements (in terms of volume/weight), because it is entirely based on the input voltage. What we really care about is how many watts (w) is required to generate thrust. Watts is calculated by taking voltage and multiplying it by the current being used. This is important because if a motor on 4s (16.0v) is using 10 amps of current (16vx10a=160w), and a motor on 6s (20.0v) is also using 10 amps of current (10ax20v=200w), the 6s is would require much more power, and a larger battery. 

Grams of Thrust per Watt

Figure 11: Grams of Thrust per Watt vs. Motor Utilization Percentage

The above chart shows how many grams of thrust are generated per watt of power, across the entire throttle range. Here you can definitely get a better picture of propeller efficiency across the throttle range, and see at which point of the throttle range you will get the most thrust for your watt of power. 

The original throttle position required to hover is now needed once again because you can clearly see in the above example that different propellers are more efficient depending on how much throttle you are using. Using the red and dark blue lines, the hover point was at a little more than 25% throttle, so you can see that hovering here would be fairly efficient vs the other propellers. 

A general rule I use for evaluating core efficiency is taking the thrust required to hover, and multiplying it by 3 (3:1 thrust to weight ratio). For casual flying, most people usually keep the throttle within 1-2x AUW thrust, and for more aggressive flying, you’d spend most of your time on the throttle between 1-3x. That’s not to say you won’t burst to much higher throttle points as you fly, but you want to make sure the throttle range you’ll be spending for the majority of flight will give you the greatest efficiency possible, increasing flight times and nominal load on the battery.  

The only problem with this chart is you still need to reference the thrust chart to get a general idea of which throttle position you need to look at to compare thrust. So we can go one step further, and once again combine the two charts into one. 

Figure 12: Watts vs. Grams of Thrust

The above chart directly compares how many watts of power are needed to generate grams of thrust. The ideal chart that you’re looking for is one that stays as low as possible (least watts) that extends as far right as possible (most thrust). The above example shows why the correct choice may not always be black and white, because if your preferred battery can not generate the watts required to drive one of the higher thrust propellers, you should start prioritizing efficiency even if it sacrifices top end thrust, since you either won’t be able to hit the thrust due to the voltage sag on the battery, or you will need to get a larger battery that is capable of bursting the required power when demanded. 

Figure 13: Watts vs. Grams of Thrust with the same 3” propeller

The above example shows a set of tests that directly compared different motors against one another, using the same propellers. This is a good example of where the thrust to efficiency quantitatively has a clear winner comparatively against motors of the same size (1404) in both efficiency and thrust (more to the bottom and the right), only losing out in overall thrust to two other larger motors.

RPM

Figure 14: Grams of Thrust vs. Motor Utilization Percentage 

Looking at motor RPMs across the throttle range for each propeller can give you a good idea of how much load is being put on the motor for it to spin the propeller. It is expected to see a more aggressive propeller spin at lower RPMs than that of a lighter, easier to drive propeller, but significant dips in RPM could be an indication that the motor doesn’t have enough torque to properly spin the propeller. If you observe a graph like you see above, especially if the dip increases as motor utilization increases, or you see a sizable dip inefficiency by motor utilization, that might be an indication that the motor may not be suitable for the propeller, and that further analysis may be required.  

Spin up RPM Data 

Figure 15: Motor RPM vs. Time in ms

Since samples from each test are captured from start to finish, the motor spinning up from idle to the setpoint RPM is also captured. Taking these samples between different propellers and motors allows for comparative analysis of the throttle response between motors and propellers, which can help determine if a motor stator is large enough to handle a propeller. The above chart shows a comparison between a 3” propeller against 3 different 3.5” propellers on the same motor. There is a 5-6ms latency between samples, so the results are not ideal for testing similar responses against one another, but it is more than good enough to identify where the difference is obvious like seen above.

Propeller Tip Speed 

Figure 16: Kph vs. Motor Utilization Percentage 

With RPM data, we can also calculate and chart other useful information, like the tip speed in Kph for the propellers across the throttle range. This data can be useful to determine if the propeller tip speed is getting too high, where efficiency can be dramatically lost the closer it approaches the speed of sound. It’s also a good reminder how dangerous the propellers can be if you stick your meaty flesh inside of them while spinning.

Conclusion

The Tyto Robotics thrust stand has allowed me to build a consistent variable controlled automated motor and propeller test methodology that allows me to focus my time towards detailed comparative analytics instead of sitting in front of a thrust stand, hoping I don’t obtain significant hearing loss as I manually write down numbers that fly by. 

Now that this milestone is reached, I will continue to do testing and analytics on motors and propellers, attempting to find unicorns while seeking the perfect balance between thrust, efficiency, and weight. I sincerely hope my efforts to publish my findings and associated data to the community will help relieve frustration induced from selecting components, and help everyone make informed decisions.  You can find videos on the results and analytics of this testing on my Recursion Labs YouTube channel here:

Now that this milestone is reached, I will continue to do testing and analytics on motors and propellers, attempting to find unicorns while seeking the perfect balance between thrust, efficiency, and weight. I sincerely hope my efforts to publish my findings and associated data to the community will help relieve frustration induced from selecting components, and help everyone make informed decisions. You can find videos on the results and analytics of this testing on my Recursion Labs YouTube channel here: https://YouTube.com/c/RecursionLabs.