Paparazzi UAS  v5.15_devel-113-g1b57ff1
Paparazzi is a free software Unmanned Aircraft System.
 All Data Structures Namespaces Files Functions Variables Typedefs Enumerations Enumerator Macros Modules Pages
cv_detect_gate module

Detects gates as used in the IROS drone races, i.e., square colored gates

It does so with snake gate detection, a computationally efficient method that works onboard of the computationally constrained Parrot Bebop 1/2 drones.

An initial version of this algorithm ran in the drone race 2016. The algorithm was first described in:

First autonomous multi-room exploration with an insect-inspired flapping wing vehicle, May 2018, IEEE International Conference on Robotics and Automation (ICRA 2018), Brisbane, Australia by Kirk Scheper, Matej Karasek, Christophe De Wagter, Bart Remes, and Guido de Croon https://www.researchgate.net/publication/327228053_First_autonomous_multi-room_exploration_with_an_insect-inspired_flapping_wing_vehicle

For the drone race, the algorithm and performance are described in more detail in: Autonomous drone race: A novel vision-based navigation and control strategy, S.Li, M.M.O.I. Ozo, C. De Wagter, G.C.H.E. de Croon. Submitted.

Example for airframe file

Add to your firmware section: This example contains all possible configuration options, not all of them are mandatory!

1 <module name="video_thread"/>
2 <module name="cv_detect_gate">
3  <define name="DETECT_GATE_FPS" value="0" />
4  <define name="DETECT_GATE_CAMERA" value="bottom_camera|front_camera" />
5  <define name="DETECT_GATE_JUST_FILTER" value="FALSE|TRUE" />
6  <define name="DETECT_GATE_N_SAMPLES" value="10000" />
7  <define name="DETECT_GATE_MIN_PIX_SIZE" value="30" />
8  <define name="DETECT_GATE_MIN_GATE_QUALITY" value="0.15" />
9  <define name="DETECT_GATE_MIN_N_SIDES" value="3" />
10  <define name="DETECT_GATE_GATE_THICKNESS" value="0.0" />
11  <define name="DETECT_GATE_Y_MIN" value="20" />
12  <define name="DETECT_GATE_Y_MAX" value="228" />
13  <define name="DETECT_GATE_U_MIN" value="42" />
14  <define name="DETECT_GATE_U_MAX" value="121" />
15  <define name="DETECT_GATE_V_MIN" value="134" />
16  <define name="DETECT_GATE_V_MAX" value="230" />
17  <define name="DETECT_GATE_EXCLUDE_PIXELS_TOP" value="0" />
18  <define name="DETECT_GATE_EXCLUDE_PIXELS_BOTTOM" value="0" />
19  <define name="DETECT_GATE_SIMPLIFIED_PNP" value="0" />
20  <define name="CAMERA_ROTATED_90DEG_RIGHT" value="FALSE" />
21 </module>

Module configuration options

Define Options

  • name: DETECT_GATE_FPS value: 0
    Description: The (maximum) frequency to run the calculations at. If zero, it will max out at the camera frame rate.
  • name: DETECT_GATE_CAMERA value: bottom_camera|front_camera
    Description: The V4L2 camera device that is used for the calculations.
  • name: DETECT_GATE_JUST_FILTER value: FALSE|TRUE
    Description: Just run a color filter.
  • name: DETECT_GATE_N_SAMPLES value: 10000
    Description: The number of samples taken in an image for snake gate detection. This number is proportional to the computatinoal effort (more = slower) and the performance (more = better).
  • name: DETECT_GATE_MIN_PIX_SIZE value: 30
    Description: The minimal size the sides of an initial (square) detection should have for further processing.
  • name: DETECT_GATE_MIN_GATE_QUALITY value: 0.15
    Description: Gate quality is checked by verifying the proportion of colored pixels on the gate outline. This is the minimal proportion of colored pixels in order to accept a gate candidate.
  • name: DETECT_GATE_MIN_N_SIDES value: 3
    Description: How many sides of the gate should have the minimal line quality (min = 0, max = 4).
  • name: DETECT_GATE_GATE_THICKNESS value: 0.0
    Description: Snaking goes up and down, and then left and right from the extreme points. If the gate is thick, it is better to start a little bit before the end point. This value is how thick the gate border is as a proportion of the total size.
  • name: DETECT_GATE_Y_MIN value: 20
    Description: Minimal Y of the color filter.
  • name: DETECT_GATE_Y_MAX value: 228
    Description: Maximal Y of the color filter.
  • name: DETECT_GATE_U_MIN value: 42
    Description: Minimal U of the color filter.
  • name: DETECT_GATE_U_MAX value: 121
    Description: Maximal U of the color filter.
  • name: DETECT_GATE_V_MIN value: 134
    Description: Minimal V of the color filter.
  • name: DETECT_GATE_V_MAX value: 230
    Description: Maximal V of the color filter.
  • name: DETECT_GATE_EXCLUDE_PIXELS_TOP value: 0
    Description: Band on top of the image in which we do not take snake gate samples.
  • name: DETECT_GATE_EXCLUDE_PIXELS_BOTTOM value: 0
    Description: Band at the bottom of the image in which we do not take snake gate samples.
  • name: DETECT_GATE_SIMPLIFIED_PNP value: 0
    Description: When set to 0, the full PnP will fit all corners. When set to 1, a simple camera model will be used.
  • name: CAMERA_ROTATED_90DEG_RIGHT value: FALSE
    Description: If the camera does not have its longest side as horizontal, then the rotated define should be enabled.

Dependencies

Module functions

Init Functions

These initialization functions are called once on startup.

Event Functions

These event functions are called in each cycle of the module event loop.

Files

Header Files

The following headers are automatically included in modules.h

Source Files

Raw cv_detect_gate.xml file:

<!DOCTYPE module SYSTEM "module.dtd">
<module name="cv_detect_gate" dir="computer_vision">
<doc>
<description>
Detects gates as used in the IROS drone races, i.e., square colored gates. It does so with snake gate detection, a computationally efficient method that works
onboard of the computationally constrained Parrot Bebop 1/2 drones.
An initial version of this algorithm ran in the drone race 2016. The algorithm was first described in:
First autonomous multi-room exploration with an insect-inspired flapping wing vehicle, May 2018,
IEEE International Conference on Robotics and Automation (ICRA 2018), Brisbane, Australia
by Kirk Scheper, Matej Karasek, Christophe De Wagter, Bart Remes, and Guido de Croon
https://www.researchgate.net/publication/327228053_First_autonomous_multi-room_exploration_with_an_insect-inspired_flapping_wing_vehicle
For the drone race, the algorithm and performance are described in more detail in:
Autonomous drone race: A novel vision-based navigation and control strategy,
S.Li, M.M.O.I. Ozo, C. De Wagter, G.C.H.E. de Croon.
Submitted.
</description>
<!-- <define name="UNDISTORT_MIN_X_NORMALIZED" value="-2.0" description="Minimal normalized x-coordinate to be used for the undistortion"/> -->
<define name="DETECT_GATE_FPS" value="0" description="The (maximum) frequency to run the calculations at. If zero, it will max out at the camera frame rate."/>
<define name="DETECT_GATE_CAMERA" value="bottom_camera|front_camera" description="The V4L2 camera device that is used for the calculations."/>
<define name="DETECT_GATE_JUST_FILTER" value="FALSE|TRUE" description="Just run a color filter."/>
<define name="DETECT_GATE_N_SAMPLES" value="10000" description="The number of samples taken in an image for snake gate detection. This number is proportional to the computatinoal effort (more = slower) and the performance (more = better)."/>
<define name="DETECT_GATE_MIN_PIX_SIZE" value="30" description="The minimal size the sides of an initial (square) detection should have for further processing."/>
<define name="DETECT_GATE_MIN_GATE_QUALITY" value="0.15" description="Gate quality is checked by verifying the proportion of colored pixels on the gate outline. This is the minimal proportion of colored pixels in order to accept a gate candidate."/>
<define name="DETECT_GATE_MIN_N_SIDES" value="3" description="How many sides of the gate should have the minimal line quality (min = 0, max = 4)."/>
<define name="DETECT_GATE_GATE_THICKNESS" value="0.0" description="Snaking goes up and down, and then left and right from the extreme points. If the gate is thick, it is better to start a little bit before the end point. This value is how thick the gate border is as a proportion of the total size."/>
<define name="DETECT_GATE_Y_MIN" value="20" description="Minimal Y of the color filter."/>
<define name="DETECT_GATE_Y_MAX" value="228" description="Maximal Y of the color filter."/>
<define name="DETECT_GATE_U_MIN" value="42" description="Minimal U of the color filter."/>
<define name="DETECT_GATE_U_MAX" value="121" description="Maximal U of the color filter."/>
<define name="DETECT_GATE_V_MIN" value="134" description="Minimal V of the color filter."/>
<define name="DETECT_GATE_V_MAX" value="230" description="Maximal V of the color filter."/>
<define name="DETECT_GATE_EXCLUDE_PIXELS_TOP" value="0" description="Band on top of the image in which we do not take snake gate samples."/>
<define name="DETECT_GATE_EXCLUDE_PIXELS_BOTTOM" value="0" description="Band at the bottom of the image in which we do not take snake gate samples."/>
<define name="DETECT_GATE_SIMPLIFIED_PNP" value="0" description="When set to 0, the full PnP will fit all corners. When set to 1, a simple camera model will be used." />
<define name="CAMERA_ROTATED_90DEG_RIGHT" value="FALSE" description="If the camera does not have its longest side as horizontal, then the rotated define should be enabled." />
</doc>
<settings>
<dl_settings>
<dl_settings NAME="Detect gate">
<dl_setting var="just_filtering" min="0" step="1" max="1" values="FALSE|TRUE" shortname="just_filter" param="DETECT_GATE_JUST_FILTER"/>
<dl_setting var="n_samples" min="0" step="10" max="10000" shortname="n_samples" param="DETECT_GATE_N_SAMPLES"/>
<dl_setting var="min_px_size" min="10" step="5" max="500" shortname="min_px" param="DETECT_GATE_MIN_PIX_SIZE"/>
<dl_setting var="min_gate_quality" min="0.0" step="0.01" max="1.0" shortname="min_qual" param="DETECT_GATE_MIN_GATE_QUALITY"/>
<dl_setting var="min_n_sides" min="0" step="1" max="4" shortname="min_sides" param="DETECT_GATE_MIN_N_SIDES"/>
<dl_setting var="gate_thickness" min="0.0" step="0.01" max="0.2" shortname="thickness" param="DETECT_GATE_GATE_THICKNESS"/>
<dl_setting var="color_Ym" min="0" step="1" max="255" shortname="Y_min" param="DETECT_GATE_Y_MIN"/>
<dl_setting var="color_YM" min="0" step="1" max="255" shortname="Y_max" param="DETECT_GATE_Y_MAX"/>
<dl_setting var="color_Um" min="0" step="1" max="255" shortname="U_min" param="DETECT_GATE_U_MIN"/>
<dl_setting var="color_UM" min="0" step="1" max="255" shortname="U_max" param="DETECT_GATE_U_MAX"/>
<dl_setting var="color_Vm" min="0" step="1" max="255" shortname="V_min" param="DETECT_GATE_V_MIN"/>
<dl_setting var="color_VM" min="0" step="1" max="255" shortname="V_max" param="DETECT_GATE_V_MAX"/>
<dl_setting var="exclude_top" min="0" step="1" max="320" shortname="exclude_top" param="DETECT_GATE_EXCLUDE_PIXELS_TOP"/>
<dl_setting var="exclude_bottom" min="0" step="1" max="320" shortname="exclude_bottom" param="DETECT_GATE_EXCCLUDE_PIXELS_BOTTOM"/>
</dl_settings>
</dl_settings>
</settings>
<depends>video_thread</depends>
<header>
<file name="detect_gate.h"/>
</header>
<init fun="detect_gate_init()"/>
<event fun="detect_gate_event()"/>
<makefile target="ap|nps">
<file name="detect_gate.c"/>
<file name="undistortion.c" dir="modules/computer_vision/lib/vision"/>
<file name="image.c" dir="modules/computer_vision/lib/vision"/>
<file name="PnP_AHRS.c" dir="modules/computer_vision/lib/vision"/>
<file name="snake_gate_detection.c" dir="modules/computer_vision"/>
</makefile>
</module>