Paparazzi UAS  v5.12_stable-4-g9b43e9b
Paparazzi is a free software Unmanned Aircraft System.
 All Data Structures Namespaces Files Functions Variables Typedefs Enumerations Enumerator Macros Modules Pages
cv_opticflow module

Compute optical flow.

Made for Linux video Devices. Computes x and y velocity using optical flow and distance to ground (using sonar).

Example for airframe file

Add to your firmware section:

1 <module name="video_thread"/>
2 <module name="pose_history"/>
3 <module name="cv_opticflow"/>

Module configuration options

Airframe file section

  • section name: OPTICFLOW prefix: OPTICFLOW_
    • name AGL_ID value: ABI_BROADCAST
      Description: ABI receive id for ABI message (sonar measurement) (default: ABI_BROADCAST)
    • name IMU_ID value: ABI_BROADCAST
      Description: ABI receive id for ABI message (accelerometers) (default: ABI_BROADCAST)
    • name BODY_TO_IMU_ID value: ABI_BROADCAST
      Description: ABI receive id for ABI message (body to imu quat) (default: ABI_BROADCAST)
    • name SEND_ABI_ID value: 1
      Description: ABI sender id for opticflow messages (default: 1)
    • name CAMERA value: bottom_camera|front_camera
      Description: The V4L2 camera device that is used for the calculations
    • name FOV_W value: 0.89360857702
      Description: The field of view width of the bottom camera (Defaults are from an ARDrone 2)
    • name FOV_H value: 0.67020643276
      Description: The field of view height of the bottom camera (Defaults are from an ARDrone 2)
    • name FX value: 343.1211
      Description: Field in the x direction of the camera (Defaults are from an ARDrone 2)
    • name FY value: 348.5053
      Description: Field in the y direction of the camera (Defaults are from an ARDrone 2)
    • name DEROTATION_CORRECTION_FACTOR_X value: 1.0
      Description: Correction factor for derotation (in x direction), estimated from a fit between the gyro's rates and the resulting flow (caused by the camera not being exactly in the middle (Defaults are from an ARDrone 2)
    • name DEROTATION_CORRECTION_FACTOR_Y value: 1.0
      Description: Correction factor for derotation (in y direction), estimated from a fit between the gyro's rates and the resulting flow (caused by the camera not being exactly in the middle (Defaults are from an ARDrone 2)
    • name METHOD value: 0
      Description: Method used to calculate optical flow
    • name WINDOW_SIZE value: 10
      Description: Window size used for block matching (pixels)
    • name SEARCH_DISTANCE value: 10
      Description: Maximum search distance for blockmatching (pixels)
    • name SUBPIXEL_FACTOR value: 10
      Description: Amount of subpixels per pixel, used for more precise (subpixel) calculations of the flow
    • name RESOLUTION_FACTOR value: 1000
      Description: The resolution factor needed to calculate the divergence without floats
    • name DEROTATION value: 1
      Description: Derotation either turned on or off (depended on gyroscope measurements)
    • name MEDIAN_FILTER value: 0
      Description: A median filter on the resulting velocities to be turned on or off (last 5 measurements)
    • name KALMAN_FILTER value: 1
      Description: A kalman filter on the resulting velocities to be turned on or off (fused with accelerometers)
    • name KALMAN_FILTER_PROCESS_NOISE value: 0.01
      Description: The expected variance of the error of the model's prediction in the kalman filter
    • name FEATURE_MANAGEMENT value: 1
      Description: Whether to keep already tracked corners in memory for the next frame or re-detect new ones every time
    • name FPS value: 0
      Description: The (maximum) frequency to run the calculations at. If zero, it will max out at the camera frame rate
    • name MAX_TRACK_CORNERS value: 25
      Description: The maximum amount of corners the Lucas Kanade algorithm is tracking between two frames
    • name MAX_ITERATIONS value: 10
      Description: Maximum number of iterations the Lucas Kanade algorithm should take
    • name THRESHOLD_VEC value: 2
      Description: TThreshold in subpixels when the iterations of Lucas Kanade should stop
    • name FAST9_ADAPTIVE value: TRUE
      Description: Whether we should use and adapative FAST9 crner detection threshold
    • name FAST9_THRESHOLD value: 20
      Description: FAST9 default threshold
    • name FAST9_MIN_DISTANCE value: 10
      Description: The amount of pixels between corners that should be detected
    • name FAST9_PADDING value: 20
      Description: The outer border in which no corners will be searched
    • name FAST9_REGION_DETECT value: 1
      Description: Whether to detect fast9 corners in regions of interest or the whole image (only works with feature management)
    • name FAST9_NUM_REGIONS value: 9
      Description: The number of regions of interest to split the image into

Dependencies

Module functions

Init Functions

These initialization functions are called once on startup.

Periodic Functions

These functions are called periodically at the specified frequency from the module periodic loop.

  • opticflow_module_run()
    • Running at maximum module frequency.
    • Autorun: TRUE
      Periodic function automatically starts after init.

Files

Header Files

The following headers are automatically included in modules.h

Source Files

Raw cv_opticflow.xml file:

<!DOCTYPE module SYSTEM "module.dtd">
<module name="cv_opticflow" dir="computer_vision">
<doc>
<description>
Compute optical flow.
Made for Linux video Devices.
Computes x and y velocity using optical flow and distance to ground (using sonar).
</description>
<!-- Optical flow calculation parameters -->
<section name="OPTICFLOW" prefix="OPTICFLOW_">
<define name="AGL_ID" value="ABI_BROADCAST" description="ABI receive id for ABI message (sonar measurement) (default: ABI_BROADCAST)"/>
<define name="IMU_ID" value="ABI_BROADCAST" description="ABI receive id for ABI message (accelerometers) (default: ABI_BROADCAST)"/>
<define name="BODY_TO_IMU_ID" value="ABI_BROADCAST" description="ABI receive id for ABI message (body to imu quat) (default: ABI_BROADCAST)"/>
<define name="SEND_ABI_ID" value="1" description="ABI sender id for opticflow messages (default: 1)"/>
<!-- Video device parameters -->
<define name="CAMERA" value="bottom_camera|front_camera" description="The V4L2 camera device that is used for the calculations"/>
<!-- Camera parameters -->
<define name="FOV_W" value="0.89360857702" description="The field of view width of the bottom camera (Defaults are from an ARDrone 2)"/>
<define name="FOV_H" value="0.67020643276" description="The field of view height of the bottom camera (Defaults are from an ARDrone 2)"/>
<define name="FX" value="343.1211" description="Field in the x direction of the camera (Defaults are from an ARDrone 2)"/>
<define name="FY" value="348.5053" description="Field in the y direction of the camera (Defaults are from an ARDrone 2)"/>
<define name="DEROTATION_CORRECTION_FACTOR_X" value="1.0" description="Correction factor for derotation (in x direction), estimated from a fit between the gyro's rates and the resulting flow (caused by the camera not being exactly in the middle (Defaults are from an ARDrone 2)"/>
<define name="DEROTATION_CORRECTION_FACTOR_Y" value="1.0" description="Correction factor for derotation (in y direction), estimated from a fit between the gyro's rates and the resulting flow (caused by the camera not being exactly in the middle (Defaults are from an ARDrone 2)"/>
<!-- General optical flow calculation parameters -->
<define name="METHOD" value="0" description="Method used to calculate optical flow"/>
<define name="WINDOW_SIZE" value="10" description="Window size used for block matching (pixels)"/>
<define name="SEARCH_DISTANCE" value="10" description="Maximum search distance for blockmatching (pixels)"/>
<define name="SUBPIXEL_FACTOR" value="10" description="Amount of subpixels per pixel, used for more precise (subpixel) calculations of the flow"/>
<define name="RESOLUTION_FACTOR" value="1000" description="The resolution factor needed to calculate the divergence without floats"/>
<define name="DEROTATION" value="1" description="Derotation either turned on or off (depended on gyroscope measurements)"/>
<define name="MEDIAN_FILTER" value="0" description="A median filter on the resulting velocities to be turned on or off (last 5 measurements)"/>
<define name="KALMAN_FILTER" value="1" description="A kalman filter on the resulting velocities to be turned on or off (fused with accelerometers)"/>
<define name="KALMAN_FILTER_PROCESS_NOISE" value="0.01" description="The expected variance of the error of the model's prediction in the kalman filter"/>
<define name="FEATURE_MANAGEMENT" value="1" description="Whether to keep already tracked corners in memory for the next frame or re-detect new ones every time"/>
<define name="FPS" value="0" description="The (maximum) frequency to run the calculations at. If zero, it will max out at the camera frame rate"/>
<!-- Lucas Kanade optical flow calculation parameters -->
<define name="MAX_TRACK_CORNERS" value="25" description="The maximum amount of corners the Lucas Kanade algorithm is tracking between two frames"/>
<define name="MAX_ITERATIONS" value="10" description="Maximum number of iterations the Lucas Kanade algorithm should take"/>
<define name="THRESHOLD_VEC" value="2" description="TThreshold in subpixels when the iterations of Lucas Kanade should stop"/>
<!-- FAST9 corner detection parameters -->
<define name="FAST9_ADAPTIVE" value="TRUE" description="Whether we should use and adapative FAST9 crner detection threshold"/>
<define name="FAST9_THRESHOLD" value="20" description="FAST9 default threshold"/>
<define name="FAST9_MIN_DISTANCE" value="10" description="The amount of pixels between corners that should be detected"/>
<define name="FAST9_PADDING" value="20" description="The outer border in which no corners will be searched"/>
<define name="FAST9_REGION_DETECT" value="1" description="Whether to detect fast9 corners in regions of interest or the whole image (only works with feature management)"/>
<define name="FAST9_NUM_REGIONS" value="9" description="The number of regions of interest to split the image into"/>
</section>
</doc>
<settings>
<dl_settings NAME="Vision stabilization">
<!-- Optical flow calculations parameters -->
<dl_settings name="vision_calc">
<dl_setting var="opticflow.method" min="0" step="1" max="1" module="computer_vision/opticflow_module" shortname="method" values="LK_Fast9|EdgeFlow" param="METHOD"/>
<dl_setting var="opticflow.window_size" module="computer_vision/opticflow_module" min="0" step="1" max="20" shortname="window_size" param="OPTICFLOW_WINDOW_SIZE"/>
<dl_setting var="opticflow.search_distance" module="computer_vision/opticflow_module" min="0" step="1" max="50" shortname="search_distance" param="SEARCH_DISTANCE"/>
<dl_setting var="opticflow.subpixel_factor" module="computer_vision/opticflow_module" min="0" step="10" max="1000" shortname="subpixel_factor" param="OPTICFLOW_SUBPIXEL_FACTOR"/>
<dl_setting var="opticflow.resolution_factor" module="computer_vision/opticflow_module" min="10" step="10" max="10000" shortname="resolution_factor" param="OPTICFLOW_RESOLUTION_FACTOR"/>
<dl_setting var="opticflow.derotation" min="0" step="1" max="1" module="computer_vision/opticflow_module" values="OFF|ON" shortname="derotation" param="OPTICFLOW_DEROTATION"/>
<dl_setting var="opticflow.median_filter" module="computer_vision/opticflow_module" min="0" step="1" max="1" values="OFF|ON" shortname="median_filter" param="OPTICFLOW_MEDIAN_FILTER"/>
<dl_setting var="opticflow.kalman_filter" module="computer_vision/opticflow_module" min="0" step="1" max="1" values="OFF|ON" shortname="kalman_filter" param="OPTICFLOW_KALMAN_FILTER"/>
<dl_setting var="opticflow.kalman_filter_process_noise" module="computer_vision/opticflow_module" min="0.0001" step="0.0001" max="0.1" shortname="KF_process_noise" param="OPTICFLOW_KALMAN_FILTER_PROCESS_NOISE"/>
<dl_setting var="opticflow.feature_management" module="computer_vision/opticflow_module" min="0" step="1" max="1" values="OFF|ON" shortname="feature_management" param="OPTICFLOW_FEATURE_MANAGEMENT"/>
<!-- Specifically for Lucas Kanade and FAST9 -->
<dl_setting var="opticflow.max_track_corners" module="computer_vision/opticflow_module" min="0" step="1" max="500" shortname="max_trck_corners" param="OPTICFLOW_MAX_TRACK_CORNERS"/>
<dl_setting var="opticflow.max_iterations" module="computer_vision/opticflow_module" min="0" step="1" max="100" shortname="max_iterations" param="OPTICFLOW_MAX_ITERATIONS"/>
<dl_setting var="opticflow.threshold_vec" module="computer_vision/opticflow_module" min="0" step="1" max="100" shortname="threshold_vec" param="OPTICFLOW_THRESHOLD_VEC"/>
<dl_setting var="opticflow.fast9_adaptive" module="computer_vision/opticflow_module" min="0" step="1" max="1" values="FALSE|TRUE" shortname="fast9_adaptive" param="OPTICFLOW_FAST9_ADAPTIVE"/>
<dl_setting var="opticflow.fast9_threshold" module="computer_vision/opticflow_module" min="0" step="1" max="255" shortname="fast9_threshold" param="OPTICFLOW_FAST9_THRESHOLD"/>
<dl_setting var="opticflow.fast9_min_distance" module="computer_vision/opticflow_module" min="0" step="1" max="500" shortname="fast9_min_distance" param="OPTICFLOW_FAST9_MIN_DISTANCE"/>
<dl_setting var="opticflow.fast9_padding" module="computer_vision/opticflow_module" min="0" step="1" max="50" shortname="fast9_padding" param="OPTICFLOW_FAST9_PADDING"/>
<dl_setting var="opticflow.fast9_region_detect" module="computer_vision/opticflow_module" min="0" step="1" max="1" values="OFF|ON" shortname="fast9_region_detect" param="OPTICFLOW_FAST9_REGION_DETECT"/>
<dl_setting var="opticflow.fast9_num_regions" module="computer_vision/opticflow_module" min="1" step="1" max="25" shortname="fast9_num_regions" param="OPTICFLOW_FAST9_NUM_REGIONS"/>
<!-- Changes pyramid level of lucas kanade optical flow. -->
<dl_setting var="opticflow.pyramid_level" module="computer_vision/opticflow_module" min="0" step="1" max="10" shortname="pyramid_level" param="OPTICFLOW_PYRAMID_LEVEL"/>
</dl_settings>
</dl_settings>
</settings>
<depends>video_thread,pose_history</depends>
<header>
<file name="opticflow_module.h"/>
</header>
<init fun="opticflow_module_init()"/>
<periodic fun="opticflow_module_run()" autorun="TRUE"/>
<makefile target="ap">
<!-- Include the needed Computer Vision files -->
<include name="modules/computer_vision"/>
<file name="image.c" dir="modules/computer_vision/lib/vision"/>
<file name="jpeg.c" dir="modules/computer_vision/lib/encoding"/>
<file name="rtp.c" dir="modules/computer_vision/lib/encoding"/>
<file name="v4l2.c" dir="modules/computer_vision/lib/v4l"/>
<!-- The optical flow module (calculator) -->
<file name="opticflow_module.c"/>
<file name="opticflow_calculator.c" dir="modules/computer_vision/opticflow"/>
<file name="size_divergence.c" dir="modules/computer_vision/opticflow"/>
<file name="linear_flow_fit.c" dir="modules/computer_vision/opticflow"/>
<file name="pprz_algebra_float.c" dir="math"/>
<file name="pprz_matrix_decomp_float.c" dir="math"/>
<!-- Main vision calculations -->
<file name="fast_rosten.c" dir="modules/computer_vision/lib/vision"/>
<file name="lucas_kanade.c" dir="modules/computer_vision/lib/vision"/>
<file name="edge_flow.c" dir="modules/computer_vision/lib/vision"/>
<file name="kalman_filter_vision.c" dir="modules/computer_vision/lib/filters"/>
</makefile>
</module>