Compute optical flow

Module XML file: cv_opticflow.xml

Made for Linux video Devices. Computes x and y velocity using optical flow and distance to ground (using sonar).

Example for airframe file

<modules>
  <load name="cv_opticflow.xml"/>
</modules>

Module configuration options

Airframe file section

• section name: OPTICFLOW prefix: OPTICFLOW_
  • name AGL_ID value: ABI_SENDER_ID
    Description: ABI sender id for AGL message (sonar measurement) (default: ABI_BROADCAST)
  • name DEVICE value: /dev/video2
    Description: The V4L2 camera device that is used for the calculations
  • name DEVICE_SIZE value: 320,240
    Description: The V4L2 camera device width and height
  • name DEVICE_BUFFERS value: 15
    Description: Amount of V4L2 video buffers
  • name SUBDEV value: None
    Description: The V4L2 subdevice to initialize before the main device
  • name FOV_W value: 0.89360857702
    Description: The field of view width of the bottom camera (Defaults are from an ARDrone 2)
  • name FOV_H value: 0.67020643276
    Description: The field of view height of the bottom camera (Defaults are from an ARDrone 2)
  • name FX value: 343.1211
    Description: Field in the x direction of the camera (Defaults are from an ARDrone 2)
  • name FY value: 348.5053
    Description: Field in the y direction of the camera (Defaults are from an ARDrone 2)
  • name MAX_TRACK_CORNERS value: 25
    Description: The maximum amount of corners the Lucas Kanade algorithm is tracking between two frames
  • name WINDOW_SIZE value: 10
    Description: Window size used in Lucas Kanade algorithm
  • name SUBPIXEL_FACTOR value: 10
    Description: Amount of subpixels per pixel, used for more precise (subpixel) calculations of the flow
  • name MAX_ITERATIONS value: 10
    Description: Maximum number of iterations the Lucas Kanade algorithm should take
  • name THRESHOLD_VEC value: 2
    Description: TThreshold in subpixels when the iterations of Lucas Kanade should stop
  • name FAST9_ADAPTIVE value: TRUE
    Description: Whether we should use and adapative FAST9 crner detection threshold
  • name FAST9_THRESHOLD value: 20
    Description: FAST9 default threshold
  • name FAST9_MIN_DISTANCE value: 10
    Description: The amount of pixels between corners that should be detected

Module functions

Init Functions

These initialization functions are called once on startup.

• opticflow_module_init()

Periodic Functions

These functions are called periodically at the specified frequency from the module periodic loop.

• opticflow_module_run()
  • Running at maximum module frequency.
  • Autorun: TRUE
    Periodic function automatically starts after init.
  • Start function: opticflow_module_start()
    Executed before the periodic function starts.
  • Stop function: opticflow_module_stop()
    Executed after the periodic function stops.

Files

Header Files

The following headers are automatically included in modules.h

• opticflow_module.h

Source Files

• modules/computer_vision/lib/vision/image.c
• modules/computer_vision/lib/encoding/jpeg.c
• modules/computer_vision/lib/encoding/rtp.c
• modules/computer_vision/lib/v4l/v4l2.c
• modules/computer_vision/opticflow_module.c
• modules/computer_vision/opticflow/opticflow_calculator.c
• modules/computer_vision/opticflow/size_divergence.c
• modules/computer_vision/opticflow/linear_flow_fit.c
• math/pprz_algebra_float.c
• math/pprz_matrix_decomp_float.c
• modules/computer_vision/lib/vision/fast_rosten.c
• modules/computer_vision/lib/vision/lucas_kanade.c

Raw cv_opticflow.xml file:

<!DOCTYPE module SYSTEM "module.dtd">

<module name="cv_opticflow" dir="computer_vision">
  <doc>
    <description>
      Compute optical flow.
      Made for Linux video Devices.
      Computes x and y velocity using optical flow and distance to ground (using sonar).
    </description>

    <!-- Optical flow calculation parameters -->
    <section name="OPTICFLOW" prefix="OPTICFLOW_">
      <define name="AGL_ID" value="ABI_SENDER_ID" description="ABI sender id for AGL message (sonar measurement) (default: ABI_BROADCAST)"/>

      <!-- Video device parameters -->
      <define name="DEVICE" value="/dev/video2" description="The V4L2 camera device that is used for the calculations"/>
      <define name="DEVICE_SIZE" value="320,240" description="The V4L2 camera device width and height"/>
      <define name="DEVICE_BUFFERS" value="15" description="Amount of V4L2 video buffers"/>
      <define name="SUBDEV" description="The V4L2 subdevice to initialize before the main device"/>

      <!-- Camera parameters -->
      <define name="FOV_W" value="0.89360857702" description="The field of view width of the bottom camera (Defaults are from an ARDrone 2)"/>
      <define name="FOV_H" value="0.67020643276" description="The field of view height of the bottom camera (Defaults are from an ARDrone 2)"/>
      <define name="FX" value="343.1211" description="Field in the x direction of the camera (Defaults are from an ARDrone 2)"/>
      <define name="FY" value="348.5053" description="Field in the y direction of the camera (Defaults are from an ARDrone 2)"/>

      <!-- Lucas Kanade optical flow calculation parameters -->
      <define name="MAX_TRACK_CORNERS" value="25" description="The maximum amount of corners the Lucas Kanade algorithm is tracking between two frames"/>
      <define name="WINDOW_SIZE" value="10" description="Window size used in Lucas Kanade algorithm"/>
      <define name="SUBPIXEL_FACTOR" value="10" description="Amount of subpixels per pixel, used for more precise (subpixel) calculations of the flow"/>
      <define name="MAX_ITERATIONS" value="10" description="Maximum number of iterations the Lucas Kanade algorithm should take"/>
      <define name="THRESHOLD_VEC" value="2" description="TThreshold in subpixels when the iterations of Lucas Kanade should stop"/>

      <!-- FAST9 corner detection parameters -->
      <define name="FAST9_ADAPTIVE" value="TRUE" description="Whether we should use and adapative FAST9 crner detection threshold"/>
      <define name="FAST9_THRESHOLD" value="20" description="FAST9 default threshold"/>
      <define name="FAST9_MIN_DISTANCE" value="10" description="The amount of pixels between corners that should be detected"/>
    </section>
  </doc>

  <settings>
  <dl_settings NAME="Vision stabilization">

      <!-- Optical flow calculations parameters for Lucas Kanade and FAST9 -->
      <dl_settings name="vision_calc">
        <dl_setting var="opticflow.max_track_corners" module="computer_vision/opticflow_module" min="0" step="1" max="500" shortname="max_trck_corners" param="OPTICFLOW_MAX_TRACK_CORNERS"/>
        <dl_setting var="opticflow.window_size" module="computer_vision/opticflow_module" min="0" step="1" max="500" shortname="window_size" param="OPTICFLOW_WINDOW_SIZE"/>
        <dl_setting var="opticflow.subpixel_factor" module="computer_vision/opticflow_module" min="0" step="1" max="100" shortname="subpixel_factor" param="OPTICFLOW_SUBPIXEL_FACTOR"/>
        <dl_setting var="opticflow.max_iterations" module="computer_vision/opticflow_module" min="0" step="1" max="100" shortname="max_iterations" param="OPTICFLOW_MAX_ITERATIONS"/>
        <dl_setting var="opticflow.threshold_vec" module="computer_vision/opticflow_module" min="0" step="1" max="100" shortname="threshold_vec" param="OPTICFLOW_THRESHOLD_VEC"/>

        <dl_setting var="opticflow.fast9_adaptive" module="computer_vision/opticflow_module" min="0" step="1" max="1" values="TRUE|FALSE" shortname="fast9_adaptive" param="OPTICFLOW_FAST9_ADAPTIVE"/>
        <dl_setting var="opticflow.fast9_threshold" module="computer_vision/opticflow_module" min="0" step="1" max="255" shortname="fast9_threshold" param="OPTICFLOW_FAST9_THRESHOLD"/>
        <dl_setting var="opticflow.fast9_min_distance" module="computer_vision/opticflow_module" min="0" step="1" max="500" shortname="fast9_min_distance" param="OPTICFLOW_FAST9_MIN_DISTANCE"/>
      </dl_settings>
    </dl_settings>
  </settings>

  <header>
    <file name="opticflow_module.h"/>
  </header>

  <init fun="opticflow_module_init()"/>
  <periodic fun="opticflow_module_run()" start="opticflow_module_start()" stop="opticflow_module_stop()" autorun="TRUE"/>

  <makefile target="ap">
    <!-- Include the needed Computer Vision files -->
    <define name="modules/computer_vision" type="include"/>
    <file name="image.c" dir="modules/computer_vision/lib/vision"/>
    <file name="jpeg.c" dir="modules/computer_vision/lib/encoding"/>
    <file name="rtp.c" dir="modules/computer_vision/lib/encoding"/>
    <file name="v4l2.c" dir="modules/computer_vision/lib/v4l"/>

    <!-- The optical flow module (calculator) -->
    <file name="opticflow_module.c"/>
    <file name="opticflow_calculator.c" dir="modules/computer_vision/opticflow"/>
    <file name="size_divergence.c" dir="modules/computer_vision/opticflow"/>
    <file name="linear_flow_fit.c" dir="modules/computer_vision/opticflow"/>
    <file name="pprz_algebra_float.c" dir="math"/>
    <file name="pprz_matrix_decomp_float.c" dir="math"/>

    <!-- Main vision calculations -->
    <file name="fast_rosten.c" dir="modules/computer_vision/lib/vision"/>
    <file name="lucas_kanade.c" dir="modules/computer_vision/lib/vision"/>
  </makefile>

</module>