Unwrap images taken through a panoramic lens

Left: AR.Drone 2.0 with Kogeto Dot 360 panoramic lens on bottom camera. Middle: raw image and region of interest (blue). Right: unwrapped image.

This module unwraps images taken through a panoramic lens. The unwrapped image is returned to the video thread or available as pano_unwrapped_image.

The user should specify the region of interest (the part of the raw image that should be unwrapped) by setting the following values:

The center of the lens (center_x and center_y)
The radii of the top and bottom of the ROI (radius_bottom and radius_top)
The body +x direction in the raw image (forward_direction)
The body +y direction in the raw image (flip_horizontal) The region of interest can be shown by setting show_calibration to TRUE and overwrite_video to FALSE.

The size of the output image can be set with width and height. The height of the image can also be set to zero, in which case it is determined automatically from the vertical_resolution.

Optionally, the image can be derotated to cancel the pitch and roll movements of the drone. This requires a correct setting of the vertical_resolution. Reasonable values can be found by looking at the unwrapped and derotated image and adjusting the vertical resolution until the environment no longer appears to move during pitch and roll motions.

The following assumptions are made:

The vertical resolution is constant throughout the region of interest.
Pitch and roll angles are small (for derotation).

Derotation (shown for large roll angle).

Example for airframe file

Add to your firmware section: This example contains all possible configuration options, not all of them are mandatory!

<module name="pano_unwrap">
  b'<define name="PANO_UNWRAP_CAMERA" value="front_camera|bottom_camera (default)" />\n    '
  b'<define name="PANO_UNWRAP_CENTER_X" value="0.50" />\n    '
  b'<define name="PANO_UNWRAP_CENTER_Y" value="0.50" />\n    '
  b'<define name="PANO_UNWRAP_RADIUS_BOTTOM" value="0.20" />\n    '
  b'<define name="PANO_UNWRAP_RADIUS_TOP" value="0.30" />\n    '
  b'<define name="PANO_UNWRAP_FORWARD_DIRECTION" value="270.0" />\n    '
  b'<define name="PANO_UNWRAP_FLIP_HORIZONTAL" value="TRUE|FALSE (default)" />\n    '
  b'<define name="PANO_UNWRAP_VERTICAL_RESOLUTION" value="0.18" />\n    '
  b'<define name="PANO_UNWRAP_DEROTATE_ATTITUDE" value="TRUE|FALSE (default)" />\n    '
  b'<define name="PANO_UNWRAP_WIDTH" value="640" />\n    '
  b'<define name="PANO_UNWRAP_HEIGHT" value="0" />\n    '
  b'<define name="PANO_UNWRAP_OVERWRITE_VIDEO_THREAD" value="TRUE (default)|FALSE" />\n    '
  b'<define name="PANO_UNWRAP_FPS" value="0" />\n  '
</module>

Module configuration options

Define Options

name: PANO_UNWRAP_CAMERA value: front_camera|bottom_camera (default)
Description: Camera to use
name: PANO_UNWRAP_CENTER_X value: 0.50
Description: Center of lens in raw image [fraction of raw image width]
name: PANO_UNWRAP_CENTER_Y value: 0.50
Description: Center of lens in raw image [fraction of raw image height]
name: PANO_UNWRAP_RADIUS_BOTTOM value: 0.20
Description: Distance from the lens center to the bottom of the region of interest [fraction of raw image height]
name: PANO_UNWRAP_RADIUS_TOP value: 0.30
Description: Distance from lens center to the top of the region of interest [fraction of raw image height]
name: PANO_UNWRAP_FORWARD_DIRECTION value: 270.0
Description: Forward direction in raw image measured counterclockwise from right [deg]
name: PANO_UNWRAP_FLIP_HORIZONTAL value: TRUE|FALSE (default)
Description: Flip output image horizontally
name: PANO_UNWRAP_VERTICAL_RESOLUTION value: 0.18
Description: Vertical resolution of region of interest [fraction of raw image height/rad]
name: PANO_UNWRAP_DEROTATE_ATTITUDE value: TRUE|FALSE (default)
Description: Derotate the image, i.e. keep the panorama aligned with the horizon
name: PANO_UNWRAP_WIDTH value: 640
Description: Width of the unwrapped image [px]
name: PANO_UNWRAP_HEIGHT value: 0
Description: Height of the unwrapped image [px]. Set to 0 to determine automatically.
name: PANO_UNWRAP_OVERWRITE_VIDEO_THREAD value: TRUE (default)|FALSE
Description: Write unwrapped image to the video thread
name: PANO_UNWRAP_FPS value: 0
Description: Maximum FPS (0: unlimited)

Module functions

Init Functions

These initialization functions are called once on startup.

pano_unwrap_init()

Files

Header Files

The following headers are automatically included in modules.h

pano_unwrap.h

Source Files

modules/pano_unwrap/pano_unwrap.c

Raw pano_unwrap.xml file:

<!DOCTYPE module SYSTEM "module.dtd">
 
<module name="pano_unwrap">
  <doc>
    <description>
    Unwrap images taken through a panoramic lens
    
    @image html images/airborne/modules/pano_unwrap/pano_unwrap.jpg "Left: AR.Drone 2.0 with Kogeto Dot 360 panoramic lens on bottom camera. Middle: raw image and region of interest (blue). Right: unwrapped image."
    
    This module unwraps images taken through a panoramic lens. The unwrapped
    image is returned to the video thread or available as
    `pano_unwrapped_image`.
    
    The user should specify the region of interest (the part of the raw image
    that should be unwrapped) by setting the following values:
    - The center of the lens (`center_x` and `center_y`)
    - The radii of the top and bottom of the ROI (`radius_bottom` and
      `radius_top`)
    - The body +x direction in the raw image (`forward_direction`)
    - The body +y direction in the raw image (`flip_horizontal`)
    The region of interest can be shown by setting `show_calibration` to TRUE
    and `overwrite_video` to FALSE.
    
    The size of the output image can be set with `width` and `height`. The
    height of the image can also be set to zero, in which case it is determined
    automatically from the `vertical_resolution`.
    
    Optionally, the image can be derotated to cancel the pitch and roll
    movements of the drone. This requires a correct setting of the
    `vertical_resolution`. Reasonable values can be found by looking at the
    unwrapped and derotated image and adjusting the vertical resolution until
    the environment no longer appears to move during pitch and roll motions.
    
    The following assumptions are made:
    - The vertical resolution is constant throughout the region of interest.
    - Pitch and roll angles are small (for derotation).
    
    @image html images/airborne/modules/pano_unwrap/unwrapped_derotated.png "Derotation (shown for large roll angle)."
    </description>
    <define name="PANO_UNWRAP_CAMERA" value="front_camera|bottom_camera (default)" description="Camera to use"/>
    <define name="PANO_UNWRAP_CENTER_X" value="0.50" description="Center of lens in raw image [fraction of raw image width]"/>
    <define name="PANO_UNWRAP_CENTER_Y" value="0.50" description="Center of lens in raw image [fraction of raw image height]"/>
    <define name="PANO_UNWRAP_RADIUS_BOTTOM" value="0.20" description="Distance from the lens center to the bottom of the region of interest [fraction of raw image height]"/>
    <define name="PANO_UNWRAP_RADIUS_TOP" value="0.30" description="Distance from lens center to the top of the region of interest [fraction of raw image height]"/>
    <define name="PANO_UNWRAP_FORWARD_DIRECTION" value="270.0" description="Forward direction in raw image measured counterclockwise from right [deg]"/>
    <define name="PANO_UNWRAP_FLIP_HORIZONTAL" value="TRUE|FALSE (default)" description="Flip output image horizontally"/>
    <define name="PANO_UNWRAP_VERTICAL_RESOLUTION" value="0.18" description="Vertical resolution of region of interest [fraction of raw image height/rad]"/>
    <define name="PANO_UNWRAP_DEROTATE_ATTITUDE" value="TRUE|FALSE (default)" description="Derotate the image, i.e. keep the panorama aligned with the horizon"/>
    <define name="PANO_UNWRAP_WIDTH" value="640" description="Width of the unwrapped image [px]"/>
    <define name="PANO_UNWRAP_HEIGHT" value="0" description="Height of the unwrapped image [px]. Set to 0 to determine automatically."/>
    <define name="PANO_UNWRAP_OVERWRITE_VIDEO_THREAD" value="TRUE (default)|FALSE" description="Write unwrapped image to the video thread"/>
    <define name="PANO_UNWRAP_FPS" value="0" description="Maximum FPS (0: unlimited)"/>
  </doc>
  <settings>
    <dl_settings>
      <dl_settings name="pano_unwrap">
        <dl_setting shortname="center_x" var="pano_unwrap.center.x" min="0.0" step="0.01" max="1.0"/>
        <dl_setting shortname="center_y" var="pano_unwrap.center.y" min="0.0" step="0.01" max="1.0"/>
        <dl_setting shortname="radius_bottom" var="pano_unwrap.radius_bottom" min="0.0" step="0.01" max="1.0"/>
        <dl_setting shortname="radius_top" var="pano_unwrap.radius_top" min="0.0" step="0.01" max="1.0"/>
        <dl_setting shortname="forward_direction" var="pano_unwrap.forward_direction" min="0" step="1" max="360"/>
        <dl_setting shortname="flip_hor" var="pano_unwrap.flip_horizontal" type="uint8" values="FALSE|TRUE" min="0" step="1" max="1"/>
        <dl_setting shortname="v_resolution" var="pano_unwrap.vertical_resolution" min="-0.2" step="0.01" max="0.2"/>
        <dl_setting shortname="derotate" var="pano_unwrap.derotate_attitude" type="uint8" values="FALSE|TRUE" min="0" step="1" max="1"/>
        <dl_setting shortname="width" var="pano_unwrap.width" min="1" step="1" max="1920"/>
        <dl_setting shortname="height" var="pano_unwrap.height" min="0" step="1" max="1080"/>
        <dl_setting shortname="overwrite_video" var="pano_unwrap.overwrite_video_thread" type="uint8" values="FALSE|TRUE" min="0" step="1" max="1"/>
        <dl_setting shortname="show_calibration" var="pano_unwrap.show_calibration" type="uint8" values="FALSE|TRUE" min="0" step="1" max="1"/>
      </dl_settings>
    </dl_settings>
  </settings>
  <dep>
    <depends>video_thread</depends>
  </dep>
  <header>
    <file name="pano_unwrap.h"/>
  </header>
  <init fun="pano_unwrap_init()"/>
  <makefile>
    <file name="pano_unwrap.c"/>
    <define name="CV_ALLOW_VIDEO_TO_CHANGE_SIZE" value="1"/>
  </makefile>
</module>
 