This example shows how OpenCV can be used on (for example) the Bebop drone.

Important to know is that sw/ext/opencv_bebop must be downloaded, and compiled. After this is done the folder sw/ext/opencv_bebop/install has an opencv.xml file. The LDFLAGS in this file should be the same as in this conf file.

Install OpenCV: WARNING! During simulation, a native version of OpenCV is needed, while for AP you need a cross-compiled version included in ./sw/ext/opencv_bebop.

The following instructions are for the AP (ARM) version:

1) For OpenCV on the bebop, make sure to use Ubuntu 20.4: This solution was only tested for this version of Ubuntu. In NPS, newer versions of OpenCV also work. Then do not follow the instructions below but install (sudo apt-get install) corresponding opencv-dev packages instead, and then skip to step 7)

2) Navigate to paparazzi/sw/ext/opencv_bebop: cd ~/paparazzi/sw/ext/opencv_bebop

3) Install the required OpenCV Libraries: sudo apt install libjpeg-turbo8-dev libpng-dev libtiff-dev zlib1g-dev libdc1394-22-dev

4) Do make (inside the current directory): make

5) Then go to the build files for the drone : (will configure the files for OpenCV on the drone) command: cd /home/username/paparazzi/sw/ext/opencv_bebop/install_arm command: make install

6) Then go to the build files for the computer: (will configure the files for OpenCV on your on computer) command: cd /home/username/paparazzi/sw/ext/opencv_bebop/install_pc command: make install

7) finally update the paths in this module file (XML file) for your drone (AP) and for your computer (NPS):

For the Bebop, check the paths are: /sw/ext/opencv_bebop/install_arm/include and lib.

Example for NPS, if you followed the steps 2 to 6 above, your opencv will normally be in /sw/ext/opencv_bebop/install_pc/include

But if you did: sudo apt-get install of opencv-dev, then you will need to update all paths to where opencv-dev is on your machine: /usr/bin/...

8) USEFUL BACKGROUND INFORMATION OpenCv only works on C++ code, therefore you need to use a special trick to call a C++ function from C:

#ifdef __cplusplus extern "C" { #endif

Example for airframe file

Add to your firmware section: This example contains all possible configuration options, not all of them are mandatory!

<module name="cv_opencvdemo">
  b'<define name="OPENCVDEMO_CAMERA" value="front_camera|bottom_camera" />\n    '
  b'<define name="OPENCVDEMO_FPS" value="0" />\n  '
</module>

Module configuration options

Define Options

name: OPENCVDEMO_CAMERA value: front_camera|bottom_camera
Description: Video device to use
name: OPENCVDEMO_FPS value: 0
Description: The (maximum) frequency to run the calculations at. If zero, it will max out at the camera frame rate

Module functions

Init Functions

These initialization functions are called once on startup.

opencvdemo_init()

Files

Header Files

The following headers are automatically included in modules.h

cv_opencvdemo.h

Source Files

Raw cv_opencvdemo.xml file:

<!DOCTYPE module SYSTEM "module.dtd">
 
<module name="cv_opencvdemo" dir="computer_vision">
  <doc>
    <description>This example shows how OpenCV can be used on (for example) the Bebop drone.
    Important to know is that sw/ext/opencv_bebop must be downloaded, and compiled.
    After this is done the folder sw/ext/opencv_bebop/install has an opencv.xml file.
    The LDFLAGS in this file should be the same as in this conf file.
 
// Install OpenCV:
WARNING! During simulation, a native version of OpenCV is needed, while for AP you need a cross-compiled version included in ./sw/ext/opencv_bebop.
 
The following instructions are for the AP (ARM) version:
 
1) For OpenCV on the bebop, make sure to use Ubuntu 20.4: This solution was only tested for this version of Ubuntu.
In NPS, newer versions of OpenCV also work. Then do not follow the instructions below but install (sudo apt-get install) corresponding opencv-dev packages instead, and then skip to step 7) 
 
2) Navigate to paparazzi/sw/ext/opencv_bebop:
cd ~/paparazzi/sw/ext/opencv_bebop
 
3) Install the required OpenCV Libraries:
sudo apt install libjpeg-turbo8-dev libpng-dev libtiff-dev zlib1g-dev libdc1394-22-dev
 
4) Do make (inside the current directory): 
make
 
5) Then go to the build files for the drone : (will configure the files for OpenCV on the drone)
command: cd /home/username/paparazzi/sw/ext/opencv_bebop/install_arm 
command: make install
 
6) Then go to the build files for the computer: (will configure the files for OpenCV on your on computer)
command: cd  /home/username/paparazzi/sw/ext/opencv_bebop/install_pc 
command: make install
 
7) finally update the paths in this module file (XML file) for your drone (AP) and for your computer (NPS):
 
For the Bebop, check the paths are: $(PAPARAZZI_SRC)/sw/ext/opencv_bebop/install_arm/include and lib.
 
Example for NPS, if you followed the steps 2 to 6 above, your opencv will normally be in $(PAPARAZZI_SRC)/sw/ext/opencv_bebop/install_pc/include
 
But if you did: sudo apt-get install of opencv-dev, then you will need to update all paths to where opencv-dev is on your machine: /usr/bin/...
 
8) USEFUL BACKGROUND INFORMATION
OpenCv only works on C++ code, therefore you need to use a special trick to call a C++ function from C:
 
#ifdef __cplusplus
extern "C" {
#endif
      
    </description>
 
    <define name="OPENCVDEMO_CAMERA" value="front_camera|bottom_camera" description="Video device to use"/>
    <define name="OPENCVDEMO_FPS" value="0" description="The (maximum) frequency to run the calculations at. If zero, it will max out at the camera frame rate"/>
  </doc>
  <header>
    <file name="cv_opencvdemo.h"/>
  </header>
  <init fun="opencvdemo_init()"/>
  <makefile target="ap">
    <file name="cv_opencvdemo.c"/>
    <file name="opencv_example.cpp"/>
    <file name="opencv_image_functions.cpp"/>
    <flag name="CXXFLAGS" value="I$(PAPARAZZI_SRC)/sw/ext/opencv_bebop/install_arm/include"/>
    
    <flag name="LDFLAGS" value="L$(PAPARAZZI_HOME)/sw/ext/opencv_bebop/install_arm/lib"/>
    <flag name="LDFLAGS" value="lopencv_world"/>
    <flag name="LDFLAGS" value="L$(PAPARAZZI_HOME)/sw/ext/opencv_bebop/install_arm/share/OpenCV/3rdparty/lib"/>
    <flag name="LDFLAGS" value="llibprotobuf"/>
    <flag name="LDFLAGS" value="llibjpeg-turbo"/>
    <flag name="LDFLAGS" value="llibpng"/>
    <flag name="LDFLAGS" value="llibtiff"/>
    <flag name="LDFLAGS" value="lzlib"/>
    <flag name="LDFLAGS" value="lquirc"/>
    <flag name="LDFLAGS" value="ltegra_hal"/>
    <flag name="LDFLAGS" value="ldl"/>
    <flag name="LDFLAGS" value="lm"/>
    <flag name="LDFLAGS" value="lpthread"/>
    <flag name="LDFLAGS" value="lrt"/>
  </makefile>
  <makefile target="nps">
    <file name="cv_opencvdemo.c"/>
    <file name="opencv_example.cpp"/>
    <file name="opencv_image_functions.cpp"/>
    
    <flag name="CXXFLAGS" value="I$(PAPARAZZI_SRC)/sw/ext/opencv_bebop/install_pc/include"/>
    
    <flag name="LDFLAGS" value="L$(PAPARAZZI_HOME)/sw/ext/opencv_bebop/install_pc/lib"/>
    <flag name="LDFLAGS" value="lopencv_world"/>
    <flag name="LDFLAGS" value="L$(PAPARAZZI_HOME)/sw/ext/opencv_bebop/install_pc/share/OpenCV/3rdparty/lib"/>
    <flag name="LDFLAGS" value="llibprotobuf"/>
    <flag name="LDFLAGS" value="lquirc"/>
    <flag name="LDFLAGS" value="L/usr/lib/x86_64-linux-gnu"/>
    <flag name="LDFLAGS" value="ljpeg"/>
    <flag name="LDFLAGS" value="lpng"/>
    <flag name="LDFLAGS" value="ltiff"/>
    <flag name="LDFLAGS" value="L/usr/lib/x86_64-linux-gnu/hdf5/serial"/>
    <flag name="LDFLAGS" value="lhdf5"/>
    <flag name="LDFLAGS" value="lpthread"/>
    <flag name="LDFLAGS" value="lsz"/>
    <flag name="LDFLAGS" value="lz"/>
    <flag name="LDFLAGS" value="ldl"/>
    <flag name="LDFLAGS" value="lm"/>
    <flag name="LDFLAGS" value="lfreetype"/>
    <flag name="LDFLAGS" value="lharfbuzz"/>
    <flag name="LDFLAGS" value="lrt"/>
  </makefile>
</module>