Paparazzi UAS  v5.18.0_stable
Paparazzi is a free software Unmanned Aircraft System.
opticflow_calculator.h File Reference

Calculate velocity from optic flow. More...

#include "std.h"
#include "inter_thread_data.h"
#include "lib/vision/image.h"
#include "lib/v4l/v4l2.h"
+ Include dependency graph for opticflow_calculator.h:
+ This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Data Structures

struct  opticflow_t
 

Macros

#define FAST9_MAX_CORNERS   512
 

Functions

void opticflow_calc_init (struct opticflow_t opticflow[])
 Initialize the opticflow calculator. More...
 
bool opticflow_calc_frame (struct opticflow_t *opticflow, struct image_t *img, struct opticflow_result_t *result)
 Run the optical flow on a new image frame. More...
 
bool calc_fast9_lukas_kanade (struct opticflow_t *opticflow, struct image_t *img, struct opticflow_result_t *result)
 Run the optical flow with fast9 and lukaskanade on a new image frame. More...
 
bool calc_edgeflow_tot (struct opticflow_t *opticflow, struct image_t *img, struct opticflow_result_t *result)
 Run the optical flow with EDGEFLOW on a new image frame. More...
 
void kalman_filter_opticflow_velocity (float *velocity_x, float *velocity_y, float *acceleration_measurement, float fps, float *measurement_noise, float process_noise, bool reinitialize_kalman)
 

Detailed Description

Calculate velocity from optic flow.

Using images from a vertical camera and IMU sensor data.

Definition in file opticflow_calculator.h.


Data Structure Documentation

◆ opticflow_t

struct opticflow_t

Definition at line 40 of file opticflow_calculator.h.

+ Collaboration diagram for opticflow_t:
Data Fields
int actfast_gradient_method Whether to use a simple or Sobel filter.
float actfast_long_step Step size to take when there is no texture.
int actfast_min_gradient Threshold that decides when there is sufficient texture for edge following.
float actfast_short_step Step size to take when there is an edge to be followed.
const struct video_config_t * camera
uint8_t corner_method Method to use for determining where the corners are.
bool derotation Derotation switched on or off (depended on the quality of the gyroscope measurement)
float derotation_correction_factor_x Correction factor for derotation in x axis, determined from a fit from the gyros and flow rotation. (wrong FOV, camera not in center)
float derotation_correction_factor_y Correction factor for derotation in Y axis, determined from a fit from the gyros and flow rotation. (wrong FOV, camera not in center)
bool fast9_adaptive Whether the FAST9 threshold should be adaptive.
uint16_t fast9_min_distance Minimum distance in pixels between corners.
uint8_t fast9_num_regions The number of regions of interest the image is split into.
uint16_t fast9_padding Padding used in FAST9 detector.
bool fast9_region_detect Decides whether to detect fast9 corners in specific regions of interest or the whole image (only for feature management)
struct point_t * fast9_ret_corners Corners.
uint16_t fast9_rsize Amount of corners allocated.
uint8_t fast9_threshold FAST9 corner detection threshold.
bool feature_management Decides whether to keep track corners in memory for the next frame instead of re-detecting every time.
bool got_first_img If we got a image to work with.
uint8_t id
struct image_t img_gray Current gray image frame.
bool just_switched_method Boolean to check if methods has been switched (for reinitialization)
uint8_t max_iterations The maximum amount of iterations the Lucas Kanade algorithm should do.
uint16_t max_track_corners Maximum amount of corners Lucas Kanade should track.
bool median_filter Decides to use a median filter on the velocity.
uint8_t method Method to use to calculate the optical flow.
struct image_t prev_img_gray Previous gray image frame.
uint8_t pyramid_level Number of pyramid levels used in Lucas Kanade algorithm (0 == no pyramids used)
uint16_t resolution_factor The resolution in EdgeFlow to determine the Divergence.
uint16_t search_distance Search distance for blockmatching alg.
bool show_flow Whether to draw the flow vectors on the image. Watch out! This changes the image as will be received by subsequent processing steps.
uint16_t subpixel_factor The amount of subpixels per pixel.
uint8_t threshold_vec The threshold in x, y subpixels which the algorithm should stop.
bool track_back Whether to track flow vectors back to the previous image, in order to check if the back-tracked flow leads to the original corner position.
uint16_t window_size Window size for the blockmatching algorithm (general value for all methods)

Macro Definition Documentation

◆ FAST9_MAX_CORNERS

#define FAST9_MAX_CORNERS   512

Definition at line 86 of file opticflow_calculator.h.

Function Documentation

◆ calc_edgeflow_tot()

bool calc_edgeflow_tot ( struct opticflow_t opticflow,
struct image_t img,
struct opticflow_result_t result 
)

Run the optical flow with EDGEFLOW on a new image frame.

Parameters
[in]*opticflowThe opticalflow structure that keeps track of previous images
[in]*stateThe state of the drone
[in]*imgThe image frame to calculate the optical flow from
[out]*resultThe optical flow result
computationsuccessful

Definition at line 1006 of file opticflow_calculator.c.

References agl_dist_value_filtered, calc_previous_frame_nr(), calculate_edge_displacement(), calculate_edge_histogram(), opticflow_t::camera, opticflow_result_t::camera_id, video_config_t::camera_intrinsics, opticflow_result_t::corner_cnt, opticflow_t::derotation, opticflow_t::derotation_correction_factor_x, opticflow_t::derotation_correction_factor_y, DISP_RANGE_MAX, opticflow_result_t::div_size, edge_flow_t::div_x, edge_flow_t::div_y, opticflow_result_t::divergence, draw_edgeflow_img(), image_t::eulers, edge_hist_t::eulers, FLOAT_EULERS_ZERO, opticflow_result_t::flow_der_x, opticflow_result_t::flow_der_y, opticflow_result_t::flow_x, edge_flow_t::flow_x, opticflow_result_t::flow_y, edge_flow_t::flow_y, camera_intrinsics_t::focal_x, camera_intrinsics_t::focal_y, opticflow_result_t::fps, edge_hist_t::frame_time, getAmountPeaks(), image_t::h, opticflow_t::id, opticflow_t::just_switched_method, line_fit(), MAX_HORIZON, MAX_WINDOW_SIZE, opticflow_t::median_filter, opticflow_result_t::noise_measurement, opticflow, FloatEulers::phi, RES, opticflow_t::resolution_factor, opticflow_t::search_distance, opticflow_t::show_flow, opticflow_result_t::surface_roughness, FloatEulers::theta, timeval_diff(), opticflow_result_t::tracked_cnt, image_t::ts, UpdateMedianFilterVect3Float, opticflow_result_t::vel_cam, vel_filt, image_t::w, opticflow_t::window_size, FloatVect3::x, edge_hist_t::x, edgeflow_displacement_t::x, FloatVect3::y, edge_hist_t::y, edgeflow_displacement_t::y, and FloatVect3::z.

Referenced by opticflow_calc_frame().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ calc_fast9_lukas_kanade()

bool calc_fast9_lukas_kanade ( struct opticflow_t opticflow,
struct image_t img,
struct opticflow_result_t result 
)

Run the optical flow with fast9 and lukaskanade on a new image frame.

Parameters
[in]*opticflowThe opticalflow structure that keeps track of previous images
[in]*stateThe state of the drone
[in]*imgThe image frame to calculate the optical flow from
[out]*resultThe optical flow result
Returns
Was optical flow successful

Definition at line 492 of file opticflow_calculator.c.

References act_fast(), ACT_FAST, opticflow_t::actfast_gradient_method, opticflow_t::actfast_long_step, opticflow_t::actfast_min_gradient, opticflow_t::actfast_short_step, agl_dist_value_filtered, analyze_linear_flow_field(), bottom_camera, opticflow_t::camera, opticflow_result_t::camera_id, video_config_t::camera_intrinsics, cmp_flow(), opticflow_result_t::corner_cnt, opticflow_t::corner_method, point_t::count, opticflow_t::derotation, opticflow_t::derotation_correction_factor_x, opticflow_t::derotation_correction_factor_y, video_config_t::dev_name, opticflow_result_t::div_size, linear_flow_fit_info::divergence, opticflow_result_t::divergence, flow_t::error, image_t::eulers, EXHAUSTIVE_FAST, f, opticflow_t::fast9_adaptive, fast9_detect(), FAST9_HIGH_THRESHOLD, FAST9_LOW_THRESHOLD, opticflow_t::fast9_min_distance, opticflow_t::fast9_padding, opticflow_t::fast9_ret_corners, opticflow_t::fast9_rsize, opticflow_t::fast9_threshold, opticflow_t::feature_management, opticflow_result_t::flow_der_x, opticflow_result_t::flow_der_y, opticflow_result_t::flow_x, flow_t::flow_x, opticflow_result_t::flow_y, flow_t::flow_y, camera_intrinsics_t::focal_x, camera_intrinsics_t::focal_y, opticflow_result_t::fps, get_size_divergence(), opticflow_t::got_first_img, image_t::h, opticflow_t::id, image_copy(), image_create(), IMAGE_GRAYSCALE, image_show_flow_color(), image_show_points(), image_switch(), image_to_grayscale(), opticflow_t::img_gray, InitMedianFilterVect3Float, opticflow_t::just_switched_method, LARGE_FLOW_ERROR, LINEAR_FIT, manage_flow_features(), opticflow_t::max_iterations, opticflow_t::max_track_corners, MEDIAN_DEFAULT_SIZE, opticflow_t::median_filter, n_agents, n_samples, n_time_steps, opticflow_result_t::noise_measurement, opticflow, opticFlowLK(), FloatEulers::phi, flow_t::pos, predict_flow_vectors(), opticflow_t::prev_img_gray, FloatEulers::psi, opticflow_t::pyramid_level, opticflow_t::show_flow, SIZE_DIV, opticflow_t::subpixel_factor, linear_flow_fit_info::surface_roughness, opticflow_result_t::surface_roughness, FloatEulers::theta, opticflow_t::threshold_vec, timeval_diff(), opticflow_t::track_back, opticflow_result_t::tracked_cnt, image_t::ts, UpdateMedianFilterVect3Float, VECT3_ASSIGN, opticflow_result_t::vel_body, opticflow_result_t::vel_cam, vel_filt, image_t::w, opticflow_t::window_size, FloatVect3::x, point_t::x, point_t::x_sub, FloatVect3::y, point_t::y, point_t::y_sub, and FloatVect3::z.

Referenced by opticflow_calc_frame().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ kalman_filter_opticflow_velocity()

void kalman_filter_opticflow_velocity ( float *  velocity_x,
float *  velocity_y,
float *  acceleration_measurement,
float  fps,
float *  measurement_noise,
float  process_noise,
bool  reinitialize_kalman 
)

◆ opticflow_calc_frame()

bool opticflow_calc_frame ( struct opticflow_t opticflow,
struct image_t img,
struct opticflow_result_t result 
)

Run the optical flow on a new image frame.

Parameters
[in]*opticflowThe opticalflow structure that keeps track of previous images
[in]*stateThe state of the drone
[in]*imgThe image frame to calculate the optical flow from
[out]*resultThe optical flow result

Definition at line 1169 of file opticflow_calculator.c.

References body_to_cam, calc_edgeflow_tot(), calc_fast9_lukas_kanade(), float_rmat_transp_vmult(), opticflow_t::id, opticflow_t::just_switched_method, opticflow_t::method, opticflow, opticflow_result_t::vel_body, and opticflow_result_t::vel_cam.

Referenced by opticflow_module_calc().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ opticflow_calc_init()

void opticflow_calc_init ( struct opticflow_t  opticflow[])

Initialize the opticflow calculator.

Parameters
[out]*opticflowThe new optical flow calculator

Definition at line 396 of file opticflow_calculator.c.

References opticflow_t::actfast_gradient_method, opticflow_t::actfast_long_step, opticflow_t::actfast_min_gradient, opticflow_t::actfast_short_step, body_to_cam, opticflow_t::camera, opticflow_t::corner_method, opticflow_t::derotation, opticflow_t::derotation_correction_factor_x, opticflow_t::derotation_correction_factor_y, opticflow_t::fast9_adaptive, FAST9_MAX_CORNERS, opticflow_t::fast9_min_distance, opticflow_t::fast9_num_regions, opticflow_t::fast9_padding, opticflow_t::fast9_region_detect, opticflow_t::fast9_ret_corners, opticflow_t::fast9_rsize, opticflow_t::fast9_threshold, opticflow_t::feature_management, float_rmat_of_eulers, opticflow_t::id, opticflow_t::max_iterations, opticflow_t::max_track_corners, opticflow_t::median_filter, opticflow_t::method, opticflow, OPTICFLOW_ACTFAST_GRADIENT_METHOD, OPTICFLOW_ACTFAST_GRADIENT_METHOD_CAMERA2, OPTICFLOW_ACTFAST_LONG_STEP, OPTICFLOW_ACTFAST_LONG_STEP_CAMERA2, OPTICFLOW_ACTFAST_MIN_GRADIENT, OPTICFLOW_ACTFAST_MIN_GRADIENT_CAMERA2, OPTICFLOW_ACTFAST_SHORT_STEP, OPTICFLOW_ACTFAST_SHORT_STEP_CAMERA2, OPTICFLOW_BODY_TO_CAM_PHI, OPTICFLOW_BODY_TO_CAM_PHI_CAMERA2, OPTICFLOW_BODY_TO_CAM_PSI, OPTICFLOW_BODY_TO_CAM_PSI_CAMERA2, OPTICFLOW_BODY_TO_CAM_THETA, OPTICFLOW_BODY_TO_CAM_THETA_CAMERA2, OPTICFLOW_CORNER_METHOD, OPTICFLOW_CORNER_METHOD_CAMERA2, OPTICFLOW_DEROTATION, OPTICFLOW_DEROTATION_CAMERA2, OPTICFLOW_DEROTATION_CORRECTION_FACTOR_X, OPTICFLOW_DEROTATION_CORRECTION_FACTOR_X_CAMERA2, OPTICFLOW_DEROTATION_CORRECTION_FACTOR_Y, OPTICFLOW_DEROTATION_CORRECTION_FACTOR_Y_CAMERA2, OPTICFLOW_FAST9_ADAPTIVE, OPTICFLOW_FAST9_ADAPTIVE_CAMERA2, OPTICFLOW_FAST9_MIN_DISTANCE, OPTICFLOW_FAST9_MIN_DISTANCE_CAMERA2, OPTICFLOW_FAST9_NUM_REGIONS, OPTICFLOW_FAST9_NUM_REGIONS_CAMERA2, OPTICFLOW_FAST9_PADDING, OPTICFLOW_FAST9_PADDING_CAMERA2, OPTICFLOW_FAST9_REGION_DETECT, OPTICFLOW_FAST9_REGION_DETECT_CAMERA2, OPTICFLOW_FAST9_THRESHOLD, OPTICFLOW_FAST9_THRESHOLD_CAMERA2, OPTICFLOW_FEATURE_MANAGEMENT, OPTICFLOW_FEATURE_MANAGEMENT_CAMERA2, OPTICFLOW_MAX_ITERATIONS, OPTICFLOW_MAX_ITERATIONS_CAMERA2, OPTICFLOW_MAX_TRACK_CORNERS, OPTICFLOW_MAX_TRACK_CORNERS_CAMERA2, OPTICFLOW_MEDIAN_FILTER, OPTICFLOW_MEDIAN_FILTER_CAMERA2, OPTICFLOW_METHOD, OPTICFLOW_METHOD_CAMERA2, OPTICFLOW_PYRAMID_LEVEL, OPTICFLOW_PYRAMID_LEVEL_CAMERA2, OPTICFLOW_RESOLUTION_FACTOR, OPTICFLOW_RESOLUTION_FACTOR_CAMERA2, OPTICFLOW_SEARCH_DISTANCE, OPTICFLOW_SEARCH_DISTANCE_CAMERA2, OPTICFLOW_SHOW_FLOW, OPTICFLOW_SHOW_FLOW_CAMERA2, OPTICFLOW_SUBPIXEL_FACTOR, OPTICFLOW_SUBPIXEL_FACTOR_CAMERA2, OPTICFLOW_THRESHOLD_VEC, OPTICFLOW_THRESHOLD_VEC_CAMERA2, OPTICFLOW_TRACK_BACK, OPTICFLOW_TRACK_BACK_CAMERA2, OPTICFLOW_WINDOW_SIZE, OPTICFLOW_WINDOW_SIZE_CAMERA2, opticflow_t::pyramid_level, opticflow_t::resolution_factor, opticflow_t::search_distance, opticflow_t::show_flow, opticflow_t::subpixel_factor, opticflow_t::threshold_vec, opticflow_t::track_back, and opticflow_t::window_size.

Referenced by opticflow_module_init().

+ Here is the caller graph for this function: