CN113450418A

CN113450418A - Improved method, device and system for underwater calibration based on complex distortion model

Info

Publication number: CN113450418A
Application number: CN202110707714.6A
Authority: CN
Inventors: 范伟平
Original assignee: Shenzhen Tomorrow System Integration Co ltd
Current assignee: Shenzhen Tomorrow System Integration Co ltd
Priority date: 2021-06-24
Filing date: 2021-06-24
Publication date: 2021-09-28

Abstract

The invention relates to the technical field of image measurement, in particular to an improved method, device and system for underwater calibration based on a complex distortion model. The method, the device and the system respectively acquire the air original image and the underwater original image when receiving the camera trigger signal; performing image processing operation on the air original image and the underwater original image based on a machine vision processing algorithm to obtain a processed air calibration image and a processed underwater calibration image; respectively carrying out calibration calculation on the underwater calibration image and the air calibration image by adopting a multi-order radial tangential distortion model to obtain an internal parameter, an external parameter, a radial distortion parameter and a tangential distortion parameter of the camera; the method, the device and the system for improving the underwater calibration based on the complex distortion model can improve the reliability of the calibration result.

Description

Improved method, device and system for underwater calibration based on complex distortion model

Technical Field

The invention relates to the technical field of image measurement, in particular to an improved method, device and system for underwater calibration based on a complex distortion model.

Background

With the development of modern image processing technology, in machine vision, in order to realize the calculation of various model algorithm formulas therein, parameters in the whole model must be known, and the process of obtaining various preset parameters of the whole model is called calibration; the accuracy degree of the whole calibration process directly determines the accuracy of the subsequent measurement of the bubbles moving at high speed underwater, so that the effective and reliable calibration technology can ensure the accuracy of the measurement of the bubbles moving at high speed underwater, the existing calibration technology is generally calibrated by a first-order radial distortion model, and the first-order radial distortion model cannot adapt to light path transformation and light path deviation caused by the change of a light path transmission medium in the underwater calibration process, so that the reliability of a calibration result is low, and the accuracy of the subsequent measurement of the bubbles moving at high speed underwater is reduced.

Disclosure of Invention

The embodiment of the invention provides an improved method, device and system for underwater calibration based on a complex distortion model, which at least solve the technical problem of low reliability of a calibration result caused by the fact that the existing calibration technology cannot adapt to underwater calibration with changed light paths.

According to an embodiment of the present invention, there is provided an improved method for underwater calibration based on a complex distortion model, including the following steps:

when a camera trigger signal is received, acquiring an air original image and an underwater original image respectively;

performing image processing operation on the air original image and the underwater original image based on a machine vision processing algorithm to obtain a processed air calibration image and a processed underwater calibration image;

respectively carrying out calibration calculation on the underwater calibration image and the air calibration image by adopting a multi-order radial tangential distortion model to obtain an internal parameter, an external parameter, a radial distortion parameter and a tangential distortion parameter of the camera;

and adjusting the original parameters of the camera by utilizing the internal and external parameters, the radial distortion parameters and the tangential distortion parameters to finish the underwater calibration of the camera.

Further, the multi-order radial tangential distortion model is composed of a three-order radial distortion model and a two-order tangential distortion model.

Further, the step of performing image processing operation on the air original image and the underwater original image based on a machine vision processing algorithm to obtain a processed air calibration image and a processed underwater calibration image comprises:

respectively filtering the air original image and the underwater original image to obtain a processed air smooth image and a processed underwater smooth image;

respectively carrying out basic edge detection on the air smooth image and the underwater smooth image to obtain an air basic edge image and an underwater basic edge image;

respectively carrying out target edge detection on the air base edge image and the underwater base edge image to obtain an air clear edge image and an underwater clear edge image;

and respectively carrying out characteristic information processing operation on the air clear edge image and the underwater clear edge image to obtain an air calibration image and an underwater calibration image.

Further, the method further comprises:

establishing an image acquisition thread between a camera and a platform provided with a machine vision processing algorithm by using the set logical link;

and carrying out setting operation of synchronous primitives with the image acquisition thread on the camera so as to acquire high-precision images.

Further, the method further comprises:

acquiring a test image in a database;

carrying out target tracking operation on the test image to obtain an imaging target in the test image;

and calculating the view range of the camera according to the range calculation formula.

Further, the method further comprises:

reading calibration parameters for completing underwater calibration of the camera;

when the calibration parameters are read out, element extraction is carried out on the shot underwater bubble image according to the calibration parameters to obtain a target image containing element information;

and according to the geometrical coordinate relation, carrying out coordinate conversion and calculation on the element information to obtain the size information corresponding to the element information in the target image.

According to another embodiment of the present invention, there is provided an improved apparatus for underwater calibration based on a complex distortion model, including:

the image acquisition module is used for acquiring the air original image and the underwater original image respectively when receiving the camera trigger signal;

the image processing module is used for carrying out image processing operation on the air original image and the underwater original image based on a machine vision processing algorithm to obtain a processed air calibration image and a processed underwater calibration image;

the calibration calculation module is used for respectively performing calibration calculation on the underwater calibration image and the air calibration image by adopting a multi-order radial tangential distortion model to obtain an internal parameter, an external parameter, a radial distortion parameter and a tangential distortion parameter of the camera;

and the parameter adjusting module is used for adjusting the original parameters of the camera by utilizing the internal and external parameters, the radial distortion parameters and the tangential distortion parameters so as to finish the underwater calibration of the camera.

Further, the image processing module includes:

the filtering processing unit is used for respectively carrying out filtering processing on the air original image and the underwater original image to obtain a processed air smooth image and a processed underwater smooth image;

the basic edge detection unit is used for respectively carrying out basic edge detection on the air smooth image and the underwater smooth image to obtain an air basic edge image and an underwater basic edge image;

the target edge detection unit is used for respectively carrying out target edge detection on the air base edge image and the underwater base edge image to obtain an air clear edge image and an underwater clear edge image;

and the characteristic processing unit is used for respectively carrying out characteristic information processing operation on the air clear edge image and the underwater clear edge image to obtain an air calibration image and an underwater calibration image.

Further, the apparatus further comprises:

the acquisition thread establishing module is used for establishing an image acquisition thread between the camera and the HALCON by utilizing the set logical link;

and the thread synchronous setting module is used for setting primitives synchronous with the image acquisition thread for the camera so as to acquire high-precision images.

According to another embodiment of the present invention, there is provided an improved system for underwater calibration based on a complex distortion model, including:

the system comprises a machine vision system, a light source lighting device and a calibration board, wherein the machine vision system comprises a camera and a machine vision processing algorithm platform;

the camera is matched with the light source lighting device and used for acquiring the air original image and the underwater original image when receiving a camera trigger signal;

the machine vision processing algorithm platform is used for carrying out image processing operation on the air original image and the underwater original image based on a machine vision processing algorithm to obtain a processed air calibration image and a processed underwater calibration image;

the machine vision processing algorithm platform is also used for respectively carrying out calibration calculation on the underwater calibration image and the air calibration image by adopting a multi-order radial tangential distortion model to obtain an internal parameter, an external parameter, a radial distortion parameter and a tangential distortion parameter of the camera;

the camera is also used for adjusting the original parameters of the camera by utilizing the internal and external parameters, the radial distortion parameters and the tangential distortion parameters so as to finish the underwater calibration of the camera;

the calibration plate is used for shooting the calibration plate through a camera in cooperation with the light source lighting device to obtain a calibration image.

According to the method, the device and the system for improving the underwater calibration based on the complex distortion model, the collected air original image and the collected underwater original image are subjected to image processing operation based on the machine vision processing algorithm, so that the air calibration image and the underwater calibration image are rapidly and accurately output, the images can be rapidly and accurately processed, repeated processing of the images is avoided, and the processing efficiency is ensured; then, the underwater calibration image and the air calibration image are respectively calibrated and calculated by adopting a multi-order radial tangential distortion model, and the underwater calibration image and the air calibration image can be used for compensating the light path change caused by the change of a light path transmission medium so as to reduce the deviation in calculation and obtain the internal and external parameters, the radial distortion parameter and the tangential distortion parameter of the camera with higher reliability; furthermore, the original parameters of the camera are adjusted by utilizing the internal and external parameters, the radial distortion parameters and the tangential distortion parameters, so that the reliability of the calibration result of the underwater calibration of the camera is high, the precision of measuring bubbles moving underwater at high speed can be ensured to a certain extent, and the method, the device and the system for improving the underwater calibration based on the complex distortion model can improve the reliability of the calibration result.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:

FIG. 1 is a flow chart of an improved method of underwater calibration based on a complex distortion model of the present invention;

FIG. 2 is a flow chart of the image processing based on the machine vision processing algorithm of the improved method of underwater calibration based on complex distortion model of the present invention;

FIG. 3 is a flow chart of the method of the present invention for improved underwater calibration based on complex distortion models to establish the relationship between the camera and the platform on which the machine vision processing algorithm is installed;

FIG. 4 is a flow chart of the method of the present invention for improving underwater calibration based on complex distortion model;

FIG. 5 is a flow chart of the method for underwater bubble measurement based on the improved method for underwater calibration of a complex distortion model;

FIG. 6 is a block diagram of an improved apparatus for underwater calibration based on a complex distortion model according to the present invention;

FIG. 7 is a block diagram of the improved device for underwater calibration based on complex distortion model for image processing based on machine vision processing algorithm;

FIG. 8 is a block diagram of the improved apparatus for underwater calibration based on complex distortion model for establishing the connection between the camera and the platform installed with the machine vision processing algorithm;

FIG. 9 is a block diagram of the computed framing range of the improved apparatus for underwater calibration based on complex distortion models of the present invention;

FIG. 10 is a block diagram of the improved apparatus for underwater calibration based on complex distortion model for underwater bubble measurement;

FIG. 11 is an aerial calibration image of an improved system for underwater calibration based on complex distortion models of the present invention;

FIG. 12 is a schematic of an underwater calibration of the improved system for underwater calibration based on complex distortion models of the present invention;

FIG. 13 is a schematic of an air calibration of the improved system for underwater calibration based on complex distortion models of the present invention;

FIG. 14 is a schematic diagram of calibration coordinates of the improved method for underwater calibration based on a complex distortion model.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Example 1

According to an embodiment of the present invention, an improved method for underwater calibration based on a complex distortion model is provided, referring to fig. 1, including the following steps:

s1: and when a camera trigger signal is received, acquiring the air original image and the underwater original image respectively.

In the present embodiment, since the communication application in a special environment, for example, underwater operation on the sea bottom or the lake bottom, the operation environment at the time of measurement is performed on a very special sea bottom or a very special lake bottom. For the butt joint of instruments, the completion of tasks such as operation indexes and target measurement cannot be completed manually, and thus a non-contact measurement mode is required to complete the tasks, so that the embodiment starts the camera to collect underwater pre-measured targets such as a calibration plate and a bubble group by sending a trigger signal to a signal sensor on the camera.

Specifically, the embodiments in this market respectively collect an air original image and an underwater original image, specifically, a camera is used to convert a pre-measured target, such as a calibration plate in the air and a calibration plate in a water container, into an image signal, and transmit the image signal to a module capable of processing the image signal for processing, so that the image signal can be converted into a digital signal according to information such as pixel distribution, brightness, color and the like, and various operations can be performed on the signals to extract features of the target, such as area, number, position and length, and then a result, including size, angle, number, pass or fail, presence or absence and the like, is output according to preset allowance and other conditions, so as to realize an automatic high-precision recognition function on the underwater measurement target.

S2: and performing image processing operation on the air original image and the underwater original image based on a machine vision processing algorithm to obtain a processed air calibration image and a processed underwater calibration image.

In the present embodiment, the machine vision processing algorithm is a plurality of operators included in machine vision software (HALCON), and in the present embodiment, the HALCON program is transplanted into the VC interface for the user to use based on the VC6 platform.

The HALCON can provide an efficient and flexible image processing function, and has high computational efficiency and high accuracy, for example, when only rough positioning is needed, a fast _ match operator in the HALCON can be used for performing fastest operation, and when precise positioning is needed, a best _ match operator in the HALCON can be used for obtaining an accurate result; HALCON can also process color and multi-channel images, and provides rapid pattern matching calculation and the like to realize rapid and accurate analysis processing of the images.

Specifically, in the embodiment, based on HALCON, image denoising is performed on an air original image and an underwater original image first, and then interesting feature extraction is performed on the denoised images, so as to output an air calibration image and an underwater calibration image which can be used for subsequent calibration.

S3: and respectively carrying out calibration calculation on the underwater calibration image and the air calibration image by adopting a multi-order radial tangential distortion model to obtain the internal and external parameters, the radial distortion parameter and the tangential distortion parameter of the camera.

In this embodiment, since the target of the pre-measurement is an underwater high-speed moving object, such as a bubble, in order to improve the measurement accuracy of the subsequent underwater bubble, the calibration plane needs to be determined for calibration when the underwater bubble is stationary, and the error of the calibration result can be reduced as much as possible; because the camera is difficult and expensive to install underwater, the embodiment adopts the mode that the camera is installed outside the observation window for measurement, so that the measurement can involve underwater calibration, and under the conventional calibration, the camera and the calibration plate are both in the same medium, and because the propagation direction and the speed of light in the same medium can not be changed, the performance of the light can be kept stable; however, calibration under water causes the propagation medium of the light to change.

Further, when underwater calibration is carried out, the camera and the calibration plate are in different media, and light enters water from air through glass, refraction angles are different due to the fact that the light passes through different media in the transmission process, wherein the larger the relative refractive index between substances is, the larger the influence on the refraction angles is; and due to the change of the optical path in different media, the deviation of the imaging point can be caused; and because the thicknesses of the glass are different, the lengths of the corresponding lights passing through the glass are different, and the points of the lights which are emitted out of the glass can be deviated when the refraction angle is reflected, so that the deviation of a light path can be generated in the underwater calibration process, the deviation of the projection point coordinates of the point coordinates in the world coordinate system in the image coordinates in the perspective projection can be influenced, and the obtained calibration result is unreliable.

Specifically, in order to avoid the optical path deviation caused by the change of the optical path transmission medium, in the present embodiment, the multi-order radial tangential distortion model is used for compensation, and in the process of respectively calibrating and calculating the underwater calibration image and the air calibration image, the complicated multi-order radial tangential distortion model can approximately represent the change of the optical path, and can well reflect the influence of the change of the optical path on the parameters of the camera, so as to obtain the internal and external parameters, the radial distortion parameter and the tangential distortion parameter of the camera with higher reliability, thereby ensuring the reliability of the calibration result.

S4: and adjusting the original parameters of the camera by utilizing the internal and external parameters, the radial distortion parameters and the tangential distortion parameters to finish the underwater calibration of the camera.

In the present embodiment, the internal and external parameters are parameters of the camera, which are called internal reference and external reference, wherein the internal reference is parameters that give optical and geometric characteristics of the camera, scale factor and lens distortion; the external parameters are that the position and the direction of the camera corresponding to a world coordinate system, namely a rotation matrix R and a translation matrix T, are given; the radial distortion parameter and the tangential distortion parameter are parameters for improving optical distortion error between an image actually formed by a target on a camera imaging surface and an ideal image due to pre-measurement, so that the precision and the stability of a calibration calculation result are improved, and the reliability of the calibration result is ensured.

Specifically, in the embodiment, the original parameter adjustment is performed on the camera by using the internal and external parameters, the radial distortion parameter and the tangential distortion parameter with good reliability to update or perfect the underwater calibration precision, so that some unavoidable errors existing in the production process of the camera and optical path deviation caused by the change of the optical path transmission medium can be avoided, the underwater calibration precision is high, and the later-stage measurement precision of underwater bubbles is ensured to a certain extent.

According to the method for improving the underwater calibration based on the complex distortion model, the edge image is obtained by performing edge extraction operation on the image to be detected, irrelevant information in the image can be effectively removed, important structural attributes in the image are reserved, and the accurate positioning of the image edge is realized; then, performing frame detection on the edge image, and when a frame is detected, taking an object corresponding to the frame as a non-living body in the image to realize quick identification of the non-living body; then, when the frame is not detected, carrying out flash point contour detection on the image to be detected corresponding to the edge image of the frame which is not detected, and further determining the contour extracted from the image as a living body if the flash point is not detected; if the sparkling point is detected, determining other contours except the sparkling point in the image as living bodies, and realizing quick and accurate identification of the living bodies in the image; the method has the advantages of low calculation complexity, simplicity, convenience, practicability and low cost.

In a preferred technical solution, the multi-order radial tangential distortion model is composed of a third-order radial distortion model and a second-order tangential distortion model.

In this embodiment, the optical path deviation caused by the change of the optical path transmission medium and the deviation of the image center during calibration may cause the distortion coefficient obtained by measurement to be inaccurate, so that the obtained calibration parameter is inaccurate, the calibration result is unreliable, and the subsequent measurement precision of the underwater bubble is affected.

Referring to fig. 14, the multi-order radial tangential distortion model is composed of a third-order radial distortion model and a second-order tangential distortion model, and the third-order radial distortion model and the second-order tangential distortion model can be expressed as:

where (x, y) is expressed as ideal image coordinates, typically in millimeters;

actual image coordinates, typically in millimeters, expressed as the correspondence of ideal image coordinates (x, y); delta_xr＝x(k₁r²+k₂r⁴+k₃r⁶)，δ_yr＝y(k₁r²+k₂r⁴+k₃r⁶) Respectively representing radial distortion in x and y directions; r is²＝(x²+y²)，k₁，k₂And k₃Is the radial distortion coefficient of the lens, p₁And p₂Is the tangential distortion coefficient of the lens.

Further, this embodiment uses

Representing the ideal coordinates, expressed as (u,v) represents the corresponding actual image coordinates, typically in pixels, including radial and tangential distortions; and then pass through

u＝u₀+αx+cy，v＝v₀+ β y, where α, β and c are distortion coefficients that are not all zero, and the following expression is obtained by substituting the distortion coefficients into the equations (1) and (2):

further, the formula (3) may be abbreviated as DC ═ d, and C ═ k₁,k₂,k₃,p₁,p₂]Then the corresponding normal equation is D^-TDC＝D^-Td, i.e. a solution in the least-squares sense can be obtained

C＝(D^-TD)^-1D^-Td (4)

Therefore, after the radial distortion and the tangential distortion of the camera lens are solved in the step (4), the internal and external parameters of the camera can be further corrected by adopting the formula (1) to obtain the internal and external parameters of the video camera and the parameters of the radial distortion and the tangential distortion.

In a preferred technical solution, referring to fig. 2, the step S2 performs an image processing operation on the air original image and the underwater original image based on a machine vision processing algorithm, and the step of obtaining the processed air calibration image and the processed underwater calibration image includes:

s201: and respectively filtering the air original image and the underwater original image to obtain a processed air smooth image and an underwater smooth image.

In the present embodiment, since external noise and internal noise [; the external noise is mainly generated by the sudden change of the current outside the camera or the intrusion of electromagnetic waves into the camera; the internal noise is mainly generated by the fact that components generate heat after long-time operation to generate noise with slightly changed performance parameters; since the generation of gaussian noise is originated from the electronic circuit and the sensor noise caused by low illuminance or high temperature, in the process of filtering the air original image and the underwater original image respectively, the embodiment mainly processes the gaussian noise, specifically, the mean filtering or the median filtering may be used to process the image, and other processing methods may also be used, which are not limited specifically here.

The median filtering method is to set the gray value of each pixel point as the median of the gray values of all pixel points in a window of a certain field of the point to realize the smoothness of the image.

Specifically, the present embodiment adopts a median filtering method to perform filtering processing on the air original image and the underwater original image, and performs median filtering by using a sliding window with odd points, which is also called a template, then replacing the value of the center point of the window with the median of each point in the window, sorting the values according to the magnitude of the values, and taking the number with the sequence number as the center point as filtering output, thereby obtaining the air smooth image and the underwater smooth image, which not only can effectively filter part of gaussian noise, but also has little influence on the edge characteristics of the image.

S202: and respectively carrying out basic edge detection on the air smooth image and the underwater smooth image to obtain an air basic edge image and an underwater basic edge image.

Specifically, the edge detection is to realize accurate positioning of image features so as to ensure the reliability of a calibration result; because the positioning of the image features is realized by using the feature positioning operator, the different positioning operators have different application ranges, the positioning accuracy and the processing result are different, and whether the feature point extraction succeeds or not determines the quality of the calibrated result, wherein the roberts operator has better definition on the edges of the image, the roberts operator is adopted to respectively perform the basic edge detection on the air smooth image and the underwater smooth image so as to output the air basic edge image and the underwater basic edge image with better edge definition.

S203: and respectively carrying out target edge detection on the air base edge image and the underwater base edge image to obtain an air clear edge image and an underwater clear edge image.

In this embodiment, in order to avoid the influence of impurities in water on the edge detection error detection during the basic edge detection, so as to realize the high-precision edge extraction of the image, the target edge detection is performed on the air basic edge image and the underwater basic edge image respectively by adopting the sub-pixel edge detection in this embodiment.

Specifically, sub-pixel edge extraction is respectively adopted for the air basic edge image and the underwater basic edge image, namely interpolation is carried out in a certain area of a pixel-level edge, and the extreme value position of a curve or a curved surface second derivative is used as an accurate edge of an edge center for a line edge; wherein, for the step edge, the extreme value of the first derivative of the curve or the curved surface is the accurate position of the step edge; thereby outputting an air clear edge image and an underwater clear edge image with high edge accuracy.

S204: and respectively carrying out characteristic information processing operation on the air clear edge image and the underwater clear edge image to obtain an air calibration image and an underwater calibration image.

In this embodiment, the image includes basic geometric elements or features such as edge points, straight line segments, curves, vertices, textures, and the like, and in order to reflect the content in the image as completely and accurately as possible, the embodiment performs feature information processing operations on the air clear edge image and the underwater clear edge image.

Specifically, the method is to adopt the tuple function of HALCON to extract and describe the interested features or elements in the air clear edge image and the underwater clear edge image for the interested feature elements so as to quickly and accurately depict the air calibration image and the underwater calibration image.

In a preferred embodiment, referring to fig. 3, before step S1, the method further includes:

s301: and establishing an image acquisition thread between the camera and the platform provided with the machine vision processing algorithm by using the set logical link.

S302: and carrying out setting operation of synchronous primitives with the image acquisition thread on the camera so as to acquire high-precision images.

In this embodiment, in order to ensure the stable association between the camera and the HALCON, so as to achieve fast and accurate processing of the image, and ensure code reuse and maintainability of software in the HALCON, the present embodiment needs to establish the HALCON capable of being applied to image acquisition and image processing according to the API interface provided by the user.

Specifically, the embodiment establishes a logical connection, then establishes an image acquisition thread between the camera and the HALCON by using the set logical link, and further sets the camera to perform a synchronization primitive with the image acquisition thread, so as to ensure acquisition, display and storage of an image.

In a preferred embodiment, referring to fig. 4, before step S301, the method further includes:

s401: test images in a database are acquired.

In this embodiment, the test image is a pre-processed and prepared image, for example, 1000 or more captured underwater calibration images are subjected to image processing means such as image cropping and size adjustment to obtain the test image.

Specifically, the present embodiment may perform indexing in the database according to the data type required by the actual application, so as to quickly and accurately obtain the test image that is processed and manufactured in advance.

S402: and carrying out target tracking operation on the test image to obtain an imaging target in the test image.

In this embodiment, the target tracking operation is performed on each test image, specifically, the test image may be input into the target tracking depth learning model for recognition and analysis, so as to output the imaging target in the test image quickly and accurately.

S403: and calculating the view range of the camera according to the range calculation formula.

In order to ensure the shooting precision of the calibration image, thereby ensuring the reliability of the calibration result, and ensuring the subsequent shooting precision of the underwater bubble, the present embodiment ensures that the calibration image or the bubble image of a sufficiently large shooting area can be acquired to a certain extent by setting the viewing range of the camera, so the present embodiment calculates the viewing range of the camera by substituting the information of the imaging target into the range calculation formula.

Wherein the range calculation formula is expressed as:

FOV＝(Dp+Lv)×3(1+Pa)

wherein the FOV is the field of view; dp is the magnitude in the direction of the imaging target; lv is the possible moving distance of the target in the direction of the imaging target; pa is a percentage or fraction that is a magnification factor set to ensure that the imaged object does not lie exactly on the edge.

Specifically, in the present embodiment, the information of the imaging target is substituted into the range calculation formula to obtain the view range of the camera, so as to ensure the view range of the camera to obtain the calibration image or the bubble image in the sufficiently large shooting area, and ensure the shooting accuracy of the underwater calibration plate or the bubble, thereby ensuring the reliability of the calibration result to a certain extent, and ensuring the subsequent measurement accuracy of the underwater bubble to a certain extent.

In a preferred embodiment, referring to fig. 5, after step S4, the method further includes:

s501: and reading the calibration parameters of the underwater calibration of the camera.

S502: and when the calibration parameters are read, performing element extraction on the shot underwater bubble image according to the calibration parameters to obtain a target image containing element information.

S503: and according to the geometrical coordinate relation, carrying out coordinate conversion and calculation on the element information to obtain the size information corresponding to the element information in the target image.

In this embodiment, after the underwater calibration, the bubble positioning and the bubble size calculation can be performed through the bubble image acquired by the calibration parameters.

Specifically, in the present embodiment, by reading the calibration parameters of the underwater calibration of the camera, while reading the calibration parameters, firstly, a series of image processing means such as image denoising, edge extraction and interested element extraction are carried out on the shot underwater bubble image, to output a target image in which element information, such as the size of the cavitation area, is detected, the element information being expressed in the target image by image coordinates, then the internal and external parameters of the camera in the calibration parameters are substituted into the geometric coordinate conversion relation to convert the element information in the target image, such as the image coordinate of the vacuole area, into world coordinates so as to realize the accurate positioning of the vacuole area, and substituting the converted world coordinates into various calculation formulas of the plane size to output the area and the length of the plane size, so as to obtain size information corresponding to the element information in the target image.

Example 2

According to another embodiment of the present invention, there is provided an improved apparatus for underwater calibration based on a complex distortion model, referring to fig. 6, including:

the image acquisition module 601 is used for acquiring an air original image and an underwater original image respectively when receiving a camera trigger signal;

The image processing module 602 is configured to perform image processing operations on the air original image and the underwater original image based on a machine vision processing algorithm to obtain a processed air calibration image and a processed underwater calibration image;

A calibration calculation module 603, configured to perform calibration calculation on the underwater calibration image and the air calibration image respectively by using a multi-order radial tangential distortion model, so as to obtain an internal parameter, an external parameter, a radial distortion parameter, and a tangential distortion parameter of the camera;

The parameter adjusting module 604 is configured to perform original parameter adjustment on the camera by using the internal and external parameters, the radial distortion parameter, and the tangential distortion parameter, so as to complete underwater calibration of the camera.

The improved device for underwater calibration based on the complex distortion model in the embodiment of the invention performs image processing operation on the collected air original image and the collected underwater original image based on HALCON to quickly and accurately output the air calibration image and the underwater calibration image, can ensure that the images are quickly and accurately processed, simultaneously avoids repeated processing on the images and ensures the processing efficiency; then, the underwater calibration image and the air calibration image are respectively calibrated and calculated by adopting a multi-order radial tangential distortion model, and the underwater calibration image and the air calibration image can be used for compensating the light path change caused by the change of a light path transmission medium so as to reduce the deviation in calculation and obtain the internal and external parameters, the radial distortion parameter and the tangential distortion parameter of the camera with higher reliability; the original parameters of the camera are adjusted by utilizing the internal and external parameters, the radial distortion parameters and the tangential distortion parameters, so that the reliability of the calibration result of the underwater calibration of the camera is high, the accuracy of measuring bubbles moving underwater at high speed can be ensured to a certain extent, and the reliability of the calibration result can be improved by the improved device of the underwater calibration based on the complex distortion model; the method has the advantages of low calculation complexity, simplicity, convenience, practicability and low cost.

Wherein, the multistage radial tangential distortion model comprises third order radial distortion model and second order tangential distortion model, and third order radial distortion model and second order tangential distortion model can be expressed as:

where (x, y) is expressed as ideal image coordinates, typically in millimeters;

Further, this embodiment uses

Representing ideal coordinates, and (u, v) corresponding actual image coordinates, usually in pixels, including radial distortion and tangential distortion; and then pass through

C＝(D^-TD)^-1D^-Td (4)

In a preferred embodiment, referring to fig. 7, the image processing module 602 includes:

the filtering processing unit 701 is configured to perform filtering processing on the air original image and the underwater original image respectively to obtain a processed air smooth image and a processed underwater smooth image;

A basic edge detection unit 702, configured to perform basic edge detection on the air smooth image and the underwater smooth image respectively to obtain an air basic edge image and an underwater basic edge image;

The target edge detection unit 703 is configured to perform target edge detection on the air-based edge image and the underwater-based edge image respectively to obtain an air clear edge image and an underwater clear edge image;

And the characteristic processing unit 704 is configured to perform characteristic information processing operations on the air clear edge image and the underwater clear edge image respectively to obtain an air calibration image and an underwater calibration image.

As a preferred technical solution, referring to fig. 8, the apparatus further includes:

an acquisition thread establishing module 801, configured to establish an image acquisition thread between a camera and a platform on which a machine vision processing algorithm is installed, using a set logical link;

and the thread synchronization setting module 802 is used for performing setting operation of synchronizing primitives with the image acquisition thread on the camera so as to acquire high-precision images.

As a preferred technical solution, referring to fig. 9, the apparatus further includes:

a test image obtaining module 901, configured to obtain a test image in a database;

An imaging target obtaining module 902, configured to perform target tracking operation on the test image to obtain an imaging target in the test image;

And a view range calculation module 903, configured to calculate a view range of the camera according to a range calculation formula for the imaging target.

Wherein the range calculation formula is expressed as:

FOV＝(Dp+Lv)×3(1+Pa)

As a preferred technical solution, referring to fig. 10, the apparatus further includes:

a parameter reading module 101, configured to read calibration parameters for completing underwater calibration of a camera;

the element extraction module 102 is configured to, when the calibration parameters are read out, perform element extraction on the shot underwater bubble image according to the calibration parameters to obtain a target image containing element information;

and the coordinate conversion module 103 is configured to perform coordinate conversion and calculation on the element information according to the geometric coordinate relationship, so as to obtain size information corresponding to the element information in the target image.

Example 3

According to another embodiment of the present invention, there is provided an improved system for underwater calibration based on a complex distortion model, see fig. 11 to 13, including:

In the improved system for underwater calibration based on the complex distortion model, the calibration plates placed in air and water are respectively shot by matching a camera with a light source lighting device so as to collect the air original image and the underwater original image of the calibration plates, and the collected air original image and the collected underwater original image are subjected to image processing operation by a machine vision processing algorithm platform so as to quickly and accurately output the air calibration image and the underwater calibration image, so that the images can be quickly and accurately processed, the images are prevented from being repeatedly processed, and the processing efficiency is ensured; then, a machine vision processing algorithm platform adopts a multi-order radial tangential distortion model to respectively perform calibration calculation on the collected underwater calibration image and the collected air calibration image, and the calibration calculation can be used for compensating the light path change caused by the change of a light path transmission medium so as to reduce the deviation in calculation and obtain the internal and external parameters, the radial distortion parameter and the tangential distortion parameter of the camera with high reliability; then, the original parameters of the camera are adjusted, so that the reliability of the calibration result of the underwater calibration of the camera is high, and the improved system of the underwater calibration based on the complex distortion model can improve the reliability and accuracy of the calibration result; the method has the advantages of low calculation complexity, simplicity, convenience, practicability and low cost.

It should be noted that, in order to implement the underwater calibration and ensure the reliability of the calibration result, the embodiment uses the calibration board with a certain specification, and calculates the internal and external parameters of the camera through accurate measurement, wherein the embodiment uses the HALCON-compatible calibration board to calculate the coordinates of the center of a circle as the known detection data, so as to improve the calibration accuracy, because the HALCON-compatible calibration board is always in the shape of an ellipse in each mapping of the circle, and the circle is only a special case of the ellipse, therefore, the coordinates of the center of a circle can be fitted through the ellipse equation, so as to obtain the accurate point coordinate data of the center of a circle, thereby reducing the error caused by the coordinate deviation of the center point in the calculation process, and ensuring the reliability of the calibration result.

It should be noted that the machine vision processing algorithm platform may be a VC6 platform installed with HALCON, and may also be other data processing platforms, which are not limited herein.

It should be noted that, in order to obtain a clear and definite image of the pre-measurement target, the present embodiment uses a camera with a CCD sensor to ensure the acquisition of the optical information on the image.

It should be noted that, because the object to be photographed is the calibration plate and the transparent bubble, the embodiment can adopt two types of surface-shaped light source and linear light source in terms of the shape of the illumination light source of the light source illumination device; for the illumination of the surface light source, the light source and the camera are coaxially arranged in spatial arrangement, namely transmission type illumination imaging, the imaging surface and the imaging depth of field of the camera are positioned near the window of the testing instrument, and the illumination light beam enters the field of view of the camera through the bubble group, so that the fuzzy of a calibration image and the overlapping of bubble imaging are avoided; for linear light source illumination, the light source and the camera are arranged on the spatial arrangement axis in a vertical mode, namely, lateral illumination imaging is carried out, the illumination light beams only illuminate the imaging surface of the camera, and the group of bubbles enter the camera view field after scattering the illumination light beams.

Compared with the existing calibration technical method, the improved method, the device and the system of the underwater calibration based on the complex distortion model have the advantages that:

1. according to the method, the collected air original image and the collected underwater original image are subjected to image processing operation based on the machine vision processing algorithm, so that the air calibration image and the underwater calibration image are quickly and accurately output, the images can be quickly and accurately processed, meanwhile, the images are prevented from being repeatedly processed, and the processing efficiency is guaranteed;

2. in the embodiment, the underwater calibration image and the air calibration image are respectively calibrated and calculated by adopting a multi-order radial tangential distortion model, so that the optical path change caused by the change of an optical path transmission medium can be compensated, the deviation in calculation is reduced, and the internal and external parameters, the radial distortion parameter and the tangential distortion parameter of the camera with higher reliability are obtained;

3. according to the embodiment, the original parameters of the camera are adjusted by utilizing the internal and external parameters with high reliability, the radial distortion parameters and the tangential distortion parameters, so that the calibration result of the underwater calibration of the camera is high in reliability, and the accuracy of measuring bubbles moving underwater at a high speed can be guaranteed to a certain extent.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described system embodiments are merely illustrative, and for example, a division of a unit may be a logical division, and an actual implementation may have another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

The foregoing is only a preferred embodiment of the present invention, and it should be noted that it is obvious to those skilled in the art that various modifications and improvements can be made without departing from the principle of the present invention, and these modifications and improvements should also be considered as the protection scope of the present invention.

Claims

1. An improved method for underwater calibration based on a complex distortion model is characterized by comprising the following steps:

respectively carrying out calibration calculation on the underwater calibration image and the air calibration image by adopting a multi-order radial tangential distortion model to obtain internal and external parameters, a radial distortion parameter and a tangential distortion parameter of a camera;

and adjusting the original parameters of the camera by using the internal and external parameters, the radial distortion parameters and the tangential distortion parameters to finish the underwater calibration of the camera.

2. The improved method for underwater calibration based on a complex distortion model as claimed in claim 1, wherein the multiple order radial tangential distortion model is composed of a third order radial distortion model and a second order tangential distortion model.

3. The improved method for underwater calibration based on complex distortion model as claimed in claim 1, wherein said step of performing image processing operation on said air original image and said underwater original image based on machine vision processing algorithm to obtain processed air calibration image and underwater calibration image comprises:

and respectively carrying out characteristic information processing operation on the air clear edge image and the underwater clear edge image to obtain the air calibration image and the underwater calibration image.

4. The improved method of underwater calibration based on complex distortion model of claim 1, wherein before the step of acquiring the air raw image and the underwater raw image respectively when receiving the camera trigger signal, the method further comprises:

establishing an image acquisition thread between the camera and a platform provided with the machine vision processing algorithm by using the set logical link;

5. The improved method of complex distortion model based underwater calibration as claimed in claim 4, wherein before said step of establishing an image acquisition thread between said camera and a platform on which said machine vision processing algorithm is installed using a set logical link, said method further comprises:

acquiring a test image in a database;

and calculating the view range of the camera according to the imaging target by a range calculation formula.

6. The improved method of complex distortion model based underwater calibration of claim 1, wherein after said step of performing raw parameter adjustments on said camera using said internal and external parameters, said radial distortion parameters, and said tangential distortion parameters to complete underwater calibration of said camera, said method further comprises:

and according to the geometrical coordinate relationship, carrying out coordinate conversion and calculation on the element information to obtain size information corresponding to the element information in the target image.

7. An improved device for underwater calibration based on a complex distortion model is characterized by comprising:

8. The apparatus of claim 7, wherein the image processing module comprises:

and the characteristic processing unit is used for respectively carrying out characteristic information processing operation on the air clear edge image and the underwater clear edge image to obtain the air calibration image and the underwater calibration image.

9. The improved apparatus for underwater calibration based on complex distortion models of claim 7, further comprising:

the acquisition thread establishing module is used for establishing an image acquisition thread between the camera and a platform provided with the machine vision processing algorithm by using the set logical link;

and the thread synchronization setting module is used for setting primitives synchronized with the image acquisition thread for the camera so as to acquire high-precision images.

10. An improved system for underwater calibration based on a complex distortion model, comprising:

the machine vision processing algorithm platform is further used for respectively carrying out calibration calculation on the underwater calibration image and the air calibration image by adopting a multi-order radial tangential distortion model to obtain an internal parameter, an external parameter, a radial distortion parameter and a tangential distortion parameter of the camera;

the camera is further used for adjusting original parameters of the camera by utilizing the internal and external parameters, the radial distortion parameters and the tangential distortion parameters so as to finish underwater calibration of the camera;

the calibration plate is used for shooting the calibration plate through the camera in cooperation with the light source lighting device to obtain a calibration image.