CN113513981A - Multi-target parallel measurement method, system, equipment and storage medium based on binocular stereo vision - Google Patents
Multi-target parallel measurement method, system, equipment and storage medium based on binocular stereo vision Download PDFInfo
- Publication number
- CN113513981A CN113513981A CN202110663144.5A CN202110663144A CN113513981A CN 113513981 A CN113513981 A CN 113513981A CN 202110663144 A CN202110663144 A CN 202110663144A CN 113513981 A CN113513981 A CN 113513981A
- Authority
- CN
- China
- Prior art keywords
- target
- binocular stereo
- stereo vision
- images
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/002—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Abstract
The invention discloses a binocular stereo vision-based multi-target parallel measurement method, system, equipment and storage medium, and belongs to the field of optical large-size measurement. Calibrating by adopting a checkerboard calibration method to obtain a conversion relation between a world coordinate system and an image pixel coordinate system and a distortion parameter of an image; then, outputting a video stream, adopting a left camera and a right camera to respectively capture a pair of images at the same moment and carry out preprocessing aiming at the output video stream, and then respectively carrying out segmentation and splicing to obtain a plurality of corresponding pairs of images only with coding targets; after traversing, quickly extracting angular point information, and after obtaining an initial sub-pixel point corresponding set, carrying out recognition, decoding and matching to obtain one-to-one corresponding sub-pixel point corresponding sets; and respectively obtaining the three-dimensional coordinates of the target points according to the optical triangular model, and completing the multi-target parallel measurement based on the binocular stereo vision. The invention realizes the on-line high-precision rapid measurement, greatly improves the portability and reduces the limitation of application scenes.
Description
Technical Field
The invention belongs to the field of optical large-size measurement, and relates to a multi-target parallel measurement method, a system, equipment and a storage medium based on binocular stereo vision.
Background
The binocular stereo vision measurement is a large-size high-precision non-contact measurement method, and the essence of the non-contact measurement is to acquire the coordinates of the characteristic points of a measured object in a certain established world coordinate system through various non-contact ways. The digital photogrammetry can ensure higher precision while realizing non-contact measurement, and can reduce the equipment complexity and cost brought by the sensor, thus being an ideal measurement method.
The binocular stereo vision measurement method can acquire the three-dimensional information of the surface of the measured object by using two area-array cameras to shoot the image of the measured object and carrying out image characteristic calculation, tracking, three-dimensional reconstruction and other processing. In recent decades, with the rapid development of digital cameras, camera models, calibration, image processing, feature matching, multi-view geometry and other technologies, the advantages of the stereoscopic vision measurement technology in the aspects of measurement speed, measurement range, measurement precision and the like are gradually highlighted, a series of applications are generated in the fields of industrial detection, aerospace, automobile manufacturing, biomedicine, remote sensing science, precision instruments and the like, and the development of related fields is greatly promoted.
In binocular stereo vision measurement, in order to obtain a high-precision measurement result, an industrial camera with high pixels is required, the processing speed of the obtained high-pixel image is greatly reduced, and the requirements of real-time performance and rapidity of measurement are difficult to meet by using a conventional image processing method. When the three-dimensional coordinates are calculated, the accurate projection point correspondence of a target point on two cameras needs to be obtained, and the conditions of matching errors and low corresponding accuracy of the matching points are easy to occur by using a conventional matching method. The existing camera is usually fixed by a tripod, cannot be moved after being calibrated, is not high in portability, and is greatly limited in use scene.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to provide a multi-target parallel measurement method, a system, equipment and a storage medium based on binocular stereo vision. The invention aims at the problems of image processing speed, image matching and application limitation of the existing binocular stereo vision measuring system, realizes online high-precision rapid measurement, greatly improves portability and reduces the limitation of application scenes.
In order to achieve the purpose, the invention adopts the following technical scheme to realize the purpose:
the invention discloses a multi-target parallel measurement method based on binocular stereo vision, which comprises the following steps:
step 1, calibrating a binocular stereo vision measuring unit by adopting a checkerboard calibration method to obtain a conversion relation between a world coordinate system and an image pixel coordinate system and a distortion parameter of an image;
step 2, outputting a video stream by using a calibrated binocular stereo vision measuring unit, adopting a left camera and a right camera in the binocular stereo vision measuring unit to respectively capture a pair of images at the same moment aiming at the output video stream, preprocessing the obtained pair of images, respectively carrying out segmentation and splicing processing on the preprocessed pair of images, and obtaining a plurality of corresponding pairs of images only with coding targets according to a plurality of targets needing to be identified;
step 3, respectively traversing the multiple pairs of images, quickly extracting angular point information to obtain an initial sub-pixel point corresponding set, and identifying, decoding and matching the codes of each coding target to obtain one-to-one corresponding sub-pixel point corresponding sets;
and 4, respectively obtaining three-dimensional coordinates of the target points according to the sub-pixel point corresponding sets which are in one-to-one correspondence and the optical triangular model, and completing the multi-target parallel measurement based on the binocular stereo vision.
Preferably, in step 1, a checkerboard calibration method is performed based on a calibration toolkit of MATLAB.
Preferably, in step 1, in the binocular stereo vision measuring unit, a short baseline and an optical axis angle of 20 to 30 degrees are adopted.
Further preferably, the short base line is 30-50 mm.
Preferably, in step 2, the performing the pretreatment comprises: firstly, filtering processing is carried out, and then binarization processing and contour detection are carried out.
Preferably, in step 3, the coding target is an annular coding mark, a black area and a white area are arranged on the coding target, the binary code corresponding to the black area is 0, the binary code corresponding to the white area is 1, any one coding ring unit is taken as a starting point, the circular rings are concentrically coded in a counterclockwise sequence to form a multi-bit binary string, one multi-bit coding mark point has a plurality of different binary strings, and the decimal value corresponding to the binary system with the smallest value can be represented as the decoded value of the coding mark point.
Preferably, in step 4, according to the obtained one-to-one sub-pixel point corresponding sets, respectively obtaining three-dimensional coordinates of the target points according to the optical triangular model, specifically including:
distortion correction is carried out on the corner point information which is obtained by correspondingly and intensively completing coding sequencing on the sub-pixel points which are obtained in the step 1 in a one-to-one correspondence mode according to the distortion parameters, an equation set is established according to the optical triangular model and the conversion relation between the world coordinate system and the image pixel coordinate system which are obtained through calibration, the corner point information which is obtained by completing coding sequencing is correspondingly brought into the equation set in sequence, the three-dimensional coordinates of the target point to be detected are solved through the least square method, and the three-dimensional coordinates of the target point are obtained respectively.
The invention discloses a multi-target parallel measurement system based on binocular stereo vision, which comprises:
the camera calibration module is used for calibrating the binocular stereoscopic vision measurement unit by adopting a checkerboard calibration method to obtain a conversion relation between a world coordinate system and an image pixel coordinate system and a distortion parameter of an image;
the image processing module is used for outputting a video stream by using the calibrated binocular stereo vision measuring unit, adopting a left camera and a right camera in the binocular stereo vision measuring unit to respectively capture a pair of images at the same moment aiming at the output video stream, preprocessing the obtained pair of images, respectively segmenting and splicing the preprocessed pair of images, and obtaining a plurality of corresponding pairs of images only with coding targets according to a plurality of targets needing to be identified;
the image target point extracting and matching module is used for respectively traversing the multiple pairs of images, extracting angular point information to obtain an initial sub-pixel point corresponding set, and identifying, decoding and matching the codes of each coding target to obtain one-to-one corresponding sub-pixel point corresponding sets;
and the three-dimensional reconstruction module is used for respectively obtaining the three-dimensional coordinates of the target points according to the sub-pixel point corresponding sets which are in one-to-one correspondence and the optical triangular model.
The invention discloses computer equipment which comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, wherein the processor executes the computer program to realize the steps of the multi-target parallel measurement method based on the binocular stereo vision.
The invention discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the multi-target parallel measurement method based on binocular stereo vision.
Compared with the prior art, the invention has the following beneficial effects:
the invention discloses a binocular stereo vision-based multi-target parallel measurement method, which can realize that each corner target has unique identity information by encoding the targets, thereby greatly reducing the matching error rate of corresponding left and right camera points, reducing the gross error of measurement and improving the accuracy of the whole measurement system. The existing binocular stereo vision measuring system needs to be calibrated on site in an industrial field, and cannot be moved again after calibration is completed. The two cameras are carried on one device to form the binocular stereoscopic vision measuring unit, so that the space occupied by the measuring system is greatly reduced, the measuring system can be directly carried into the field after the field calibration is finished outside the industrial field, the defect that the measuring system cannot move in the using process is overcome, and the portability and the practicability of the measuring system are greatly improved. Therefore, the invention effectively solves the problems of the existing binocular stereo vision measuring system in the aspects of image processing speed, image matching and limited application.
Further, the invention can improve the speed of image processing, thereby improving the whole operation speed. The existing binocular stereo vision measurement method is slow in measurement speed, and all pixel points of the whole image need to be traversed and detected. According to the invention, through filtering, binaryzation, segmentation and splicing of the image, the target is quickly positioned, the time for subsequently extracting angular points is reduced, and the measurement speed of a measurement system is improved.
Furthermore, the method realizes the rapid extraction of the corner points based on the FAST library function of OPENCV. The existing binocular stereoscopic vision measuring method is low in measuring speed, a Harris corner extraction method or an elliptical mark method is adopted, the Harris corner extraction method is low in corner extraction speed, the speed of the whole measuring system is influenced, the measurement of a moving object with high speed cannot be met, the elliptical mark method is strict in measuring scene requirements, and accurate extraction of the circle center is difficult to achieve. The invention adopts a rapid angular point extraction method, which can greatly accelerate the extraction of the angular point. The present invention encodes a target. Each angular point target can have unique identity information, so that the matching error rate of the left camera point and the right camera point is greatly reduced, the coarse error of measurement is reduced, and the accuracy of the whole measuring system is improved.
Drawings
FIG. 1 is a schematic diagram of three-dimensional reconstruction;
FIG. 2 is a flow chart of a method of the present invention;
FIG. 3 is a pictorial representation of a coded target of the present invention;
FIG. 4 is a pictorial view of a camera module having a mounting apparatus according to the present invention;
FIG. 5 is a flow chart of data processing in the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The invention discloses an on-line high-precision rapid measurement system based on binocular stereo vision, which can be realized based on a FAST library function of OPENCV and a calibration toolkit of MATLAB.
Specifically, the following aspects may be included:
a physical carrying device structure of the camera calibration module;
the invention relates to a method for measuring the binocular stereoscopic vision of a camera, which is characterized in that the structure of a carrying device of the camera is designed, and the carrying mode that a left camera and a right camera are respectively arranged on a tripod in the prior art is replaced. Wherein, aiming at the binocular stereo vision measuring unit, a short base line and an optical axis included angle of 20-30 degrees are adopted. Preferably, the short base line is 30-50 mm.
A camera calibration module;
the pinhole camera projection model is provided with four coordinate systems, namely a world coordinate system, a camera coordinate system, an image physical coordinate system and an image pixel coordinate system, and the camera calibration is used for obtaining the transformation relation between the world coordinate system and the image pixel coordinate system and the distortion parameter of the image, so that the camera calibration is very important for the measurement precision of the system. By adopting a checkerboard calibration method and a MATLAB-based calibration toolkit, the rapid calibration can be realized, and the parameters obtained by the calibration are stored.
An image processing module;
in order to realize real-time measurement, a camera is required to realize video stream output, a calibrated binocular stereo vision measurement unit is used for outputting the video stream, specifically, a pair of images at the same moment are respectively captured by a left camera and a right camera of the binocular stereo vision measurement unit, and image preprocessing is performed through the pair of images, including filtering processing and image binarization, so that the rapid positioning of an image target is realized. Before the angular points are extracted, the preprocessed pair of images are respectively segmented and spliced, and a plurality of corresponding pairs of images only with coding targets are obtained according to a plurality of targets needing to be identified, so that the number of pixel points of the traversed images when the angular points are extracted is reduced.
An image target point extracting and matching module;
and traversing the multiple pairs of images respectively, quickly extracting angular point information, identifying and decoding the codes of each coding target, and matching based on the codes of each coding target to obtain one-to-one corresponding sub-pixel point corresponding sets. When a plurality of targets are measured in parallel, the targets are coded, and accurate matching of the same target point among a plurality of cameras is achieved. The angular point to be measured is located at the center of the target, and the outer side of the target is designed with an annular coding mark and a reflecting device, so that rapid positioning can be realized. The method for extracting the corner points rapidly is used, and the speed of extracting the corner points is far higher than that of a common Harris corner point extraction method. The designed ring-shaped coding mark can be made into black or white, the binary code corresponding to black is 0, white is 1, any coding ring unit is taken as a starting point, the concentric coding rings can form a multi-bit binary string according to the anticlockwise sequence, one multi-bit coding mark point has a plurality of different binary strings, and the decimal value corresponding to the binary with the minimum value can be represented as the decoding value of the coding mark point.
A three-dimensional reconstruction module;
and respectively obtaining the three-dimensional coordinates of the target points according to the obtained one-to-one corresponding sub-pixel point corresponding sets and the optical triangular model. When calculating the three-dimensional coordinates of the target point, the optical centers of the left and right cameras are respectively connected with the target point on the left and right images according to the optical triangular model, and the two straight lines are located on the same plane, so that the intersection point is the target point to be obtained. After distortion correction of the image is carried out, an equation is established by the model and is solved by a least square method, and then the three-dimensional coordinates of the target point can be obtained.
The preferable and reliable physical carrying structure solves the problem that the measuring system cannot move after the camera calibration is finished, and realizes the separation of the calibration of the measuring system and an industrial field, namely the calibration is finished in a laboratory and is directly brought into the field for measurement. The space occupied by the measuring system is reduced, and the practicability of the measuring system based on binocular stereoscopic vision is greatly improved.
Preferably, the calibration toolkit based on MATLAB can realize that the folder for directly reading the pictures can quickly finish calibration, so that the camera calibration speed is improved.
Preferably, the image is preprocessed before the angular points are extracted, and the image is segmented and spliced, so that the traversal of the whole image during the angular point extraction is avoided, and the image processing speed is greatly improved.
Preferably, the method for rapidly extracting the angular points improves the angular point extraction speed and the image processing speed on the algorithm level.
Preferably, the target is coded, and the coded mark points have unique identity information, so that the method can be conveniently and reliably used for matching corresponding points between camera images, avoids measurement errors caused by matching errors of the corresponding points of a measurement system, and improves the precision of the measurement system.
The invention is described in further detail below with reference to the accompanying drawings:
fig. 2 is a block diagram of a multi-target parallel measurement method based on binocular stereo vision according to the present invention. As can be seen from fig. 2, the multi-target parallel measurement method based on binocular stereo vision specifically includes the following steps:
before measurement is started, the coding target needs to be manufactured, the two cameras are fixed in the carrying device, and pictures used for calibration are collected by a computer to finish calibration of a binocular stereo vision measuring unit in the measuring system, so that the conversion relation between a world coordinate system and an image pixel coordinate system and the distortion parameters of the image are obtained. The method comprises the steps that a light supplementing device is used for supplementing light to coding targets during measurement, a measuring system collects images, a calibrated binocular stereo vision measuring unit is used for outputting video streams, the output video streams are obtained, a left camera and a right camera are used for respectively capturing a pair of images at the same moment, image preprocessing such as filtering and binaryzation is sequentially carried out on the pair of collected images, the approximate position of the targets is rapidly determined, then the images are segmented and spliced, a plurality of pairs of corresponding images only with the coding targets are obtained according to a plurality of targets needing to be identified, the plurality of pairs of images are traversed respectively, angular point information is rapidly extracted firstly, angular point information of the targets is rapidly and accurately extracted, then codes of each coding target are identified and decoded, and codes based on each coding target are matched to obtain sub-pixel point corresponding sets which correspond to one by one; and matching between the point correspondences is completed. And according to the sub-pixel point corresponding sets which are in one-to-one correspondence, calculating required three-dimensional coordinate information by using an equation for calculating the three-dimensional coordinates according to the optical triangular model, respectively obtaining the three-dimensional coordinates of the target points, and completing the parallel measurement of multiple targets.
The invention mainly comprises a camera carrying device and a measuring program, wherein the measuring program comprises camera calibration, image preprocessing, image matching and three-dimensional reconstruction, and the specific contents are as follows.
1. A camera mounting device;
before designing and manufacturing the camera carrying device, the model of the camera and the model of the lens are selected according to the factors such as the required field size and the shooting distance to be measured, then the corresponding camera carrying device is designed and manufactured according to the factors such as the model of the camera and the model of the lens and the included angle between the two cameras, and before calibration, the two cameras are fixed in the carrying device, as shown in fig. 4.
2. Calibrating a camera;
before calibration, a left camera and a right camera of a binocular stereo vision measuring unit are used for taking a plurality of pairs of photos (15-20 pairs) in different chequer postures and respectively stored in two folders. And then opening folders for storing images through a MATLAB calibration toolkit, extracting corner point information of each pair of images one by one, calibrating internal parameters and external parameters (rotation and translation matrixes) required by the conversion relation between a world coordinate system and an image pixel coordinate system through a checkerboard calibration algorithm, and storing the internal parameters, the external parameters and the distortion parameters of the images.
3. Processing an image;
the coded target should be manufactured before the whole measurement, as shown in fig. 3, specifically: the annular coding mark is provided with a black area and a white area, the binary code corresponding to the black area is 0, the binary code corresponding to the white area is 1, any coding ring unit is taken as a starting point, the circular rings are concentrically coded in a counterclockwise sequence to form a multi-bit binary string, one multi-bit coding mark point has a plurality of different binary strings, and the decimal value corresponding to the binary with the minimum value can be represented as the decoded value of the coding mark point.
Preferably, a layer of reflecting device is designed on the outer side of the target, so that the coded target can be quickly and accurately positioned.
Referring to fig. 5, the image processing module is implemented by the following steps.
Step 1, capturing a pair of images at the same time in video streams output by a left camera and a right camera of a binocular stereo vision measuring unit, and waiting for the subsequent processing of the pair of images.
And 2, filtering the pair of images obtained in the step 1, so that the noise of the images is reduced, and the measurement precision is improved.
And 3, after the processing in the step 2 is finished, continuing to perform binarization processing and performing contour detection, thereby reducing the influence of external environment, realizing the rapid positioning of the target and finishing the pretreatment.
And 4, segmenting and splicing the pair of preprocessed images obtained in the step 3, and obtaining a plurality of corresponding pairs of images only with the coding targets according to a plurality of targets to be identified so as to improve the subsequent angular point detection speed.
4. Extracting and matching image target points;
the image target point extracting and matching module comprises two parts of rapid extraction and accurate matching of image target points. And quickly extracting angular point information to obtain an initial sub-pixel point corresponding set, and identifying, decoding and matching the codes of each coding target to obtain the sub-pixel point corresponding sets in one-to-one correspondence.
4.1, image target point extraction:
the invention is based on the FAST library function of OPENCV, adopts a rapid corner extraction method to detect that the corners are superior to Harris corner extraction method in speed, traverses a plurality of pairs of images respectively, firstly carries out rapid corner information extraction processing on a plurality of pairs of images obtained by image processing, extracts and obtains pixel information (corner information) of a required target point, and stores the pixel information to obtain an initial sub-pixel point corresponding set.
4.2, matching image target points:
by the binary code identification coding method, the corner information is extracted, and simultaneously the codes of each coding target of the extracted corner information are sequentially coded and cached. After the detection of the corner points of a pair of images is completed, the codes of each coding target of all the corner point information extracted from the pair of images are decoded and matched to obtain a one-to-one corresponding sub-pixel point corresponding set, and the sub-pixel point corresponding set is cached.
5. Three-dimensional reconstruction module
A schematic diagram of a three-dimensional reconstruction is shown in fig. 1. Before three-dimensional reconstruction, distortion correction should be performed on the corner information which completes coding sequencing in the corresponding set of the sub-pixel points corresponding to one according to the distortion parameters obtained by calibration. And establishing an equation set according to the optical triangular model and internal and external parameters of the conversion relation between the world coordinate system and the image pixel coordinate system obtained by calibration, correspondingly bringing the angular point information subjected to coding sequencing into the equation set in sequence, and solving the three-dimensional coordinates of the target point to be measured by a least square method, thereby completing multi-target parallel online high-precision rapid measurement.
Specifically, in a specific embodiment of the present invention, the method for measuring multiple targets in parallel based on binocular stereo vision is stable and reliable, increases the measuring speed, can realize accurate matching of point correspondences, improves the measuring accuracy, realizes multiple target parallel measurement, improves the portability and the practicability of the measuring system, and realizes multiple target real-time rapid measurement of the measuring system based on binocular stereo vision. Experiments prove that when the measuring distance is 1.2m, a 1200-pixel and 12-mm focal length lens and a 1200-pixel industrial camera are selected, and the error of the length of 300mm is within 0.02 mm. The method successfully realizes the multi-target parallel online high-precision rapid measurement based on the binocular stereo vision.
The invention designs a novel physical structure carried by the measuring system, and the measuring system is modularized. The existing binocular stereo vision measuring system needs to be calibrated on site in an industrial field, and cannot be moved again after calibration is completed. According to the invention, two cameras are carried on one device, so that the space occupied by the measuring system is greatly reduced, the camera can be directly measured on site after calibration outside an industrial site, the defect that the camera cannot move is overcome in the using process, and the portability and the practicability of the measuring system are greatly improved.
In another embodiment of the present invention, a binocular stereo vision-based multi-target parallel measurement system is provided, which can be used to implement the above binocular stereo vision-based multi-target parallel measurement method, and specifically, the binocular stereo vision-based multi-target parallel measurement system includes a camera calibration module, an image processing module, an image target point extraction and matching module, and a three-dimensional reconstruction module.
The camera calibration module is used for calibrating the binocular stereoscopic vision measurement unit by adopting a checkerboard calibration method to obtain a conversion relation between a world coordinate system and an image pixel coordinate system and a distortion parameter of an image; the image processing module is used for outputting a video stream by using the calibrated binocular stereo vision measuring unit, adopting a left camera and a right camera in the binocular stereo vision measuring unit to respectively capture a pair of images at the same moment aiming at the output video stream, preprocessing the obtained pair of images, respectively segmenting and splicing the preprocessed pair of images, and obtaining a plurality of corresponding pairs of images only with coding targets according to a plurality of targets needing to be identified; the image target point extracting and matching module is used for respectively traversing the multiple pairs of images, extracting angular point information to obtain an initial sub-pixel point corresponding set, and identifying, decoding and matching the codes of each coding target to obtain one-to-one corresponding sub-pixel point corresponding sets; and the three-dimensional reconstruction module is used for respectively obtaining the three-dimensional coordinates of the target points according to the sub-pixel point corresponding sets which are in one-to-one correspondence and the optical triangular model.
In yet another embodiment of the present invention, a terminal device is provided that includes a processor and a memory for storing a computer program comprising program instructions, the processor being configured to execute the program instructions stored by the computer storage medium. The Processor may be a Central Processing Unit (CPU), or may be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable gate array (FPGA) or other Programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc., which is a computing core and a control core of the terminal, and is adapted to implement one or more instructions, and is specifically adapted to load and execute one or more instructions to implement a corresponding method flow or a corresponding function; the processor provided by the embodiment of the invention can be used for operating the multi-target parallel measurement method based on binocular stereo vision, and comprises the following steps: step 1, calibrating a binocular stereo vision measuring unit by adopting a checkerboard calibration method to obtain a conversion relation between a world coordinate system and an image pixel coordinate system and a distortion parameter of an image; step 2, outputting a video stream by using a calibrated binocular stereo vision measuring unit, adopting a left camera and a right camera in the binocular stereo vision measuring unit to respectively capture a pair of images at the same moment aiming at the output video stream, preprocessing the obtained pair of images, respectively carrying out segmentation and splicing processing on the preprocessed pair of images, and obtaining a plurality of corresponding pairs of images only with coding targets according to a plurality of targets needing to be identified; step 3, respectively traversing the multiple pairs of images, quickly extracting angular point information to obtain an initial sub-pixel point corresponding set, and identifying, decoding and matching the codes of each coding target to obtain one-to-one corresponding sub-pixel point corresponding sets; and 4, respectively obtaining three-dimensional coordinates of the target points according to the sub-pixel point corresponding sets which are in one-to-one correspondence and the optical triangular model, and completing the multi-target parallel measurement based on the binocular stereo vision.
In still another embodiment, the present invention also provides a computer-readable storage medium (Memory) which is a Memory device in a terminal device and stores programs and data. It is understood that the computer readable storage medium herein may include a built-in storage medium in the terminal device, and may also include an extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space storing an operating system of the terminal. Also, one or more instructions, which may be one or more computer programs (including program code), are stored in the memory space and are adapted to be loaded and executed by the processor. It should be noted that the computer-readable storage medium may be a high-speed RAM memory, or may be a non-volatile memory (non-volatile memory), such as at least one disk memory.
The processor can load and execute one or more instructions stored in the computer readable storage medium to realize the corresponding steps of the multi-target parallel measurement method based on the binocular stereo vision in the embodiment; one or more instructions in the computer-readable storage medium are loaded by the processor and perform the steps of: step 1, calibrating a binocular stereo vision measuring unit by adopting a checkerboard calibration method to obtain a conversion relation between a world coordinate system and an image pixel coordinate system and a distortion parameter of an image; step 2, outputting a video stream by using a calibrated binocular stereo vision measuring unit, adopting a left camera and a right camera in the binocular stereo vision measuring unit to respectively capture a pair of images at the same moment aiming at the output video stream, preprocessing the obtained pair of images, respectively carrying out segmentation and splicing processing on the preprocessed pair of images, and obtaining a plurality of corresponding pairs of images only with coding targets according to a plurality of targets needing to be identified; step 3, respectively traversing the multiple pairs of images, quickly extracting angular point information to obtain an initial sub-pixel point corresponding set, and identifying, decoding and matching the codes of each coding target to obtain one-to-one corresponding sub-pixel point corresponding sets; and 4, respectively obtaining three-dimensional coordinates of the target points according to the sub-pixel point corresponding sets which are in one-to-one correspondence and the optical triangular model, and completing the multi-target parallel measurement based on the binocular stereo vision.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above-mentioned contents are only for illustrating the technical idea of the present invention, and the protection scope of the present invention is not limited thereby, and any modification made on the basis of the technical idea of the present invention falls within the protection scope of the claims of the present invention.
Claims (10)
1. A multi-target parallel measurement method based on binocular stereo vision is characterized by comprising the following steps:
step 1, calibrating a binocular stereo vision measuring unit by adopting a checkerboard calibration method to obtain a conversion relation between a world coordinate system and an image pixel coordinate system and a distortion parameter of an image;
step 2, outputting a video stream by using a calibrated binocular stereo vision measuring unit, adopting a left camera and a right camera in the binocular stereo vision measuring unit to respectively capture a pair of images at the same moment aiming at the output video stream, preprocessing the obtained pair of images, respectively carrying out segmentation and splicing processing on the preprocessed pair of images, and obtaining a plurality of corresponding pairs of images only with coding targets according to a plurality of targets needing to be identified;
step 3, respectively traversing the multiple pairs of images, quickly extracting angular point information to obtain an initial sub-pixel point corresponding set, and identifying, decoding and matching the codes of each coding target to obtain one-to-one corresponding sub-pixel point corresponding sets;
and 4, respectively obtaining three-dimensional coordinates of the target points according to the sub-pixel point corresponding sets which are in one-to-one correspondence and the optical triangular model, and completing the multi-target parallel measurement based on the binocular stereo vision.
2. The method for multi-target parallel measurement based on binocular stereo vision according to claim 1, wherein in step 1, a checkerboard calibration method is performed based on a MATLAB calibration toolkit.
3. The binocular stereo vision-based multi-target parallel measurement method according to claim 1, wherein in the step 1, a short baseline and an optical axis included angle of 20-30 degrees are adopted in the binocular stereo vision measurement unit.
4. The binocular stereo vision-based multi-target parallel measurement method according to claim 3, wherein the short baseline is 30-50 mm.
5. The method for multi-target parallel measurement based on binocular stereo vision according to claim 1, wherein in the step 2, the preprocessing comprises: firstly, filtering processing is carried out, and then binarization processing and contour detection are carried out.
6. The method as claimed in claim 1, wherein in step 3, the coded target is a ring-shaped coded mark, and black and white regions are provided thereon, the binary code corresponding to the black region is 0, the binary code corresponding to the white region is 1, the rings are concentrically coded in a counterclockwise order by using any one coded ring unit as a starting point to form a multi-bit binary string, and a multi-bit coded mark point has a plurality of different binary strings, wherein the decimal value corresponding to the binary with the smallest value can be represented as the decoded value of the coded mark point.
7. The method for multi-target parallel measurement based on binocular stereo vision according to claim 1, wherein in step 4, the three-dimensional coordinates of the target points are respectively obtained according to the obtained one-to-one corresponding sub-pixel point corresponding sets and the optical triangular model, and the method specifically comprises the following steps:
distortion correction is carried out on the corner point information which is obtained by correspondingly and intensively completing coding sequencing on the sub-pixel points which are obtained in the step 1 in a one-to-one correspondence mode according to the distortion parameters, an equation set is established according to the optical triangular model and the conversion relation between the world coordinate system and the image pixel coordinate system which are obtained through calibration, the corner point information which is obtained by completing coding sequencing is correspondingly brought into the equation set in sequence, the three-dimensional coordinates of the target point to be detected are solved through the least square method, and the three-dimensional coordinates of the target point are obtained respectively.
8. The utility model provides a multi-target parallel measurement system based on binocular stereo vision which characterized in that includes:
the camera calibration module is used for calibrating the binocular stereoscopic vision measurement unit by adopting a checkerboard calibration method to obtain a conversion relation between a world coordinate system and an image pixel coordinate system and a distortion parameter of an image;
the image processing module is used for outputting a video stream by using the calibrated binocular stereo vision measuring unit, adopting a left camera and a right camera in the binocular stereo vision measuring unit to respectively capture a pair of images at the same moment aiming at the output video stream, preprocessing the obtained pair of images, respectively segmenting and splicing the preprocessed pair of images, and obtaining a plurality of corresponding pairs of images only with coding targets according to a plurality of targets needing to be identified;
the image target point extracting and matching module is used for respectively traversing the multiple pairs of images, extracting angular point information to obtain an initial sub-pixel point corresponding set, and identifying, decoding and matching the codes of each coding target to obtain one-to-one corresponding sub-pixel point corresponding sets;
and the three-dimensional reconstruction module is used for respectively obtaining the three-dimensional coordinates of the target points according to the sub-pixel point corresponding sets which are in one-to-one correspondence and the optical triangular model.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the binocular stereo based multi-objective parallel measurement method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the steps of the binocular stereo based multi-objective parallel measurement method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110663144.5A CN113513981B (en) | 2021-06-15 | 2021-06-15 | Multi-target parallel measurement method, system, equipment and storage medium based on binocular stereo vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110663144.5A CN113513981B (en) | 2021-06-15 | 2021-06-15 | Multi-target parallel measurement method, system, equipment and storage medium based on binocular stereo vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113513981A true CN113513981A (en) | 2021-10-19 |
| CN113513981B CN113513981B (en) | 2022-10-25 |
Family
ID=78065420
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110663144.5A Active CN113513981B (en) | 2021-06-15 | 2021-06-15 | Multi-target parallel measurement method, system, equipment and storage medium based on binocular stereo vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113513981B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115830118A (en) * | 2022-12-08 | 2023-03-21 | 重庆市信息通信咨询设计院有限公司 | Crack detection method and system for cement electric pole based on binocular camera |
| CN115880376A (en) * | 2023-03-01 | 2023-03-31 | 中科慧眼(天津)研究开发有限公司 | Binocular camera depth calibration method and system based on deep learning |
| CN116563292A (en) * | 2023-07-11 | 2023-08-08 | 聚时科技(深圳)有限公司 | Measurement method, detection device, detection system, and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105469418A (en) * | 2016-01-04 | 2016-04-06 | 中车青岛四方机车车辆股份有限公司 | Photogrammetry-based wide-field binocular vision calibration device and calibration method |
| CN107358631A (en) * | 2017-06-27 | 2017-11-17 | 大连理工大学 | A kind of binocular vision method for reconstructing for taking into account three-dimensional distortion |
| CN109272570A (en) * | 2018-08-16 | 2019-01-25 | 合肥工业大学 | A kind of spatial point three-dimensional coordinate method for solving based on stereoscopic vision mathematical model |
| CN112161586A (en) * | 2020-11-20 | 2021-01-01 | 苏州睿牛机器人技术有限公司 | Line structured light vision sensor calibration method based on coding checkerboard |
-
2021
- 2021-06-15 CN CN202110663144.5A patent/CN113513981B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105469418A (en) * | 2016-01-04 | 2016-04-06 | 中车青岛四方机车车辆股份有限公司 | Photogrammetry-based wide-field binocular vision calibration device and calibration method |
| CN107358631A (en) * | 2017-06-27 | 2017-11-17 | 大连理工大学 | A kind of binocular vision method for reconstructing for taking into account three-dimensional distortion |
| CN109272570A (en) * | 2018-08-16 | 2019-01-25 | 合肥工业大学 | A kind of spatial point three-dimensional coordinate method for solving based on stereoscopic vision mathematical model |
| CN112161586A (en) * | 2020-11-20 | 2021-01-01 | 苏州睿牛机器人技术有限公司 | Line structured light vision sensor calibration method based on coding checkerboard |
Non-Patent Citations (1)
| Title |
|---|
| 林达: "城市主战消防车自动消防炮研究", 《中国优秀硕士学位论文全文数据库（电子期刊）工程科技Ⅱ辑》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115830118A (en) * | 2022-12-08 | 2023-03-21 | 重庆市信息通信咨询设计院有限公司 | Crack detection method and system for cement electric pole based on binocular camera |
| CN115830118B (en) * | 2022-12-08 | 2024-03-19 | 重庆市信息通信咨询设计院有限公司 | Crack detection method and system for cement electric pole based on binocular camera |
| CN115880376A (en) * | 2023-03-01 | 2023-03-31 | 中科慧眼(天津)研究开发有限公司 | Binocular camera depth calibration method and system based on deep learning |
| CN116563292A (en) * | 2023-07-11 | 2023-08-08 | 聚时科技(深圳)有限公司 | Measurement method, detection device, detection system, and storage medium |
| CN116563292B (en) * | 2023-07-11 | 2023-09-26 | 聚时科技(深圳)有限公司 | Measurement method, detection device, detection system, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113513981B (en) | 2022-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113513981B (en) | Multi-target parallel measurement method, system, equipment and storage medium based on binocular stereo vision | |
| CN109859272B (en) | Automatic focusing binocular camera calibration method and device | |
| KR101121034B1 (en) | System and method for obtaining camera parameters from multiple images and computer program products thereof | |
| WO2016037486A1 (en) | Three-dimensional imaging method and system for human body | |
| CN113205593B (en) | High-light-reflection surface structure light field three-dimensional reconstruction method based on point cloud self-adaptive restoration | |
| CN109544628B (en) | Accurate reading identification system and method for pointer instrument | |
| CN109427046B (en) | Distortion correction method and device for three-dimensional measurement and computer readable storage medium | |
| CN113129430B (en) | Underwater three-dimensional reconstruction method based on binocular structured light | |
| WO2022126870A1 (en) | Three-dimensional imaging method and method based on light field camera and three-dimensional imaging measuring production line | |
| CN112686950B (en) | Pose estimation method, pose estimation device, terminal equipment and computer readable storage medium | |
| Li et al. | A practical comparison between Zhang's and Tsai's calibration approaches | |
| CN110136048B (en) | Image registration method and system, storage medium and terminal | |
| CN112257713A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN115761126A (en) | Three-dimensional reconstruction method and device based on structured light, electronic equipment and storage medium | |
| Gu et al. | Dot-coded structured light for accurate and robust 3D reconstruction | |
| CN116468764A (en) | Multi-view industrial point cloud high-precision registration system based on super-point space guidance | |
| CN114332349B (en) | Binocular structured light edge reconstruction method, system and storage medium | |
| CN116935013B (en) | Circuit board point cloud large-scale splicing method and system based on three-dimensional reconstruction | |
| Chen et al. | Finding optimal focusing distance and edge blur distribution for weakly calibrated 3-D vision | |
| CN111383264A (en) | Positioning method, positioning device, terminal and computer storage medium | |
| CN117333367A (en) | Image stitching method, system, medium and device based on image local features | |
| CN116205993A (en) | Double-telecentric lens high-precision calibration method for 3D AOI | |
| CN108564626B (en) | Method and apparatus for determining relative pose angle between cameras mounted to an acquisition entity | |
| CN116295113A (en) | Polarization three-dimensional imaging method integrating fringe projection | |
| Song et al. | Automatic calibration method based on improved camera calibration template |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |