CN115601423A - Edge enhancement-based round hole pose measurement method in binocular vision scene - Google Patents
Edge enhancement-based round hole pose measurement method in binocular vision scene Download PDFInfo
- Publication number
- CN115601423A CN115601423A CN202211115345.2A CN202211115345A CN115601423A CN 115601423 A CN115601423 A CN 115601423A CN 202211115345 A CN202211115345 A CN 202211115345A CN 115601423 A CN115601423 A CN 115601423A
- Authority
- CN
- China
- Prior art keywords
- image
- round hole
- edge
- resolution
- pose
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20061—Hough transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a round hole pose measuring method based on edge enhancement in a binocular vision measuring scene. Firstly, in a preprocessing stage of an image hyper-resolution reconstruction process, a model is helped to learn the position and the degree of the hyper-resolution reconstruction through fuzzy filling operation; based on an enhanced countermeasure generation network, edge enhancement of the circular hole super-resolution reconstruction image is realized by adding cross-layer residual connection, introducing context loss in a perception loss item, adding a pixel rearrangement mechanism and the like; further extracting sub-pixel level fine edges of the round holes, and fitting parameters by combining a Hough transform method improved by geometric characteristics; completing calibration on the binocular camera by a calibration method of multi-constraint fusion; and measuring the pose parameters of the round holes by using a stereo matching technology through the constructed binocular camera observation model. According to the invention, the accuracy and stability of the position and pose measurement of the round hole are improved by edge enhancement and fine round hole edge extraction and fitting methods.
Description
Technical Field
The invention relates to a round hole pose vision measurement method, in particular to a round hole pose measurement method based on edge enhancement in a binocular vision scene.
Background
The circular hole characteristic is one of the most common characteristics of parts, and is commonly used in shaft sleeve type parts, disc cover type parts and box body type parts. In the actual production process, the assembling and butt joint process of the parts with holes also often takes round holes as characteristics, and the automatic assembling, butt joint and other processes of the parts with holes taking the holes as characteristics need to ensure the coincidence of the axes between the shafts and the holes. Therefore, it is particularly important to realize real-time and high-precision detection of the pose of the target circular hole.
At present, the existing round hole pose vision measurement method is limited by the influence of limited performance of hardware equipment of a vision acquisition system such as a lens, a camera, an image acquisition card and the like, illumination change, complex background and other factors, so that the problems of insufficient resolution of an acquired image, unclear round hole edge and large measurement error are often caused. Therefore, rapid and high-precision round hole pose measurement is still a challenging task.
Safaee-Rad et al in 1992 in the IEEE Transactions on Robotics and Automation "Three-dimensional localization estimation of circular features for machine vision" propose a general equation passing through a cone to the center thereof, provide an analytical solution of a closed form for circular hole pose measurement, and propose solutions of two closed forms according to whether the radius is known or not. Xu et al, 2012 in 12th International Conference on control, automation, robotics and Vision, "A position measurement method of a non-cooperative GEO spaced based on stereo Vision", solved the orientation duality problem, proposed to estimate the pose of a circular hole from the left and right images, and then removed the monocular Vision ambiguity solution according to the constraint condition that the measured pose difference of the left and right images should be within the threshold. Liu et al, in 2016, "position measurement of a non-cooperative spatial base on circular features" in IEEE International Conference on Real-time Computing and Robotics, proposed a closed solution for Pose measurement of a circular hole of unknown radius, using a quadratic form of an ellipse in an image and a pinhole projection model to obtain a quadratic curve of a 3D circle. And then calculating the pose information of the circle by constructing an equation according to a quadratic curve of the 3D circle obtained by equaling the ellipses in the left and right images. Zhang in 2019 in the master academic thesis round hole pose visual detection based on image super-resolution reconstruction, proposed that image resolution is improved by an image super-resolution reconstruction method, and further round hole pose measurement accuracy is improved. Xia et al propose pixel intensity information based on in 2020' paper "An accurate and robust method for the Measurement of spatial hole based on binary vision" and combine epipolar geometric constraint to obtain a high-precision circular hole edge point, and then three-dimensionally reconstruct the edge point to obtain a spatial circular hole.
The research improves the precision of the measurement of the pose of the round hole, but still faces the problems of fuzzy edge of the round hole, inaccurate high-frequency information and long time consumption.
Disclosure of Invention
In order to solve the problems in the background art, the invention provides a round hole pose measuring method based on edge enhancement in a binocular vision measuring scene, and the method has the advantages of high speed, high efficiency, high measuring precision and the like.
In the invention, firstly, in the process of image super-resolution reconstruction, edge enhancement of the round hole super-resolution reconstruction image is realized by methods of fuzzy filling, network addition of cross-layer residual connection, introduction of context loss in a perception loss item, addition of a pixel rearrangement mechanism and the like; secondly, extracting sub-pixel level edges of the circular holes, and calibrating the binocular camera by combining Hough transform method fitting parameters improved by geometric features and a calibration method of multi-constraint fusion; and finally, measuring the pose parameters of the round holes by using a stereo matching technology through the constructed binocular camera observation model.
The invention adopts the specific technical scheme that:
1) Shooting a hole-containing part by using a binocular camera to obtain a plurality of high-resolution round hole images, respectively preprocessing each high-resolution round hole image to obtain a corresponding fuzzy round hole image, and forming a data set by each high-resolution round hole image and the corresponding fuzzy round hole image;
2) Inputting the data set into a round hole image super-resolution reconstruction depth network to train the network, and obtaining a trained round hole image super-resolution reconstruction depth network;
3) The binocular camera collects a high-resolution round hole image to be reconstructed and inputs the high-resolution round hole image to the trained round hole image hyper-resolution reconstruction depth network, and the network outputs the edge-enhanced high-resolution round hole image;
4) Performing sub-pixel level edge extraction on the edge-enhanced high-resolution circular hole image by using a circular hole edge sub-pixel level extraction method to obtain a sub-pixel level edge image;
5) Fitting circular hole parameters of a sub-pixel level edge image by combining a Hough transform method improved by geometric characteristics to obtain parameters of a target circular hole;
6) According to the parameters of the target round hole, coordinate mapping is carried out by using the super-resolution reconstruction camera observation model, and pose information of the target round hole in a camera coordinate system is obtained;
7) And calculating and obtaining the pose parameters of the target round hole in the world coordinate system by using a stereo matching method based on a binocular camera according to the pose information of the target round hole in the camera coordinate system, and finishing the measurement of the pose of the round hole.
In the step 1), each high-resolution round hole image is subjected to degradation processing to obtain a corresponding low-resolution round hole image, each high-resolution round hole image and the corresponding low-resolution round hole image form each pair of round hole image pairs, and the round hole image pairs are subjected to fuzzy filling to obtain fuzzy round hole images.
The round hole image hyper-resolution reconstruction depth network is composed of a Pixel rearrangement mechanism module and an improved enhanced countermeasure generation network cascade, wherein the Pixel rearrangement mechanism module is obtained by carrying out inverse operation on a Pixel-shuffle module, and the improved enhanced countermeasure generation network is obtained by carrying out cross-layer residual connection on 3 RRDB blocks in the enhanced countermeasure generation network.
And replacing an L2 loss function in the perception loss function of the enhanced countermeasure generation network with a context loss function.
The step 4) is specifically as follows:
4.1 The edge-enhanced high-resolution round hole image is input into a round hole semantic segmentation model for prediction, and a round hole edge Mask is output;
4.2 Utilizing an edge extraction operator to perform edge extraction on the edge-enhanced high-resolution circular hole image to obtain a part initial edge image, and multiplying the part initial edge image by a circular hole edge Mask to obtain a part coarse edge image;
4.3 Performing edge fine extraction on the part rough edge image to obtain an accurate edge image;
4.4 The sub-pixel level edge image is obtained through calculation according to the edge-enhanced high-resolution circular hole image and the accurate edge image.
In the step 4.1), the circular hole semantic segmentation model is formed by adding an attention mechanism module in front of a pyramid pooling ASPP module of the Deeplabv3+ semantic segmentation network, and the pyramid pooling ASPP module is cascaded with the attention mechanism module.
In the step 4.2), the edge extraction operator is specifically a roberts operator.
The step 4.3) is specifically as follows:
and sequentially fitting, connecting and refining the hole edge of the rough edge image of the part to obtain an accurate edge image.
The step 4.4) is specifically as follows:
and extracting full-image sub-pixel points of the to-be-predicted hole-containing part image to obtain full-image sub-pixel points, and performing intersection operation on the full-image sub-pixel points and the accurate edge image to obtain circular hole edge sub-pixel points, so that a sub-pixel level edge image is obtained.
The super-resolution reconstruction camera observation model is specifically a mapping relation between a camera coordinate system of a binocular camera and a virtual coordinate system of a reconstructed image.
The invention has the beneficial effects that:
1) The training set is preprocessed through a fuzzy filling method, so that the super-resolution reconstruction depth network learns the super-resolution reconstruction position and the super-resolution reconstruction scale, the reconstruction image is prevented from being excessively sharpened, and the network reconstruction process has a regularization effect.
2) By adding cross-layer residual connection in the super-resolution reconstruction depth network, introducing context loss in a perception loss item and introducing a pixel rearrangement mechanism module, the edge enhancement of the reconstructed round hole image is realized, and the high-resolution image with rich high-frequency information and real edge details is reconstructed.
3) The fine edges of the round holes are extracted by a round hole edge sub-pixel level extraction method, the ellipse parameters are calculated by a Hough transform ellipse parameter fitting method combined with geometric feature improvement, and the precision of the round hole pose parameters measured by a stereo matching method is further improved by reconstructing a camera observation model based on super-resolution.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is an example of a preprocessed image pair of a sample library of hole-containing parts.
FIG. 3 is an example of a reconstructed edge-enhanced circular hole image.
FIG. 4 is a round hole edge sub-pixel extraction process.
FIG. 5 is a circular hole image ellipse detection process.
Fig. 6 is a block diagram of a multi-constraint fusion binocular calibration process.
FIG. 7 is an example of a measurement result of the attitude of a circular hole of a typical hole-containing part.
Fig. 8 is a schematic diagram of the structure of 3 RRDB blocks in an improved enhanced countermeasure generation network.
Detailed Description
The invention is further illustrated with reference to the following figures and examples.
The embodiment of the invention and the implementation process thereof are as follows:
as shown in fig. 1, the present invention comprises the steps of:
1) In specific implementation, the binocular cameras are calibrated by combining 2D and 3D fusion constraints to obtain calibrated binocular cameras, the calibrated binocular cameras are used for acquisition in the following steps, and calibration accuracy is improved by the 2D-3D multi-constraint fusion camera calibration method. As shown in fig. 6, the calibration flow chart calculates camera calibration errors such as adjacent distance error, collinear error, right angle error and the like of the 2D image layer based on the zhangying friend camera calibration method; calculating a reprojection error and an epipolar error at a 3D level, namely fusing constraint conditions of a 2D image level and a 3D space level; and a Levenberg-Marquardt iterative algorithm is used for solving the optimal solution of the camera parameters to complete the calibration of the binocular camera.
Shooting a hole-containing part by using a binocular camera to obtain a plurality of high-resolution round hole images, as shown in fig. 2 (a), respectively preprocessing each high-resolution round hole image to obtain a corresponding fuzzy round hole image, and forming a data set by each high-resolution round hole image and the corresponding fuzzy round hole image;
in the step 1), performing degradation processing on each high-resolution round hole image respectively to obtain a corresponding low-resolution round hole image, as shown in fig. 2 (b), specifically, performing noise addition, interpolation and JPEG compression on the high-resolution round hole image to generate a corresponding low-resolution round hole image, wherein the three steps of noise addition, interpolation and JPEG compression are not in sequence. Each pair of circular hole image pairs is formed by each high-resolution circular hole image and the corresponding low-resolution circular hole image, wherein the resolutions of the high-resolution circular hole image and the corresponding low-resolution circular hole image are relative to each other, and the circular hole image pairs are subjected to fuzzy filling to obtain fuzzy circular hole images, as shown in fig. 2 (c), specifically: and amplifying the low-resolution round hole image of the round hole image pair to the same size as the high-resolution image through up-sampling, then cutting the round hole area of the high-resolution image and filling the round hole area to the corresponding area of the low-resolution round hole image to obtain a fuzzy round hole image.
2) Inputting the data set into a round hole image hyper-resolution reconstruction depth network to train the network, and obtaining the trained round hole image hyper-resolution reconstruction depth network;
the round hole image hyper-resolution reconstruction depth network is composed of a Pixel rearrangement mechanism module and an improved enhanced countermeasure generation network cascade, wherein the Pixel rearrangement mechanism module is obtained by carrying out inverse operation on a Pixel-shuffle module, and a 2-time hyper-resolution reconstruction process of an image is taken as an example: and the Pixel up-sampling module Pixel-buffer performs 2 times up-sampling operation on the image to realize the image super-resolution reconstruction target of multiplied by 2. Before the super-resolution reconstruction of the image, a pixel rearrangement mechanism module is introduced, namely, 2 times of down-sampling operation is firstly carried out on the image to be reconstructed, the space size is reduced, and meanwhile, the information is rearranged to the channel dimension; and performing 4 times of super-resolution reconstruction to finally realize the image super-resolution reconstruction target of multiplied by 2.
The improved enhanced countermeasure generation network is obtained by connecting 3 RRDB blocks in the enhanced countermeasure generation network with cross-layer residual errors; namely, the original one-way linear information transmission mode of 3 RRDB blocks in the enhanced countermeasure generation network is converted into cross-layer residual connection, information transmission among different residual blocks is realized, and a cross-layer super-resolution reconstruction network based on the enhanced countermeasure generation network architecture is built. As shown in fig. 8, specifically: the input of the first RRDB block is subjected to channel addition with the output of the first RRDB block after passing through the first convolution layer and then input into the second RRDB block, the input of the second RRDB block is subjected to channel addition with the output of the second RRDB block after passing through the second convolution layer and then input into the third RRDB block, the input of the third RRDB block is subjected to channel addition with the output of the third RRDB block after passing through the third convolution layer and then output as the first output, and the input of the first RRDB block is subjected to channel addition with the first output after passing through the fourth convolution layer and then output as the input of the average pooling layer.
The L2 loss function is replaced by a context loss function in the perception loss function of the enhanced countermeasure generation network.
The formula for the perceptual loss function is as follows:
wherein L is G Is a value of a perceptual loss function, L CX Is the value of the context loss function,
is the value of the generation loss, λ and η are the coefficient factors regulating the generation loss and the content loss, respectively, L 1 High resolution circular hole image G (x) representing edge enhancement obtained by network reconstruction i ) And the content between the high-resolution circular hole image y and the high-resolution circular hole image y is lost by 1 norm distance.
The context loss function is shown as:
wherein L is CX The (-) expression shows the context loss at a certain point in the edge-enhanced high-resolution round hole image obtained by network reconstruction, D (-) is a standard discrimination network, x and y are respectively a low-resolution image block and a corresponding high-resolution image block corresponding to the training set image pair,
for the k low resolution image block x k And the jth high-resolution image block y j The cosine distance between the two image blocks is normalized, h is a scale factor, and N represents the number of the high-resolution image blocks. exp (-) denotes an exponent operation, max denotes a maximum value operation, and log (-) denotes a logarithm operation.
3) A binocular camera collects a high-resolution round hole image to be reconstructed and inputs the high-resolution round hole image to a trained round hole image super-resolution reconstruction depth network, and the network outputs a high-resolution round hole image with enhanced edge; as shown in fig. 3, (a) of fig. 3 is a high-resolution circular hole image to be reconstructed, and (b) of fig. 3 is an edge-enhanced high-resolution circular hole image.
4) Performing sub-pixel level edge extraction on the edge-enhanced high-resolution circular hole image by using a circular hole edge sub-pixel level extraction method to obtain a sub-pixel level edge image;
the step 4) is specifically as follows:
4.1 The edge-enhanced high-resolution round hole image is input into a round hole semantic segmentation model for prediction, and a round hole edge Mask is output; as shown in fig. 4, (a) of fig. 4 is an output round hole edge Mask, and (b) of fig. 4 is a result diagram obtained by pasting the round hole edge Mask to the image of the hole-containing component to be predicted.
The circular hole semantic segmentation model is formed by adding an attention mechanism module in front of a pyramid pooling ASPP module of a Deeplabv3+ semantic segmentation network, the pyramid pooling ASPP module is cascaded with the attention mechanism module, and a module which is arranged in front of the pyramid pooling ASPP module and cascaded with the pyramid pooling ASPP module in the Deeplabv3+ semantic segmentation network is also cascaded with the attention mechanism module. The image of the hole-containing part is screened for characteristic information by an attention mechanism module, and then semantic segmentation is performed on the result in a targeted manner.
4.2 Utilizing an edge extraction operator to perform edge extraction on the edge-enhanced high-resolution circular hole image to obtain a part initial edge image, and multiplying the part initial edge image by a circular hole edge Mask to obtain a part coarse edge image;
in the step 4.2), the edge extraction operator is specifically a roberts operator.
4.3 Performing edge fine extraction on the part rough edge image to obtain an accurate edge image;
the step 4.3) is specifically as follows:
and sequentially fitting, connecting and refining the hole edge of the rough edge image of the part to obtain an accurate edge image.
4.4 According to the edge-enhanced high-resolution circular hole image and the accurate edge image, calculating to obtain a sub-pixel level edge image.
The step 4.4) is specifically as follows:
and (3) extracting full-image sub-pixel points of the to-be-predicted hole-containing part image to obtain full-image sub-pixel points, and performing intersection operation on the full-image sub-pixel points and the accurate edge image to obtain circular hole edge sub-pixel points as shown in (a) of fig. 5, so as to obtain a sub-pixel level edge image as shown in (b) of fig. 5.
5) Fitting circular hole parameters of the sub-pixel level edge image by combining a Hough transform method improved by geometric characteristics to obtain parameters of a target circular hole, wherein the parameters of the target circular hole comprise a circle center coordinate and a normal vector;
the improved Hough transform method combined with the geometric features specifically comprises the following steps:
firstly, carrying out edge detection on an image edge, and storing a detected point coordinate into an array; secondly, calculating the maximum distance from the middle point of the distance array, wherein the point with the minimum maximum distance in all the points is the center of the ellipse, and the maximum distance is the length of the major axis of the ellipse; thirdly, substituting the obtained numerical value of the point and the ellipse parameters obtained in the process into an ellipse equation, counting the rest parameters, and defining a relevant threshold; and finally, outputting the parameter with the peak value exceeding the threshold value as the target ellipse parameter.
Theoretically, the following theorem exists: and (3) setting the center of the ellipse on the plane as a point c, arbitrarily taking 1 point different from the point c on the plane, and recording the point as a point p, wherein the maximum distance from the point p to the point on the ellipse is larger than the maximum distance from the point c to the point on the ellipse.
According to the above principle, the calculation process of the ellipse parameters is as follows:
the major axis, the minor axis and the rotation angle of the target ellipse are respectively recorded as a, b and theta, and the central point of the ellipse is marked as a point (p, q), so that the calculation process of the ellipse equation is shown as the following formula:
the specific detection process comprises the following steps: and carrying out edge detection on the image edge, and storing the detected point coordinates into an array. And calculating the maximum distance from the middle points of the distance array, wherein the point with the minimum maximum distance in all the points is the center (p, q) of the ellipse, and the maximum distance is the length a of the long axis of the ellipse. And substituting the obtained numerical value of the point and the ellipse parameters p, q and a obtained in the process into the ellipse equation, counting the parameters b and theta, defining a relevant threshold value, and outputting the parameter with the peak value exceeding the threshold value, namely the target ellipse parameter.
6) According to the parameters of the target round hole, coordinate mapping is carried out by utilizing a super-resolution reconstruction camera observation model, and pose information of the target round hole in a camera coordinate system is obtained;
the super-resolution reconstruction camera observation model is specifically a mapping relation between a camera coordinate system of a binocular camera and a virtual coordinate system of a reconstructed image. The reconstructed image virtual coordinate system is an image coordinate system of the image reconstructed by the hyper-resolution reconstruction method.
7) And calculating and obtaining the pose parameters of the target round hole in the world coordinate system by using a stereo matching method based on a binocular camera according to the pose information of the target round hole in the camera coordinate system, and finishing the measurement of the pose of the round hole.
Specifically, the method comprises the following steps: the left camera and the right camera of the binocular camera system can respectively obtain two normal vector measurement values, the included angle between the vectors is calculated, interference factors such as measurement errors and accidental errors are considered, and the average value of the two vectors with the minimum measurement included angle is taken as the final normal vector;
wherein the first and second normal vectors of the round hole of the left camera measuring target are
And
the first and second normal vectors of the right camera measuring target circular hole are
And
the final normal vector is
Represents the angle between the ith normal vector measured by the left camera and the jth normal vector measured by the right camera,
representing the angle between the first normal vector measured by the left camera and the first normal vector measured by the right camera,
representing the angle between a first normal vector measured by the left camera and a second normal vector measured by the right camera,
representing the angle between the second normal vector measured by the left camera and the first normal vector measured by the right camera,
and representing the included angle between the second normal vector measured by the left camera and the second normal vector measured by the right camera.
Similarly, the left camera and the right camera can respectively measure two groups of circle center coordinate values, the final circle center coordinate is obtained according to the least square method, the radius value of the round hole is obtained according to the obtained optimal circle center coordinate value, and the calculation of the pose parameter of the target round hole is completed.
In order to detect the efficiency of the algorithm and verify the over-resolution reconstruction performance and the edge enhancement effect of the invention, the example firstly uses a Basler acA2440-20gm type high-resolution camera (2448 px multiplied by 2048 px) to collect 350 round hole part images, and the round hole part images are expanded to 2000 by image augmentation methods such as turning, cutting and the like to be used as a self-built sample library; and introducing a public super-resolution data set comprising DIV2K and Flickr2K, wherein 3650 pairs of pictures are used as a supplementary database, and the content of the supplementary database is as follows, by 96%:2%: the 2% ratio was randomly divided into a training set, a test set, and a validation set. The adopted evaluation indexes are the peak signal-to-noise ratio (PSNR), the Structural Similarity (SSIM) and the image perception quality (PI) of the local image. The comparative results are shown in Table 1. As can be seen from table 1, compared to the current main super-resolution reconstruction depth network model, the reconstruction effect of the method of the present invention is better and more suitable for human visual perception.
TABLE 1 comparison of over-resolution reconstruction results of round hole parts
In addition, by the method, a typical hole-containing part is selected, the pose of a target round hole of the part is measured, a measurement result graph is shown in fig. 7, and (a) and (b) in fig. 7 are respectively a hole-containing part graph containing checkered corner points and corresponding ellipse detection and reference point detection graphs shot by a binocular camera in an experiment. The comparison with other methods is shown in table 2. Compared with a measuring algorithm which does not use the hyper-resolution reconstruction and uses Zhang, the method provided by the invention has the advantages that the measuring precision and stability show more excellent results.
Table 2 Experimental results of 215mm distance between parts and binocular camera
The experimental results are combined to show that the invention realizes the edge enhancement in the process of the circular hole image super-resolution reconstruction: meanwhile, according to the method disclosed by the invention, the precision of the position and posture information of the round hole measured in a binocular scene is greatly improved; in addition, compared with the existing algorithm, the method has better real-time performance. The method can meet the measurement requirements of high speed, high efficiency and high precision in practical application.
The above examples should not be construed as limiting the present invention, but any modifications made based on the spirit of the present invention should be within the scope of protection of the present invention.
Claims (10)
1. A round hole pose measurement method based on edge enhancement in a binocular vision scene is characterized by comprising the following steps:
1) Shooting hole-containing parts by using a binocular camera to obtain a plurality of high-resolution round hole images, respectively preprocessing each high-resolution round hole image to obtain a corresponding fuzzy round hole image, and forming a data set by each high-resolution round hole image and the corresponding fuzzy round hole image;
2) Inputting the data set into a round hole image hyper-resolution reconstruction depth network to train the network, and obtaining the trained round hole image hyper-resolution reconstruction depth network;
3) A binocular camera collects a high-resolution round hole image to be reconstructed and inputs the high-resolution round hole image to a trained round hole image super-resolution reconstruction depth network, and the network outputs a high-resolution round hole image with enhanced edge;
4) Performing sub-pixel level edge extraction on the edge-enhanced high-resolution circular hole image by using a circular hole edge sub-pixel level extraction method to obtain a sub-pixel level edge image;
5) Fitting circular hole parameters of a sub-pixel level edge image by combining a Hough transform method improved by geometric characteristics to obtain parameters of a target circular hole;
6) According to the parameters of the target round hole, coordinate mapping is carried out by utilizing a super-resolution reconstruction camera observation model, and pose information of the target round hole in a camera coordinate system is obtained;
7) And calculating and obtaining the pose parameters of the target round hole in the world coordinate system by using a stereo matching method based on a binocular camera according to the pose information of the target round hole in the camera coordinate system, and finishing the measurement of the pose of the round hole.
2. The binocular vision scene edge enhancement-based round hole pose measurement method according to claim 1, wherein in the step 1), each high-resolution round hole image is subjected to degradation processing to obtain a corresponding low-resolution round hole image, each high-resolution round hole image and the corresponding low-resolution round hole image form each pair of round hole image pairs, and the round hole image pairs are subjected to fuzzy filling to obtain fuzzy round hole images.
3. The round hole pose measurement method based on edge enhancement in binocular visual scene according to claim 1, wherein the round hole image super-resolution reconstruction depth network is composed of a Pixel rearrangement mechanism module and an improved enhanced countermeasure generation network cascade, the Pixel rearrangement mechanism module is obtained after inverse operation is performed on a Pixel upsampling module Pixel-shuffle, and the improved enhanced countermeasure generation network is obtained after cross-layer residual error connection is performed on 3 RRDB blocks in the enhanced countermeasure generation network.
4. The binocular vision scene edge enhancement-based round hole pose measurement method according to claim 3, wherein an L2 loss function in a perception loss function of the enhanced countermeasure generation network is replaced by a context loss function.
5. The round hole pose measurement method based on edge enhancement in the binocular vision scene according to claim 1, wherein the step 4) specifically comprises:
4.1 The edge-enhanced high-resolution round hole image is input into a round hole semantic segmentation model for prediction, and a round hole edge Mask is output;
4.2 Utilizing an edge extraction operator to perform edge extraction on the edge-enhanced high-resolution circular hole image to obtain a part initial edge image, and multiplying the part initial edge image by a circular hole edge Mask to obtain a part coarse edge image;
4.3 Performing edge fine extraction on the part rough edge image to obtain an accurate edge image;
4.4 The sub-pixel level edge image is obtained through calculation according to the edge-enhanced high-resolution circular hole image and the accurate edge image.
6. The round hole pose measurement method based on edge enhancement in the binocular vision scene according to claim 5, wherein in the step 4.1), the round hole semantic segmentation model is formed by adding an attention mechanism module in front of a pyramid pooling ASPP module of a Deeplabv3+ semantic segmentation network, and the pyramid pooling ASPP module and the attention mechanism module are cascaded.
7. The round hole pose measurement method based on edge enhancement in the binocular vision scene according to claim 5, wherein in the step 4.2), the edge extraction operator is specifically a roberts operator.
8. The round hole pose measurement method based on edge enhancement in the binocular vision scene according to claim 5, wherein the step 4.3) is specifically as follows:
and sequentially fitting, connecting and refining the hole edge of the rough edge image of the part to obtain an accurate edge image.
9. The round hole pose measurement method based on edge enhancement in the binocular vision scene according to claim 5, wherein the step 4.4) specifically comprises:
and extracting full-image sub-pixel points of the to-be-predicted hole-containing part image to obtain full-image sub-pixel points, and performing intersection operation on the full-image sub-pixel points and the accurate edge image to obtain circular hole edge sub-pixel points, so that a sub-pixel level edge image is obtained.
10. The round hole pose measurement method based on edge enhancement in the binocular vision scene as recited in claim 1, wherein the super-resolution reconstruction camera observation model is a mapping relation between a camera coordinate system of a binocular camera and a virtual coordinate system of a reconstructed image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211115345.2A CN115601423A (en) | 2022-09-14 | 2022-09-14 | Edge enhancement-based round hole pose measurement method in binocular vision scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211115345.2A CN115601423A (en) | 2022-09-14 | 2022-09-14 | Edge enhancement-based round hole pose measurement method in binocular vision scene |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115601423A true CN115601423A (en) | 2023-01-13 |
Family
ID=84843152
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211115345.2A Pending CN115601423A (en) | 2022-09-14 | 2022-09-14 | Edge enhancement-based round hole pose measurement method in binocular vision scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115601423A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115880376A (en) * | 2023-03-01 | 2023-03-31 | 中科慧眼(天津)研究开发有限公司 | Binocular camera depth calibration method and system based on deep learning |
-
2022
- 2022-09-14 CN CN202211115345.2A patent/CN115601423A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115880376A (en) * | 2023-03-01 | 2023-03-31 | 中科慧眼(天津)研究开发有限公司 | Binocular camera depth calibration method and system based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Strecha et al. | On benchmarking camera calibration and multi-view stereo for high resolution imagery | |
| CN107833181B (en) | Three-dimensional panoramic image generation method based on zoom stereo vision | |
| CN110009674B (en) | Monocular image depth of field real-time calculation method based on unsupervised depth learning | |
| CN111340944B (en) | Single-image human body three-dimensional reconstruction method based on implicit function and human body template | |
| CN108876836A (en) | A kind of depth estimation method, device, system and computer readable storage medium | |
| CN116309757B (en) | Binocular stereo matching method based on machine vision | |
| CN111043988B (en) | Single stripe projection measurement method based on graphics and deep learning | |
| CN113393439A (en) | Forging defect detection method based on deep learning | |
| CN116129037B (en) | Visual touch sensor, three-dimensional reconstruction method, system, equipment and storage medium thereof | |
| CN115147709B (en) | Underwater target three-dimensional reconstruction method based on deep learning | |
| CN116778288A (en) | Multi-mode fusion target detection system and method | |
| CN116958419A (en) | Binocular stereoscopic vision three-dimensional reconstruction system and method based on wavefront coding | |
| CN115601423A (en) | Edge enhancement-based round hole pose measurement method in binocular vision scene | |
| CN112581626B (en) | Complex curved surface measurement system based on non-parametric and multi-attention force mechanism | |
| CN117115359A (en) | Multi-view power grid three-dimensional space data reconstruction method based on depth map fusion | |
| Loaiza et al. | Matching segments in stereoscopic vision | |
| CN117197333A (en) | Space target reconstruction and pose estimation method and system based on multi-view vision | |
| CN116091793A (en) | Light field significance detection method based on optical flow fusion | |
| CN115731345A (en) | Human body three-dimensional reconstruction method based on binocular vision | |
| CN114943776A (en) | Three-dimensional reconstruction method and device based on cross-correlation function and normal vector loss | |
| CN113808019A (en) | Non-contact measurement system and method | |
| Xie et al. | 3D Reconstruction Method of Image Based on Digital Microscope. | |
| CN112529960A (en) | Target object positioning method and device, processor and electronic device | |
| CN112381721A (en) | Human face three-dimensional reconstruction method based on binocular vision | |
| Lee et al. | 3D surface reconstruction using structured circular light patterns |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |