CN117764983A - Visual detection method for binocular identification of intelligent manufacturing production line - Google Patents
Visual detection method for binocular identification of intelligent manufacturing production line Download PDFInfo
- Publication number
- CN117764983A CN117764983A CN202410041698.5A CN202410041698A CN117764983A CN 117764983 A CN117764983 A CN 117764983A CN 202410041698 A CN202410041698 A CN 202410041698A CN 117764983 A CN117764983 A CN 117764983A
- Authority
- CN
- China
- Prior art keywords
- image
- fitting
- gray
- gray level
- edge
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004519 manufacturing process Methods 0.000 title claims abstract description 43
- 238000001514 detection method Methods 0.000 title claims abstract description 29
- 230000000007 visual effect Effects 0.000 title abstract description 9
- 238000000034 method Methods 0.000 claims abstract description 67
- 238000001914 filtration Methods 0.000 claims abstract description 30
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 25
- 238000011179 visual inspection Methods 0.000 claims abstract description 25
- 238000003708 edge detection Methods 0.000 claims abstract description 21
- 238000012937 correction Methods 0.000 claims abstract description 10
- 238000007781 pre-processing Methods 0.000 claims abstract description 5
- 238000012545 processing Methods 0.000 claims description 18
- 238000010586 diagram Methods 0.000 claims description 10
- 230000001629 suppression Effects 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000014509 gene expression Effects 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000005516 engineering process Methods 0.000 claims description 5
- 238000009499 grossing Methods 0.000 claims description 5
- 230000000295 complement effect Effects 0.000 claims description 4
- 230000001186 cumulative effect Effects 0.000 claims description 4
- 238000005315 distribution function Methods 0.000 claims description 4
- 238000003384 imaging method Methods 0.000 claims description 4
- 230000002159 abnormal effect Effects 0.000 claims description 3
- 238000004891 communication Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 2
- 230000009466 transformation Effects 0.000 claims description 2
- 238000009827 uniform distribution Methods 0.000 claims description 2
- 238000005530 etching Methods 0.000 claims 2
- 230000011218 segmentation Effects 0.000 abstract description 12
- 230000000694 effects Effects 0.000 abstract description 8
- 238000004364 calculation method Methods 0.000 abstract description 5
- 238000007689 inspection Methods 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 5
- 230000000877 morphologic effect Effects 0.000 description 4
- 230000002146 bilateral effect Effects 0.000 description 3
- 238000003709 image segmentation Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 230000005764 inhibitory process Effects 0.000 description 2
- 150000003839 salts Chemical class 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004040 coloring Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000036039 immunity Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 235000012434 pretzels Nutrition 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of visual inspection, and provides a visual inspection method for binocular identification of an intelligent manufacturing production line. Firstly, carrying out gray-scale treatment on an RGB image acquired by binocular identification detection equipment, and preprocessing the image through image filtering and histogram equalization; and adjusting the brightness and contrast of the image by Gamma correction. And then, segmenting the unordered stacked parts by using a Canny edge detection and segmentation algorithm and evaluating the segmentation effect. In order for the Canny operator to generate more continuous edge segments, broken edge segments can be connected in a straight line fitting manner. And then, segmenting the ROI region of the part by adopting a Canny operator gradient method, and finally, carrying out three-dimensional reconstruction, calculation and pose acquisition. The invention can enhance the robustness of the equipment and improve the accuracy of visual detection.
Description
Technical Field
The invention relates to the technical field of visual detection, in particular to a visual detection method for binocular identification of an intelligent manufacturing production line.
Background
Under the industrial background environment of manufacturing modeling enterprises, stacking parts randomly placed in a material frame or a fixed area have the phenomena of stacking, tilting, shielding, uneven illumination and the like, and the recognition and tracking requirements of various targets are difficult to deal with.
For example, the present chinese patent application publication No. CN113019988A discloses a visual inspection apparatus, which relates to a visual inspection apparatus, and includes a frame, a loading device mounted on the frame, an inspection platform for receiving a workpiece to be inspected conveyed from the loading device, an inspection device for visually inspecting the workpiece to be inspected on the inspection platform, a blanking device mounted on the frame for blanking a qualified workpiece and a unqualified workpiece after the inspection device detects the workpiece, and an adjusting device mounted on the frame for mounting the inspection device and adjusting the relative position of the inspection device and the workpiece to be inspected. The visual detection equipment transmits the workpiece to be detected to the detection platform through the feeding device, the workpiece to be detected on the detection platform is visually detected through the detection device, the relative position of the detection device and the workpiece is adjusted through the adjusting device, and the qualified workpiece and the unqualified workpiece which are detected are subjected to differential blanking through the blanking device, so that the automatic detection of the workpiece can be realized, the detection efficiency can be improved, and the labor cost can be saved. However, the robustness of the detection device is not high, the detection range is not comprehensive, and the accuracy of the detection result is not enough.
Disclosure of Invention
The invention aims to provide a visual detection method for binocular identification of an intelligent manufacturing production line, which achieves the effects of improving the robustness of detection equipment, expanding the detection area range and improving the accuracy of detection results by acquiring a plurality of images through analysis and obtaining three-dimensional information of a target object based on parallax.
In order to solve the technical problems, the invention provides the following technical scheme:
a method for visual inspection of binocular identification of an intelligent manufacturing line, comprising the steps of:
s1: carrying out graying treatment on the RGB image acquired by the binocular identification detection equipment by using a weighted average method to obtain a gray scale image;
s2: image preprocessing is carried out on the gray level image through image filtering and histogram equalization;
s3: correcting the gray level image obtained by pretreatment by Gamma;
s4: performing edge detection on the corrected gray level image by using a Canny operator;
s41: connecting broken parts in the gray scale map edges generated by the Canny operator by adopting a straight line fitting connection technology;
s5: selecting an ROI (region of interest) in the gray level diagram after straight line fitting, and dividing the ROI of the gray level diagram by adopting a Canny operator gradient method;
s6: and (3) carrying out edge detection on the segmented ROI by adopting a Canny operator gradient method, fitting the segmented ROI to obtain the outer circle edge contour of the part, converting the segmented ROI into a three-dimensional coordinate system, and carrying out space circle fitting on the outer circle edge contour converted into the three-dimensional coordinate system, thereby obtaining the space pose of the target part.
As a preferred solution of the present invention for visual inspection of binocular identification of an intelligent manufacturing line, wherein: the step of graying the RGB image collected by the binocular identification detection equipment by using a weighted average method comprises the following steps: r, G, B is given different weights according to the importance of R, G, B and applies a weighted average as shown in the formula:
RGB＝ωR+υG+μB
ω, v, μ represent R, G, B weights, respectively, where ω+v+μ=1; where ω=0.29, v=0.59, μ=0.12; and adding the calculated gray values of the R, G, B channels by using a formula to construct a gray map.
As a preferred solution of the present invention for visual inspection of binocular identification of an intelligent manufacturing line, wherein:
s21: filtering the gray scale map by using a filter of a hybrid filtering algorithm in image filtering;
s22: traversing each pixel in the gray level graph, and counting the occurrence frequency of gray level of each pixel to form a gray level histogram;
s23: carrying out cumulative summation on the gray histograms to obtain a cumulative distribution function CDF;
s24: performing pixel mapping according to the accumulated distribution function CDF to generate an equalized image; from the image after the equalization,
and obtaining the image gray values with more uniform distribution.
As a preferred solution of the present invention for visual inspection of binocular identification of an intelligent manufacturing line, wherein:
and correcting the gray level image obtained by the pretreatment by using Gamma, wherein the mathematical expression of the Gamma for correcting is as follows:
F＝cf(i，j) γ
wherein: f represents the gray value of the pixel point (i, j) of the output image after Gamma correction; c represents a conversion scaling factor; gamma represents Gamma coefficient; when gamma is less than 1, the contrast of the region with lower gray value is high, and the contrast of the region with high gray value is low; when gamma > 1, the contrast ratio of the region with lower gray value is low, and the contrast ratio of the region with higher gray value is high.
As a preferred solution of the present invention for visual inspection of binocular identification of an intelligent manufacturing line, wherein:
a1: smoothing the corrected gray level diagram by adopting a Gaussian filter;
a2: calculating the gradient amplitude and gradient direction of the corrected graph by using Sobel and Prewitt gradient operators;
a3: performing non-maximum suppression in the gradient direction, retaining the pixel with the maximum edge intensity;
a4: and setting a high threshold value and a low threshold value, wherein the high threshold value is used for distinguishing obvious strong edges in candidate edge points from other pixels, the low threshold value is used for detecting weaker edges, the high threshold value marks strong edge pixels, the low threshold value marks weak edge pixels, and the communication between the weak edge pixels and the strong edge pixels is checked through threshold processing.
As a preferred solution of the present invention for visual inspection of binocular identification of an intelligent manufacturing line, wherein:
the edge generated by the Canny operator comprises discontinuous broken edge segments, the broken edge segments are connected by adopting a linear fitting connection technology to obtain continuous edge segments, and a RANSAC algorithm is adopted to obtain the best linear fitting result; the best line fitting using the RANSAC algorithm is performed as follows:
b1: randomly selecting a subset of samples from all point sets of the gray scale after edge detection as an inner point set, wherein the subset is a set of the gray scale
The included sample points are used for determining a model;
b2: fitting a model by using the selected sample subset, and fitting the model by using a least square method; presetting a threshold value, calculating the distance from unselected sample points to a fitting model, dividing the sample points with the distance smaller than the threshold value into inner points, and determining the distance larger than the inner points
The sample points of the threshold value are divided into outer points or abnormal points;
b3: if the number of the current internal points is larger than a preset threshold value and the quality of the fitting model meets a preset condition, terminating the algorithm, and returning to the best fitting model; if the number of the current internal points is smaller than or equal to a threshold value or the quality of the fitting model does not meet the requirement, randomly selecting a group of sample subsets again, returning to the step B2, and carrying out a new round of iteration;
after iteration, the model with the largest number of interior points is selected as the best fit model.
As a preferred solution of the present invention for visual inspection of binocular identification of an intelligent manufacturing line, wherein:
in the gray level diagram after the selected straight line fittingThe ROI region is segmented by a Canny operator gradient method, and specifically comprises the following steps: after the gray level image is segmented, the segmented region is selected and marked by adopting a hit or miss conversion method, and the segmented gray level image takes any square or round region as a structural element, and the marking method specifically comprises the following steps: when structural element A is hit or miss by structural element B, it is symbolized byThe hit and miss transformation formula is expressed as:
wherein: b1 and B2 respectively represent internal structural elements and external structural elements of the detected image; a is that c Representing a complement of gray map structural elements a;
and taking the segmented target image, marking the identified parts as 1, marking the parts except the parts in the target image as 0, and marking the parts which are identified to be segmented as red locally in the marking process.
As a preferred solution of the present invention for visual inspection of binocular identification of an intelligent manufacturing line, wherein:
performing edge detection on the ROI area set after the binocular identification and segmentation, fitting out the outer circle edge contour of the part, converting the fitted out outer circle edge contour of the part into a three-dimensional space, and performing space circle fitting to obtain the space pose of the target part, wherein the method specifically comprises the following steps: a three-dimensional coordinate system is established by taking the position of a camera as the origin of coordinates, and a space point P is acquired after the camera shoots a picture w (X, Y, Z) the pixel coordinate of the left view is P L (U L ，V L ) After stereo matching calculation, it can be known that the expected pixel coordinate corresponding to the right view angle is P R (U R ，V R ) Wherein M is used for a projection matrix of three-dimensional coordinates of a real space point of left view angle pixel coordinates and image pixel coordinates of corresponding image points L A representation; right-view pixelM for projection matrix of three-dimensional coordinates of coordinate actual space point and image pixel coordinates of corresponding image point R A representation; one point P of space W Coordinates in the left and right camera coordinate systems respectively (x CL ,y CL ,z CL ) And (x) CR ,y CR ,z CR ) The method comprises the steps of carrying out a first treatment on the surface of the Setting the parallax of the left and right viewing angles as d;
setting P L 、P R 、P W The modulus of (a) is respectivelyThe camera imaging principle refers to a camera
The basic principle of converting a three-dimensional scene into a two-dimensional image is known from the principle of camera imaging:
from the camera model in combination with the binocular metrology model:
the baseline B of the camera is known by camera calibration, from which:
the mutual solution of the expressions can be obtained:
the analysis of the above method shows that if the left-right parallax of a point in space and the internal and external parameters calibrated by a camera are known, the three-dimensional coordinate of the point can be calculated; and then, performing space circle fitting on the ROI region to calculate the circle center three-dimensional coordinate and normal vector, thereby obtaining the space pose information of the part.
Compared with the prior art, the invention has the beneficial effects that: the binocular identification visual detection method for the intelligent manufacturing production line ensures that workpieces are orderly placed, thereby effectively reducing labor cost and improving production efficiency; the problem that defects or defects on the surface of a product cannot be directly judged by naked eyes is solved by more comprehensively observing the parts at multiple angles, the robustness of equipment is enhanced, and the accuracy of visual detection is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Wherein:
FIG. 1 is a technical roadmap of a method for visual inspection of binocular identification of an intelligent manufacturing line according to the present invention.
Fig. 2 is a gray scale artwork histogram of a method for visual inspection of binocular identification of an intelligent manufacturing line according to the present invention.
Fig. 3 is a gray map equalization histogram of a method for visual inspection of binocular identification of an intelligent manufacturing line according to the present invention.
Detailed Description
It should be noted that: the technical concept of the invention is that the principle of binocular measurement is based on parallax, and the method is a method for acquiring three-dimensional information of a target object through analysis by collecting a plurality of images.
In order that the described objects, features and advantages of the invention will be readily understood, a more particular description of the invention will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways other than those described herein, and persons skilled in the art will readily appreciate that the present invention is not limited to the specific embodiments disclosed below.
Example 1
Referring to fig. 1, a technical roadmap is provided for describing a method for visual inspection of binocular identification of an intelligent manufacturing line, as shown in fig. 1, a method for visual inspection of binocular identification of an intelligent manufacturing line includes the steps of:
s1: and carrying out graying treatment on the RGB image acquired by the binocular identification detection equipment by using a weighted average method to obtain a gray scale image. It should be noted that: the step of graying the RGB image collected by the binocular identification detection equipment by using a weighted average method comprises the following steps:
different weights are given to R, G, B according to importance, and the values thereof are weighted and averaged, rgb=ωr+νg+μb, ω, ν, μ representing R, G, B weights, respectively, where ω+ν+μ=1. The gray weights are as follows:
| channel | ω | ν | μ |
| Gray scale weight | 0.29 | 0.59 | 0.12 |
Multiplying the pixel value of the RGB channel of each pixel by the corresponding weight, and then adding the results of the R, G and B channels to obtain a weighted average value as the gray value of the pixel;
a new gray scale map is constructed using the calculated gray scale values. For better acquisition of image information, accurate recognition, contrast enhancement of the image is required. Therefore, histogram equalization is used to enhance contrast, thereby displaying foreground versus background differences. Fig. 2 is a coil skeleton histogram before equalization, fig. 3 is a coil skeleton histogram after equalization, and gray value changes are reflected in the histogram, so that it can be obviously seen that after equalization, the contrast of a target part increases, and the gray value distribution in an image is more uniform.
S2: the gray level map is subjected to image preprocessing through image filtering and histogram equalization, and the following needs to be described:
firstly, simulating actual working conditions to acquire images, respectively adding Gaussian noise and spiced salt noise to the acquired images, and sequentially discussing three single filtering algorithms. And the three filtering algorithms are used for respectively denoising the noise added picture, and then the noise added picture is compared with the designed mixed filtering algorithm, so that the effectiveness of the mixed filtering algorithm can be verified. The image filtering process is as follows:
wherein: g (x, y) represents the gray value at the filtered pixel point (x, y); a. b represents a length of a rectangular area established centering on (x, y); k (s, t) represents a filter; f (x-s, y-t) represents the gray value of the pixel point in the window;
the three filtering algorithms are respectively:
mean filtering is also linear filtering, and mainly uses a model method to implement neighborhood operation. That is, given a model covering surrounding neighboring pixels, the average value of all pixels is then used to replace the original pixel value;
gaussian filtering is a linear smoothing filter that has a significant effect on removing gaussian noise, essentially the process of weighting and averaging the entire image. More specifically, a model (or convolution mask) is used to search each pixel in the digitized image and the weighted average gray value of adjacent pixels is determined by the model instead of the pixel value at the center of the model.
The median filtering is a nonlinear filtering whose basic idea is to use the median value of the pixels in a pixel-specific neighborhood instead of the gray value of the center pixel to obtain a smoothing effect. The effect of removing the salt noise is very good, and the edge information can be effectively reserved while the random noise is removed.
The three filtering algorithms analyzed above were followed by adding gaussian noise and pretzel noise at a concentration of 0.02 to the histogram equalized picture. And then, the three single filtering algorithms analyzed above are used for suppressing the two types of noise, the result graphs are compared, and the experimental results show that: the noise suppression effect of the hybrid filter is clear and obvious, and the edge information is clear.
S3: gamma correction is carried out on the preprocessed gray level diagram, and the following description is needed: in the image acquisition process, the robot system can be influenced by external environment and working conditions (tilting, stacking and shielding) of the robot system, so that the surface illumination of the part is uneven, and the subsequent feature extraction and part identification are influenced. Thus, the image is subjected to a certain processing by Gamma correction. Wherein the mathematical expression of Gamma correction is:
F＝f(i，j) γ
wherein: f represents the gray value of the pixel point of the output image; gamma represents Gamma coefficient;
when Gamma is less than 1, the contrast ratio of the area with lower gray value is stronger, otherwise, the contrast ratio of the area with higher gray value is lower, and the whole image after Gamma correction is lightened; when Gamma is larger than 1, the contrast of the region with lower gray value is lower, otherwise, the contrast of the region with higher gray value is stronger, and the whole image darkens after Gamma correction.
S4: the gray level image after correction processing is subjected to edge detection by using a Canny operator, and the unordered stacked part image is segmented by using a morphological processing color marking Fu Fenshui ridge segmentation algorithm, and the following needs to be described: compared with Canny's algorithm, sobel and Prewitt operators do not fully utilize the direction of edge gradients, so that the generated binary image can be processed by only a single threshold. The Canny algorithm is established on a gradient operator, and a calculation strategy is introduced into the algorithm, wherein the strategy can acquire a single pixel outline with excellent noise resistance and higher positioning precision, so that the efficiency of edge detection is improved.
Performing Gaussian denoising and anti-noise processing on the preprocessed image: gaussian denoising: the preprocessed image is loaded using an image processing library such as OpenCV. The gaussian filter function is invoked, with the size of the incoming image and the filter as parameters. The filter size determines the degree of smoothing and typically selects the size of the odd values, e.g., 3x3, 5x5, etc. The standard deviation parameters of the gaussian function can be adjusted to control the filtering effect as required. A larger standard deviation may blur the image and a smaller standard deviation may not be effective in reducing noise, so a suitable standard deviation is selected. After performing the gaussian filtering, a denoised image is obtained. Anti-noise treatment: one of the common anti-noise processing methods is bilateral filtering. The gaussian denoised image is passed in using bilateral filtering functions provided in an image processing library, as well as other parameters such as filter size, color space standard deviation and gray space standard deviation. The filter size determines the range of the local environment, the color space standard deviation controls the pixel value similarity weight, and the gray space standard deviation controls the pixel position similarity weight. Adjusting these parameters balances the anti-noise effect and preserves the level of image detail. And after bilateral filtering is carried out, a final image subjected to Gaussian denoising and noise immunity processing is obtained.
Calculating the direction and magnitude of the gradient from the difference of the first partial derivatives for oneFor a two-dimensional image, the direction of the gradient is along the direction of maximum derivative, while the magnitude of the gradient represents the intensity of the derivative change. The specific calculation method is as follows: first, first order partial derivatives of the image in the horizontal direction and first order partial derivatives in the vertical direction are calculated. These derivatives can be calculated using a common edge detection operator such as the Sobel operator. From the derivative values in the horizontal and vertical directions, the direction and magnitude of the gradient can be calculated. Gradient direction is calculated byWhere dx is the derivative in the x direction and dy is the derivative in the y direction. This angle represents the direction of the gradient vector and can be used to indicate the steepest direction. The gradient magnitude is calculated using the formula: magnitiude = sqrt (dx) 2 +dy 2 ) The magnitude of the gradient is calculated, where dx and dy are derivative values in the x-direction and y-direction, respectively. Finally, the direction and the magnitude of the gradient are calculated.
Non-maximum value suppression processing is performed. Non-maximum suppression is a commonly used edge refinement algorithm for preserving pixels of local maximum gradient values in the edge detection result, suppressing unimportant edge responses. The following is a basic step of performing non-maximum suppression processing: firstly, gradient computing operation is carried out, and gradient amplitude and angle of each pixel point in the image are obtained. Then, each pixel is compared with two adjacent pixels by taking the gradient direction as a reference. If the gradient amplitude of the current pixel is not the local maximum value, the current pixel is restrained, the amplitude is set to be 0, otherwise, the gradient amplitude of the pixel is reserved. In the comparison process, two adjacent pixel points are mapped onto a straight line according to the gradient direction. If the gradient magnitude of the current pixel is smaller than the gradient magnitude of any one of the adjacent pixel points, namely, is not the local maximum value of the gradient, the gradient magnitude of the current pixel is suppressed to 0. If the gradient amplitude of the current pixel is greater than or equal to the gradient amplitude of two adjacent pixel points, namely the local maximum value of the gradient, the gradient amplitude of the current pixel is reserved. And finally traversing all pixel points, and obtaining a processed edge detection result after finishing non-maximum value inhibition processing.
Edges are detected and restored using a double thresholding method. Performing non-maximum suppression processing is a commonly used post-processing method for edge detection to extract the edges of thin lines in an image. The following basic steps are as follows: firstly, calculating the gradient direction and the gradient size of each pixel point according to the gradient information of the image. The gradient of the image can be calculated by means of an edge detection operator, such as a Sobel operator. And secondly, carrying out non-maximum value suppression processing on each pixel point. Specifically, it is determined at each pixel point whether it is a local maximum point along the gradient direction. Comparing the gradient magnitude of the pixel point with the gradient magnitude of the pixel point in two adjacent directions, and if the gradient magnitude is the maximum value in one direction, reserving the gradient magnitude; otherwise, it is suppressed. And performing edge detection by using a double-threshold method, and classifying the gradient images subjected to non-maximum value inhibition treatment according to two set thresholds. The higher threshold is generally referred to as the strong edge threshold and the lower threshold is referred to as the weak edge threshold. Pixels exceeding the strong edge threshold are considered strong edges, remain; pixels below the weak edge threshold are considered non-edge, suppressed; pixels that lie between the two thresholds are considered weak edges and are determined to remain based on whether they are in communication with strong edges. And finally, carrying out edge connection processing on the weak edge part. A weak edge pixel is classified as a strong edge if it is directly or indirectly connected to at least one strong edge pixel, and is otherwise classified as a non-edge. This allows the broken edge line to be recovered by connecting the weak edges.
The edge detection result based on the Canny operator can show that the edges of each part can be completely detected under the working conditions of scattered parts, no stacking and no connection, the stacking exists, the shielding exists, all the edges can be better detected under the working condition of inclination, but the shielding part can not be detected, so that the stacked parts can be regarded as a whole, the edges are incomplete, and therefore the stacked parts are required to be segmented; the parts are randomly and randomly stacked, shielding and connection exist, if a simpler segmentation method is adopted, the segmentation requirement is difficult to meet, and therefore a morphological processing color mark Fu Fenshui ridge segmentation algorithm is adopted; the watershed image segmentation specific method comprises the following steps: consider three types of points: the local minimum point-influence range is called a water collecting basin, other position points of the basin, edge points of the basin-junctions of the basin and other basins; finding all different catchment basin and watershed boundaries; the areas consisting of the water collecting basin and the watershed are the targets to be segmented; the detected object is processed by morphology and then color markers are added, so that an accurate part edge profile graph is obtained, and the aim of improving segmentation is fulfilled.
S41: the broken parts in the edges generated by the Canny operator are connected by adopting a straight line fitting connection technology, and the following needs to be described: the edges generated by the Canny operator may contain discontinuous broken parts, in order to obtain more continuous edge line segments, broken edge segments can be connected by adopting straight line fitting, the RANSAC algorithm is an iterative method for estimating parameters, a model is fitted by randomly selecting sample points, and the influence of abnormal points is removed in an iterative mode, so that the best fitting result is obtained; the general procedure for best line fitting using the RANSAC algorithm is as follows: first, randomly selecting a subset of samples from the dataset as an interior point set, the subset should contain a sufficient number of sample points to determine a model; next, a model is fitted using the selected subset of samples. Model fitting can be performed by using least squares and other methods; again, for other unselected sample points, their distances to the fitting model are calculated. Sample points with a distance less than a certain threshold are classified as inner points, and sample points with a distance greater than the threshold are classified as outer points or outliers. Finally, if the number of the current internal points exceeds a preset threshold value and the quality of the fitting model meets a preset condition, terminating the algorithm, and returning to the optimal fitting model; if the number of the current internal points does not reach the threshold value or the quality of the fitting model does not meet the requirement, randomly selecting a group of sample subsets again, and carrying out a new iteration; after a number of iterations, the model with the largest number of interior points is selected as the best fit model.
S5: selecting an ROI (region of interest) in the gray level diagram after straight line fitting, and dividing the ROI of the gray level diagram by adopting a Canny operator gradient method, wherein the following steps are to be described: the unordered stacked part image segmentation can separate part areas with different angles and texture characteristics in the image, but to obtain information, the segmented areas are required to be subjected to ROI area selection, image information analysis can be performed, three-dimensional reconstruction is performed to obtain the space pose of the part, and accurate grabbing is facilitated.
The image segmentation can separate the part areas with different angles and texture features in the image, but to obtain information, the segmented areas need to be subjected to ROI area selection and marked by adopting a hit or miss transformation method, and the marking specific method comprises the following steps:
when the divided gray level graph takes any square or round area as a structural element, and the structural element A is hit or hit-miss by the structural element B, the structural element A is represented by a symbol, and then the hit-miss conversion formula is as follows:
wherein: b1 and B2 are structural elements respectively representing the interior and the exterior of the detected image; a is that c Representing a complement of structural element a;
taking the segmented target image, marking the identified part as 1, the black part and the background which is not distinguished as 0, and in the marking process, in order to highlight the detection effect, locally coloring the identified segmented part. The specific segmentation flow is as follows: firstly dividing the acquired image into a left view angle and a right view angle, taking the left view angle as a reference, carrying out pretreatment such as filtering, histogram equalization, and the like, secondly adopting a Canny operator to carry out gradient treatment, then using watershed conversion of morphological treatment, and finally adding a color mark into an original image, thus obtaining the divided image. In morphological processing, square or circular structural elements can be applied, parameters can be adjusted, and the segmentation result can be optimized, so that the adjacent part contours form obvious watershed boundaries, and part images in a stacked state can be accurately identified and segmented.
S6: performing edge detection on the ROI region set after binocular identification segmentation, fitting the outline of the outer circle edge of the part, and turningAnd (3) transforming the three-dimensional space into the three-dimensional space, and performing space circle fitting on the outline of the outer circle edge transformed into the three-dimensional coordinate system, so that the space pose of the target part is obtained. It should be noted that: the method comprises the steps of establishing a three-dimensional coordinate system by taking a camera position as a coordinate origin, performing edge detection on a ROI region set after binocular identification and segmentation, fitting an outer circle edge contour of a part, converting the outer circle edge contour of the part into a three-dimensional space, and performing space circle fitting to obtain a space pose of a target part, wherein the method specifically comprises the following steps: acquiring a space point P after taking a picture by a camera w (X, Y, Z) the pixel coordinate of the left view is P L (U L ，V L ) After stereo matching calculation, it can be known that the expected pixel coordinate corresponding to the right view angle is P R (U R ，V R ) Wherein M is used for a projection matrix of three-dimensional coordinates of a real space point of left view angle pixel coordinates and image pixel coordinates of corresponding image points L A representation; m for projection matrix of actual space point three-dimensional coordinates of right view angle pixel coordinates and image pixel coordinates of corresponding image point R A representation; one point P of space W Coordinates in the left and right camera coordinate systems respectively (x CL ,y CL ,z CL ) And (x) CR ,y CR ,z CR ) The method comprises the steps of carrying out a first treatment on the surface of the Setting the parallax of the left and right viewing angles as d;
setting P L 、P R 、P W The modulus of (a) is respectivelyThe imaging principle of a camera is as follows:
from the camera model in combination with the binocular metrology model:
the baseline B of the camera is known by camera calibration, from which:
the mutual solution of the expressions can be obtained:
the analysis of the above method shows that if the left-right parallax of a point in space and the internal and external parameters calibrated by a camera are known, the three-dimensional coordinate of the point can be calculated; and then, performing space circle fitting on the ROI region to calculate the circle center three-dimensional coordinate and normal vector, thereby obtaining the space pose information of the part.
It should be noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention may be modified or substituted without departing from the spirit and scope of the technical solution of the present invention, which is intended to be covered in the scope of the claims of the present invention.
Claims (8)
1. A method for visual inspection of binocular identification of intelligent manufacturing production lines, characterized by: the method comprises the following steps:
s1: carrying out graying treatment on the RGB image acquired by the binocular identification detection equipment by using a weighted average method to obtain a gray scale image;
s2: image preprocessing is carried out on the gray level image through image filtering and histogram equalization;
s3: correcting the gray level image obtained by pretreatment by Gamma;
s4: performing edge detection on the corrected gray level image by using a Canny operator;
s41: connecting broken parts in the gray scale map edges generated by the Canny operator by adopting a straight line fitting connection technology;
s5: selecting an ROI (region of interest) in the gray level diagram after straight line fitting, and dividing the ROI of the gray level diagram by adopting a Canny operator gradient method;
s6: and (3) carrying out edge detection on the segmented ROI by adopting a Canny operator gradient method, fitting the segmented ROI to obtain the outer circle edge contour of the part, converting the segmented ROI into a three-dimensional coordinate system, and carrying out space circle fitting on the outer circle edge contour converted into the three-dimensional coordinate system, thereby obtaining the space pose of the target part.
2. A method for visual inspection of binocular identification of intelligent manufacturing production lines according to claim 1, characterized by: the step of carrying out graying treatment on the RGB image acquired by the binocular identification detection equipment by using a weighted average method comprises the following steps:
r, G, B is given different weights according to the importance of R, G, B, and a weighted average formula is used to construct a gray scale map, wherein the weighted average formula is as follows:
RGB＝ωR+υG+μB
wherein ω, v, μ represent R, G, B weights, respectively, wherein ω+v+μ=1; where ω=0.29, v=0.59, μ=0.12 is assigned.
3. A method for visual inspection of binocular identification of intelligent manufacturing production lines according to claim 1, characterized by: the specific steps of preprocessing the gray level map through image filtering and histogram equalization include:
s21: filtering the gray scale map by using a filter of a hybrid filtering algorithm in image filtering;
s22: traversing each pixel in the gray level graph, and counting the occurrence frequency of gray level of each pixel to form a gray level histogram;
s23: carrying out cumulative summation on the gray histograms to obtain a cumulative distribution function CDF;
s24: performing pixel mapping according to the accumulated distribution function CDF to generate an equalized image; and obtaining the image gray values with more uniform distribution from the equalized image.
4. A method for visual inspection of binocular identification of intelligent manufacturing production lines according to claim 3, characterized in that: and correcting the gray level image obtained by the pretreatment by using Gamma, wherein the mathematical expression of the Gamma for correcting is as follows:
F＝cf(i，j) γ
wherein: f represents the gray value of the pixel point (i, j) of the output image after Gamma correction; c represents a conversion scaling factor; gamma represents Gamma coefficient; when gamma is less than 1, the contrast of the region with lower gray value is high, and the contrast of the region with high gray value is low; when gamma > 1, the contrast ratio of the region with lower gray value is low, and the contrast ratio of the region with higher gray value is high.
5. A method for visual inspection of binocular identification of intelligent manufacturing production lines according to claim 1, characterized by: the method for carrying out edge detection on the gray level image after correction processing by using the Canny operator comprises the following specific steps:
a1: smoothing the corrected gray level diagram by adopting a Gaussian filter;
a2: calculating the gradient amplitude and gradient direction of the corrected graph by using Sobel and Prewitt gradient operators;
a3: performing non-maximum suppression in the gradient direction, retaining the pixel with the maximum edge intensity;
a4: and setting a high threshold value and a low threshold value, wherein the high threshold value is used for distinguishing obvious strong edges in candidate edge points from other pixels, the low threshold value is used for detecting weaker edges, the high threshold value marks strong edge pixels, the low threshold value marks weak edge pixels, and the communication between the weak edge pixels and the strong edge pixels is checked through threshold processing.
6. A method for visual inspection of binocular identification of intelligent manufacturing production lines according to claim 5, characterized by: the edge generated by the Canny operator comprises discontinuous broken edge segments, the broken edge segments are connected by adopting a linear fitting connection technology to obtain continuous edge segments, and a RANSAC algorithm is adopted to obtain the best linear fitting result; the best line fitting using the RANSAC algorithm is performed as follows:
b1: randomly selecting a sample subset from all point sets of the gray level graph after edge detection as an inner point set, wherein the sample points contained in the subset are used for determining a model;
b2: fitting a model by using the selected sample subset, and fitting the model by using a least square method; a threshold value is preset, the distance from the unselected sample points to the fitting model is calculated, the sample points with the distance smaller than the threshold value are divided into inner points, and the sample points with the distance larger than the threshold value are divided into outer points or abnormal points;
b3: if the number of the current internal points is larger than a preset threshold value and the quality of the fitting model meets a preset condition, terminating the algorithm, and returning to the best fitting model; if the number of the current internal points is smaller than or equal to a threshold value or the quality of the fitting model does not meet the requirement, randomly selecting a group of sample subsets again, returning to the step B2, and carrying out a new round of iteration;
after iteration, the model with the largest number of interior points is selected as the best fit model.
7. A method for visual inspection of binocular identification of intelligent manufacturing production lines according to claim 1, characterized by: selecting the ROI in the gray level graph after straight line fitting, and dividing the ROI in the gray level graph by adopting a Canny operator gradient method specifically comprises the following steps: after the gray level image is divided, the divided areas are selected and marked by adopting a hit or miss conversion method, any square or round area is taken as a structural element in the divided gray level image,the specific method for marking comprises the following steps: when structural element A is hit or miss by structural element B, it is symbolized byThe hit and miss transformation formula is expressed as:
wherein: b1 and B2 respectively represent internal structural elements and external structural elements of the detected image; a is that c Complement representing gray map structural element ARepresenting a new vector space whose elements consist of linear combinations of elements in linear element a and linear element B; aΘB 1 Representing structural element A and structural element B 1 Etching operation; a is that c ΘB 2 Representing complement of structural element A to structural element B 2 Etching operation;
and taking the segmented target image, marking the identified parts as 1, marking the parts except the parts in the target image as 0, and marking the parts which are identified to be segmented as red locally in the marking process.
8. A method for visual inspection of binocular identification of intelligent manufacturing production lines according to claim 1, characterized by: the method comprises the steps of dividing an ROI region by a Canny operator gradient method, carrying out edge detection, fitting an outer circle edge contour of a part, converting the outer circle edge contour into a three-dimensional coordinate system, and carrying out space circle fitting on the outer circle edge contour converted into the three-dimensional coordinate system, so as to obtain a space pose of a target part, wherein the method specifically comprises the following steps: a three-dimensional coordinate system is established by taking the position of a camera as the origin of coordinates, and a space point P is acquired after the camera shoots a part w (X, Y, Z), the pixel coordinate of the left view shot is P L (U L ，V L ) Obtaining expected pixel coordinates corresponding to the right view angle through stereo matching calculationIs P R (U R ，V R ) Wherein M is used for a projection matrix of left view angle pixel coordinates and pixel coordinates L A representation; m for right view angle pixel coordinates and projection matrix of pixel coordinates R A representation; spatial point P W The coordinates in the left and right camera coordinate systems are (x) CL ,y CL ,z CL ) And (x) CR ,y CR ,z CR ) The method comprises the steps of carrying out a first treatment on the surface of the Setting a base line of a camera obtained through camera calibration as B and setting parallax of left and right viewing angles as d;
setting P L 、P R 、P W The modulus of (a) is respectivelyThe camera imaging principle refers to the basic principle that a camera converts a three-dimensional scene into a two-dimensional image, and is known by the camera principle:
from the camera model in combination with the binocular metrology model:
the baseline B of the camera is known by camera calibration, from which:
the mutual solution of the expressions can be obtained:
analyzing the above, and calculating the three-dimensional coordinates of a point through the left-right parallax of the point and the internal and external parameters calibrated by the camera; and then, carrying out space circle fitting on the ROI area to calculate the three-dimensional coordinates and normal vector of the circle center, thereby obtaining the space pose information of the part.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410041698.5A CN117764983A (en) | 2024-01-11 | 2024-01-11 | Visual detection method for binocular identification of intelligent manufacturing production line |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410041698.5A CN117764983A (en) | 2024-01-11 | 2024-01-11 | Visual detection method for binocular identification of intelligent manufacturing production line |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117764983A true CN117764983A (en) | 2024-03-26 |
Family
ID=90310836
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410041698.5A Pending CN117764983A (en) | 2024-01-11 | 2024-01-11 | Visual detection method for binocular identification of intelligent manufacturing production line |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117764983A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118015005A (en) * | 2024-04-10 | 2024-05-10 | 合肥工业大学 | Machine vision-based whiskering detection method and portable detection device |
-
2024
- 2024-01-11 CN CN202410041698.5A patent/CN117764983A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118015005A (en) * | 2024-04-10 | 2024-05-10 | 合肥工业大学 | Machine vision-based whiskering detection method and portable detection device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108921176B (en) | Pointer instrument positioning and identifying method based on machine vision | |
| CN109870461B (en) | Electronic components quality detection system | |
| Ahmed | Comparative study among Sobel, Prewitt and Canny edge detection operators used in image processing | |
| CN103984961B (en) | A kind of image detecting method for being used to detect underbody foreign matter | |
| CN108629775B (en) | Thermal state high-speed wire rod surface image processing method | |
| CN108416789A (en) | Method for detecting image edge and system | |
| CN102156996B (en) | Image edge detection method | |
| CN112819772B (en) | High-precision rapid pattern detection and recognition method | |
| EP2085928B1 (en) | Detection of blobs in images | |
| CN114372983B (en) | Shielding box coating quality detection method and system based on image processing | |
| CN108376403B (en) | Grid colony image segmentation method based on Hough circle transformation | |
| CN113034452B (en) | Weldment contour detection method | |
| CN111126174A (en) | Visual detection method for robot to grab parts | |
| CN109815822B (en) | Patrol diagram part target identification method based on generalized Hough transformation | |
| CN110189290A (en) | Metal surface fine defects detection method and device based on deep learning | |
| CN117764983A (en) | Visual detection method for binocular identification of intelligent manufacturing production line | |
| CN112085700B (en) | Automatic extraction method, system and medium for weld joint region in X-ray image | |
| CN114549441A (en) | Sucker defect detection method based on image processing | |
| CN112017223A (en) | Heterologous image registration method based on improved SIFT-Delaunay | |
| CN112669286A (en) | Infrared thermal image-based method for identifying defects and evaluating damage degree of external thermal insulation system of external wall | |
| CN111242888A (en) | Image processing method and system based on machine vision | |
| Alazzawi | Edge detection-application of (first and second) order derivative in image processing: communication | |
| CN108109154A (en) | A kind of new positioning of workpiece and data capture method | |
| CN115108466A (en) | Intelligent positioning method for container spreader | |
| CN117853510A (en) | Canny edge detection method based on bilateral filtering and self-adaptive threshold |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |