CN116188763A - Method for measuring carton identification positioning and placement angle based on YOLOv5 - Google Patents
Method for measuring carton identification positioning and placement angle based on YOLOv5 Download PDFInfo
- Publication number
- CN116188763A CN116188763A CN202211695904.1A CN202211695904A CN116188763A CN 116188763 A CN116188763 A CN 116188763A CN 202211695904 A CN202211695904 A CN 202211695904A CN 116188763 A CN116188763 A CN 116188763A
- Authority
- CN
- China
- Prior art keywords
- carton
- cartons
- image
- yolov5
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/26—Measuring arrangements characterised by the use of optical techniques for measuring angles or tapers; for testing the alignment of axes
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/22—Measuring arrangements characterised by the use of optical techniques for measuring depth
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/36—Applying a local operator, i.e. means to operate on image points situated in the vicinity of a given point; Non-linear local filtering operations, e.g. median filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Nonlinear Science (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a measuring method of carton identification positioning and placement angle based on YOLOv5, which comprises the steps of preprocessing a plurality of images to obtain a multi-target training set, training the training set by using a YOLOv5 algorithm, and classifying and identifying cartons by using a trained model; detecting the placement angle of the paper box by combining an OpenCV software library on the basis of a YOLOv5 algorithm; the binocular vision system acquires detection information of the cartons to be grabbed, then calculates the space depth distance of the cartons through an algorithm, transmits calculation data to the mechanical arm control assembly to complete grabbing, and the mechanical arm adjusts the angles of the grabbed cartons according to the acquired angle information and orderly stacks the cartons with the angles adjusted at corresponding positions. The measuring method for the recognition, positioning and placement angles of the paper boxes based on the YOLOv5 is simple in structure and high in detection precision, and can obtain a better target detection effect, so that the real-time grabbing of the mechanical arm is realized, and the measuring method can be applied to actual paper box grabbing and stacking scenes.
Description
Technical Field
The invention belongs to the technical field of target detection technology and mechanical intelligent control, and particularly relates to a measuring method for recognizing, positioning and placing angles of cartons based on YOLOv 5.
Background
In the production process of the traditional production line, specific cartons are generally sorted out by a manual sorting method, but manual sorting is time-consuming and labor-consuming and has low efficiency. The existing mechanical sorting device is complex in structure, more in operation steps and lower in grabbing reliability, and therefore working efficiency cannot be improved slowly.
Disclosure of Invention
The invention aims to provide a measuring method for recognizing, positioning and placing angles of cartons based on YOLOv5, which has the advantages of simple structure and high detection precision, and can obtain a better target detection effect, thereby realizing real-time grabbing of a mechanical arm and being applied to actual carton grabbing and stacking scenes.
In order to achieve the above purpose, the invention provides a measuring method for recognizing, positioning and placing angles of cartons based on YOLOv5, which comprises the following steps:
s1, shooting a paper box picture, transmitting the paper box picture into computer equipment, marking and amplifying data, and dividing a data set;
s2, training a training set by using a YOLOv5 algorithm, and classifying and identifying by using a trained model to obtain an optimal detection model;
s3, shooting a carton picture in an actual application scene by using a binocular vision system, detecting the carton picture by using a detection model to obtain carton type and predicted frame coordinate information, and calculating a center point coordinate of the carton;
s4, combining an OpenCV software library on the basis of a YOLOv5 algorithm, further processing the obtained image set of the target detection information, and detecting the placement angle of the carton; openCV (Open Source Computer Vision Library) is a computer vision and machine learning software library. OpenCV is providing a general infrastructure for computer vision applications, with over 2500 optimization algorithms, covering classical and most advanced computer vision and machine learning algorithms. These algorithms may be used to detect and identify faces, identify objects, differentiate actions of people in video, track camera movements, track moving objects, extract 3D models of objects, generate 3D point clouds from stereo cameras, stitch images together to produce high resolution images of the entire scene, find similar images from an image database, remove red eyes from images taken using flash, track eye movements, identify scenes and build markers to superimpose them with augmented reality, etc.
S5, the binocular vision system acquires detection information of the carton to be finally grabbed, and image information acquired by the left camera and the right camera of the binocular vision system is processed by utilizing a correlation matching algorithm and a triangulation principle to calculate the spatial depth distance of the carton;
s6, transmitting the calculated space depth distance of the cartons to a mechanical arm control assembly in a real-time detection state, and completing real-time grabbing of the corresponding cartons by the mechanical arm control assembly according to the space depth information of the cartons;
s7, the mechanical arm control assembly adjusts the angle of the grasped paper box according to the acquired angle information;
s8, orderly stacking the cartons with the adjusted angles at corresponding placement positions.
Preferably, the step S1 specifically includes:
s11, shooting pictures of various postures of cartons of different sizes in an actual carton stacking scene by using a binocular vision system, and transmitting the pictures into computer equipment;
s12, labeling the carton pictures by using a person to generate a data set;
s13, carrying out data amplification by a method of rotation, translation, overturning and mirroring to obtain an amplified picture data set.
Preferably, the step S3 specifically includes:
s31, identifying and classifying the carton pictures by the optimal model obtained in the step S2 to obtain carton type and predicted frame position information;
s32, calculating the center coordinates (m, n) of the carton according to the obtained predicted frame coordinates, wherein the calculation formula is as follows:
wherein (x, y) is the upper left corner coordinate of the prediction frame, w is the width of the prediction frame, and h is the height of the prediction frame;
s33, when the binocular vision system shoots a carton image in an actual application scene, a cvtColor function in OpenCV is utilized to obtain a corresponding gray level image, and the principle is as follows:
Gray＝R*0.3+G*0.59+B*0.11
wherein RGB (R, G, B) is the color of a certain point;
s34, according to the obtained coordinate information, gray values of the center points of the cartons are obtained on the corresponding gray images, and gray value sorting is carried out; the smaller the gray value, the closer the carton is to the fixed binocular vision system and the mechanical arm. Therefore, the carton with the smallest central gray value is selected, carton category is output, and frame coordinate information and calculated central coordinates are predicted.
Preferably, the step S4 specifically includes:
s41, cutting all pixels except the predicted frame of the selected carton on the gray level image according to the coordinate information of the predicted frame;
s42, carrying out Gaussian filtering on the gray level image of the carton after cutting, carrying out weighted average on the whole image, and obtaining the value of each pixel point by carrying out weighted average on the pixel point and other pixel values in the neighborhood. Most of noise is removed through Gaussian filtering, and noise interference is reduced. The gaussian filter formula is as follows:
wherein (x, y) is the coordinates of the gray scale image, σ is the variance of x;
s43, performing binarization processing on the gray level image by using an OpenCV binarization function threshold, and setting the gray level value of a pixel point on the image to be 0 or 255, so that the whole image shows obvious black-and-white effect. Binarization of the image greatly reduces the amount of data in the image, thereby highlighting the contours of the object. The influence of light on the picture is well removed through an OpenCV binarization function threshold, so that the edges of the paper boxes are more obvious. The binarization formula is as follows:
wherein maxval is the maximum threshold, scr (x, y) is the pixel value corresponding to the original gray image, and thresh is the threshold;
s44, utilizing OpenCV-Scharr edge detection to find out edge lines of the cartons. The purpose of edge detection is to identify points in the digital image where the brightness change is obvious, so that the data size can be greatly reduced, irrelevant information is eliminated, and important structural attributes of the image are reserved.
Wherein, first order Scharr edge detection operator in X direction is:
the first order Scharr edge detection operator in the Y direction is:
s45, performing binarization operation on the image again, and obviously dividing the edge line and surrounding pixel points into black and white;
s46, hough transform is a method of finding straight lines, circles and other simple shapes in an image. The method comprises the steps of searching a straight line segment of a current diagram by using OpenCV-HoughLinesP (Hough transform straight line detection), traversing data obtained through Hough transform, and then drawing a line on an original diagram, wherein the principle of the OpenCV-HoughLinesP is as follows:
r＝xcosθ+ysinθ
where (x, y) is a coordinate point and θ is an angle, each pair of (r, θ) is a straight line passing through the point (x, y). Finding out straight lines passing through all points (x, y) on the polar angle plane of the polar diameter to obtain a sine curve. If the curves obtained by the above operations of two coordinate points of the image have an intersection point, which means that the two coordinate points pass through a straight line, the straight line of the parameter represented by the intersection point in the original image can be obtained.
S47, mainly solving a rectangle containing the minimum area of the point set of the data obtained by Hough transformation by using an OpenCV-minArearect function, solving a deflection angle (placement angle) of the rectangle, namely a deflection angle relative to an x axis in an x-y coordinate system, and performing space positioning on the carton to obtain final target detection information.
Compared with the existing method, the method for measuring the recognition, positioning and placement angles of the paper box based on the YOLOv5 has the following beneficial effects:
(1) An accurate classification, positioning model and algorithm can be constructed.
(2) The method that YOLOv5 and OpenCV are combined is used for target detection and processing, important structural attributes of images are reserved, and space positioning of the carton can be accurately and rapidly achieved, and final target detection information can be obtained.
(3) The carton grabbing and stacking device can meet the requirements of carton grabbing and stacking in an actual application environment, unnecessary operations in the use process of personnel are reduced, and the working efficiency is improved.
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Drawings
FIG. 1 is a flow chart of a method for measuring the recognition, positioning and placement angle of a paper box based on YOLOv 5;
FIG. 2 is a schematic diagram showing a method for measuring the recognition, positioning and placement angle of a paper box based on YOLOv 5;
FIG. 3 is a schematic diagram of calculating the depth distance of the carton space by using the measuring method of the recognition, positioning and placement angle of the carton based on YOLOv 5;
FIG. 4 is a block diagram of a method for measuring the recognition, positioning and placement angle of a carton based on YOLOv 5;
FIG. 5 is an actual scene graph taken by a binocular vision system;
fig. 6 is a result diagram of classification detection of a carton picture;
FIG. 7 is a gray scale image corresponding to a carton picture;
FIG. 8 is a graph of the result of the cropped gray scale image;
FIG. 9 is a graph of the result of Gaussian filtering;
FIG. 10 is a graph showing the result of the binarization process;
FIG. 11 is a graph of the results of edge detection;
FIG. 12 is a graph of the result of a binarization operation performed again on an image;
fig. 13 is a graph of the result of finding straight line segments.
Reference numerals
1. A binocular vision system; 2. a computer device; 3. and (5) a mechanical arm.
Detailed Description
The technical scheme of the invention is further described below through the attached drawings and the embodiments.
Unless defined otherwise, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this invention belongs.
Example 1
For better understanding of the purpose, structure and function of the present invention, a method for measuring the recognition, positioning and placement angle of a paper box based on YOLOv5 according to the present invention will be described in further detail with reference to fig. 1 and 2: the method comprises the following steps:
s1, acquiring and preprocessing images, and shooting 200 pictures of various postures of cartons with different sizes in an actual carton stacking scene by using a binocular vision system 1 and transmitting the pictures into computer equipment 2. And (3) performing image preprocessing in a data preprocessing frame, and manually labeling cartons with different length-width ratios in the acquired images by adopting an image segmentation method. And carrying out data amplification by the methods of rotation, translation, overturning, mirroring and the like to obtain a data set of 2000 amplified pictures. And randomly extracting and dividing the training set and the verification set according to the cartons with different length-width ratios in a ratio of 5:1.
S2, training the training set through 300 rounds by using a YOLOv5 target detection algorithm, namely a deep convolutional neural network, wherein the number of parameters which are transmitted to a program for training in a single time in the training process is 16, and the best training model best.
S3, as shown in FIG. 5, taking a carton picture in an actual application scene by using the binocular vision system 1, transmitting the carton picture into the computer equipment 2, and detecting, classifying and identifying the obtained picture by using a trained model to obtain carton type and predicted frame coordinate information of the carton picture.
S4, calculating the center coordinates (m, n) of the carton according to the predicted frame coordinate information of the carton, wherein the calculation formula is as follows:
where (x, y) is the upper left corner coordinate of the prediction block, w is the width of the prediction block, and h is the height of the prediction block. As shown in fig. 6, a carton class is obtained: box Six, the upper left corner coordinates of the predicted box: (475,617) the center coordinates were obtained as: (774, 1246).
S5, when the binocular vision system 1 shoots a carton image in an actual application scene, a cvtColor function in OpenCV is utilized to acquire a corresponding gray level image. According to the obtained coordinate information, gray values of the central points of the cartons are obtained on the corresponding gray images, and gray value sorting is performed, as shown in fig. 7, the gray values of the central points of the cartons are obtained on the gray images, wherein the gray values are the following that: 118. the smaller the gray value, the closer the carton is to the fixed binocular vision system 1 and the robot arm 3. Therefore, the carton with the smallest central gray value is selected, carton category is output, frame coordinate information is predicted, and the central coordinate obtained through S4 calculation is obtained. All pixels except the predicted frame of the selected carton are cut out on the gray-scale image according to the coordinate information of the predicted frame, as shown in fig. 8.
S6, gaussian filtering is carried out on the gray level image of the paper box after cutting, most of noise is removed, and noise interference is reduced, as shown in fig. 9. The gaussian filter formula is as follows:
where (x, y) is the coordinates of the gray scale image and σ is the variance of x.
S7, performing binarization processing on the gray level image by using an OpenCV binarization function threshold, and well removing the influence of light on the image, so that the edges of the carton are more obvious, as shown in FIG. 10. The binarization formula is as follows:
where maxval is the maximum threshold, scr (x, y) is the pixel value corresponding to the original gray scale image, and thresh is the threshold.
And S8, utilizing OpenCV-Scharr edge detection to find out edge lines of the carton, as shown in FIG. 11.
Wherein, first order Scharr edge detection operator in X direction is:
the first order Scharr edge detection operator in the Y direction is:
s9, performing binarization operation on the image again, and obviously dividing the edge line and surrounding pixel points into black and white colors, as shown in FIG. 12.
S10, searching a straight line segment of the current drawing by using OpenCV-HoughLinesP (Hough transform straight line detection), traversing data obtained through Hough transform, and drawing a line on the original drawing, as shown in FIG. 13. The OpenCV-HoughLinesP principle is:
r＝xcosθ+ysinθ
where (x, y) is a coordinate point and θ is an angle, each pair of (r, θ) is a straight line passing through the point (x, y). Finding out straight lines passing through all points (x, y) on the polar angle plane of the polar diameter to obtain a sine curve. If the curves obtained by the above operations of two coordinate points of the image have an intersection point, which means that the two coordinate points pass through a straight line, the straight line of the parameter represented by the intersection point in the original image can be obtained.
S11, using an OpenCV-minArearact function to mainly obtain a rectangle containing the minimum area of a point set of data obtained through Hough transformation, and obtaining a deflection angle (placement angle) of the rectangle, namely a deflection angle relative to an x axis in an x-y coordinate system, and performing space positioning on the carton to obtain final target detection information, wherein the deflection angle obtained through calculation is 1.13818359375.
S12, the binocular vision system 1 acquires final target detection information of the carton to be grabbed through an information transmission flow, and processes image information acquired by the left camera and the right camera of the binocular vision assembly by utilizing a correlation matching algorithm and a triangulation principle to calculate the spatial depth distance of the carton. The principle of calculating the spatial depth distance of the carton is as follows:
as shown in fig. 3, P is a certain point On the object to be measured, two optical centers of the binocular camera are Om and On respectively, imaging points of a point P On the object to be measured On two camera photoreceptors of the binocular camera are P 'and P ", f is a focal length of the left and right cameras of the binocular camera, B is a distance between the optical centers of the two cameras, Z is required depth information, and a distance from the set point P' to the set point P" is dis, which includes:
dis＝B-(Xm-Xn)
according to the principle of similar triangles:
the focal length f and the camera optical center distance B in the formula can be obtained through a calibration process, so that the depth information Z of the point P on the object to be measured can be obtained only by obtaining the values of Xm-Xn (namely parallax d).
And S13, transmitting the calculated carton space depth distance to a mechanical arm control assembly through an information transmission flow under the state of real-time detection, and completing real-time grabbing of corresponding workpieces by the mechanical arm control assembly according to the space depth information of the workpieces.
S14, the mechanical arm 3 adjusts the angle of the grasped paper box according to the acquired angle information.
S15, orderly stacking the cartons with the adjusted angles at corresponding placement positions.
And repeating the steps S3-S15 to quickly and accurately finish carton classification, positioning, grabbing and stacking.
Example two
As shown in fig. 4, the measuring method of the recognition, positioning and placement angle of the carton based on YOLOv5 comprises a carton image acquisition and preprocessing module, a carton detection and recognition module, a carton information acquisition module and a carton grabbing and stacking module.
The carton image acquisition and preprocessing module comprises: the carton image acquisition and preprocessing module comprises a carton image acquisition sub-module and a data set generation sub-module.
Carton detection and identification module: and the carton detection and identification module uses a YOLOv5 target detection algorithm to train the training set through 300 rounds to obtain a model with optimal training.
The carton information acquisition module is used for: the carton positioning module comprises a carton category information acquisition sub-module, a carton angle information acquisition sub-module and a carton depth information acquisition sub-module.
Carton snatchs and pile up neatly module: the carton grabbing and stacking module comprises a carton grabbing sub-module and a carton stacking sub-module.
Wherein, carton image acquisition submodule: in an actual carton stacking scene, pictures of various postures of cartons with different sizes are shot by using the binocular vision system 1.
A data set generation sub-module: and manually labeling cartons with different length-width ratios in the acquired images by adopting an image segmentation method. And carrying out data amplification by means of rotation, translation, overturning, mirroring and the like to obtain a final data set. And randomly extracting and dividing the training set and the verification set according to cartons with different length-width ratios.
Carton category information acquisition submodule: and shooting a carton picture in an actual application scene by using the binocular vision system 1, and simultaneously acquiring a corresponding gray level image by using a cvtColor function in OpenCV. And detecting, classifying and identifying the obtained original carton pictures by using the trained models to obtain carton type and predicted frame coordinate information of the original carton pictures. According to the predicted frame coordinate information of the cartons, calculating the center coordinates (m, n) of the cartons, obtaining gray values of the center points of the cartons on the corresponding gray images, sequencing the gray values, and selecting the cartons closest to the binocular vision system 1 and the mechanical arm 3. Outputting the carton category, predicting frame coordinate information and center coordinates.
Carton angle information obtains submodule: and cutting all pixels except the predicted frame of the selected carton out of the gray image according to the coordinate information of the predicted frame. And carrying out Gaussian filtering on the gray level image of the cut carton, removing most of noise and reducing noise interference. And then using an OpenCV function to calculate the deflection angle (placement angle) of the paper box, namely the deflection angle relative to the x axis in an x-y coordinate system, and carrying out space positioning on the paper box to obtain final target detection information.
Carton depth information obtains submodule: the binocular vision system 1 acquires final target detection information of the carton to be grabbed, and processes image information acquired by the left camera and the right camera of the binocular vision assembly by utilizing a correlation matching algorithm and a triangulation principle to calculate the spatial depth distance of the carton.
Carton snatchs submodule: and in a state of real-time detection, transmitting the calculated carton space depth distance to a mechanical arm control assembly, and completing real-time grabbing of corresponding workpieces by the mechanical arm control assembly according to the space depth information of the workpieces.
Carton stacking sub-module: the mechanical arm 3 adjusts the angle of the grasped carton according to the acquired angle information. And neatly stacking the cartons with the adjusted angles at corresponding placement positions.
Therefore, the method for measuring the recognition, positioning and placing angles of the paper boxes based on the YOLOv5 has the advantages of simple system structure and high detection precision, can obtain a better target detection effect, can meet the requirements of grabbing and stacking the paper boxes in an actual application environment, reduces the use and unnecessary operation of personnel, and improves the working efficiency.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention and not for limiting it, and although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that: the technical scheme of the invention can be modified or replaced by the same, and the modified technical scheme cannot deviate from the spirit and scope of the technical scheme of the invention.
Claims (4)
1. A measuring method for recognizing, positioning and placing angles of cartons based on YOLOv5 is characterized by comprising the following steps: the method comprises the following steps:
s1, shooting a paper box picture, transmitting the paper box picture into computer equipment, marking and amplifying data, and dividing a data set;
s2, training the training set by using a YOLOv5 algorithm, and classifying and identifying by using a trained model to obtain a detection model;
s3, shooting a carton picture in an actual application scene by using a binocular vision system, detecting the carton picture by using a detection model to obtain carton type and predicted frame coordinate information, and calculating a center point coordinate of the carton;
s4, combining an OpenCV software library on the basis of a YOLOv5 algorithm, further processing the obtained image set of the target detection information, and detecting the placement angle of the carton;
s5, the binocular vision system acquires detection information of the carton to be finally grabbed, and image information acquired by the left camera and the right camera of the binocular vision system is processed by utilizing a correlation matching algorithm and a triangulation principle to calculate the spatial depth distance of the carton;
s6, transmitting the calculated space depth distance of the cartons to a mechanical arm control assembly in a real-time detection state, and completing real-time grabbing of the corresponding cartons by the mechanical arm control assembly according to the space depth information of the cartons;
s7, the mechanical arm control assembly adjusts the angle of the grasped paper box according to the acquired angle information;
s8, orderly stacking the cartons with the adjusted angles at corresponding placement positions.
2. The method for measuring the recognition, positioning and placement angle of the paper box based on YOLOv5 according to claim 1, wherein the method comprises the following steps: the step S1 specifically comprises the following steps:
s11, shooting pictures of various postures of cartons of different sizes in an actual carton stacking scene by using a binocular vision system, and transmitting the pictures into computer equipment;
s12, labeling the carton pictures by using a person to generate a data set;
s13, carrying out data amplification by a method of rotation, translation, overturning and mirroring to obtain an amplified picture data set.
3. The method for measuring the recognition, positioning and placement angle of the paper box based on YOLOv5 according to claim 1, wherein the method comprises the following steps: the step S3 specifically comprises the following steps:
s31, identifying and classifying the carton pictures by using the detection model obtained in the step S2 to obtain carton type and predicted frame position information;
s32, calculating the center coordinates (m, n) of the carton according to the obtained predicted frame coordinates, wherein the calculation formula is as follows:
wherein (x, y) is the upper left corner coordinate of the prediction frame, w is the width of the prediction frame, and h is the height of the prediction frame;
s33, when the binocular vision system shoots a carton image in an actual application scene, a cvtColor function in OpenCV is utilized to acquire a corresponding gray level image;
s34, according to the obtained coordinate information, gray values of the center points of the cartons are obtained on the corresponding gray images, and gray value sorting is carried out; and selecting a carton with the smallest central gray value, and outputting carton category, predicted frame coordinate information and the calculated central coordinate.
4. The method for measuring the recognition, positioning and placement angle of the paper box based on YOLOv5 according to claim 1, wherein the method comprises the following steps: the step S4 specifically includes:
s41, cutting all pixels except the predicted frame of the selected carton from the gray level image according to the coordinate information of the predicted frame;
s42, carrying out Gaussian filtering on the gray level image of the carton after cutting, carrying out weighted average on the whole image, and obtaining the value of each pixel point by carrying out weighted average on the pixel point and other pixel values in the neighborhood, wherein the Gaussian filtering formula is as follows:
wherein (x, y) is the coordinates of the gray scale image, σ is the variance of x;
s43, performing binarization processing on the gray level image by using an OpenCV binarization function threshold, setting the gray level value of a pixel point on the image to be 0 or 255, wherein a binarization formula is as follows:
wherein maxval is the maximum threshold, scr (x, y) is the pixel value corresponding to the original gray image, and thresh is the threshold;
s44, finding out edge lines of the carton by utilizing OpenCV-Scharr edge detection, wherein a first-order Scharr edge detection operator in the X direction is as follows:
the first order Scharr edge detection operator in the Y direction is:
s45, performing binarization operation on the image again, and obviously dividing the edge line and surrounding pixel points into black and white;
s46, searching a straight line segment of the current diagram by using OpenCV-HoughLinesP, traversing the data obtained through Hough transformation, and drawing a line on the original diagram;
s47, using an OpenCV-minArearect function to mainly obtain a rectangle containing the minimum area of a point set of data obtained through Hough transformation, obtaining the deflection angle of the rectangle relative to an x axis in an x-y coordinate system, and performing space positioning on the carton to obtain final target detection information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211695904.1A CN116188763A (en) | 2022-12-28 | 2022-12-28 | Method for measuring carton identification positioning and placement angle based on YOLOv5 |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211695904.1A CN116188763A (en) | 2022-12-28 | 2022-12-28 | Method for measuring carton identification positioning and placement angle based on YOLOv5 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116188763A true CN116188763A (en) | 2023-05-30 |
Family
ID=86449940
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211695904.1A Pending CN116188763A (en) | 2022-12-28 | 2022-12-28 | Method for measuring carton identification positioning and placement angle based on YOLOv5 |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116188763A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117670875A (en) * | 2024-01-31 | 2024-03-08 | 深圳市恒星包装机械有限公司 | Visual detection method and system in canning tail sealing process |
| CN118411576A (en) * | 2024-07-04 | 2024-07-30 | 江苏剑山包装科技有限公司 | Carton classification method and device based on data processing |
-
2022
- 2022-12-28 CN CN202211695904.1A patent/CN116188763A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117670875A (en) * | 2024-01-31 | 2024-03-08 | 深圳市恒星包装机械有限公司 | Visual detection method and system in canning tail sealing process |
| CN117670875B (en) * | 2024-01-31 | 2024-04-02 | 深圳市恒星包装机械有限公司 | Visual detection method and system in canning tail sealing process |
| CN118411576A (en) * | 2024-07-04 | 2024-07-30 | 江苏剑山包装科技有限公司 | Carton classification method and device based on data processing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11958197B2 (en) | Visual navigation inspection and obstacle avoidance method for line inspection robot | |
| CN109903331B (en) | Convolutional neural network target detection method based on RGB-D camera | |
| CN104778721B (en) | The distance measurement method of conspicuousness target in a kind of binocular image | |
| CN103530599B (en) | The detection method and system of a kind of real human face and picture face | |
| US8578299B2 (en) | Method and computing device in a system for motion detection | |
| CN116188763A (en) | Method for measuring carton identification positioning and placement angle based on YOLOv5 | |
| CN110189375B (en) | Image target identification method based on monocular vision measurement | |
| US20150371391A1 (en) | 2D/3D Localization and Pose Estimation of Harness Cables Using A Configurable Structure Representation for Robot Operations | |
| CN109145803B (en) | Gesture recognition method and device, electronic equipment and computer readable storage medium | |
| CN107392929B (en) | Intelligent target detection and size measurement method based on human eye vision model | |
| JP6912215B2 (en) | Detection method and detection program to detect the posture of an object | |
| CN111160291B (en) | Human eye detection method based on depth information and CNN | |
| CN111998862B (en) | BNN-based dense binocular SLAM method | |
| CN110751097B (en) | Semi-supervised three-dimensional point cloud gesture key point detection method | |
| CN109752855A (en) | A kind of method of hot spot emitter and detection geometry hot spot | |
| CN113688846A (en) | Object size recognition method, readable storage medium, and object size recognition system | |
| CN111127556A (en) | Target object identification and pose estimation method and device based on 3D vision | |
| CN108154496B (en) | Electric equipment appearance change identification method suitable for electric power robot | |
| CN112560704A (en) | Multi-feature fusion visual identification method and system | |
| CN113689365B (en) | Target tracking and positioning method based on Azure Kinect | |
| CN108388854A (en) | A kind of localization method based on improvement FAST-SURF algorithms | |
| Li et al. | Deep-trained illumination-robust precision positioning for real-time manipulation of embedded objects | |
| CN110942092A (en) | Graphic image recognition method and recognition system | |
| CN113932712A (en) | Melon and fruit vegetable size measuring method based on depth camera and key points | |
| CN114842335B (en) | Grooving target identification method and system for construction robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |