CN117152175A - Machine vision-based waste plastic material position and orientation recognition method - Google Patents

Abstract

The invention discloses a method for identifying the pose of waste plastics based on machine vision, which includes: acquiring images of waste plastics through an industrial color camera; performing grayscale processing on the acquired RGB images according to the grayscale formula; dividing the image into horizontal Multiple regions, and threshold segmentation of limited empirical gray levels for each channel; extract the waste plastic contours of each channel, and calculate the channel centroid, and finally calculate the centroid and angle of the entire waste plastic based on the centroid coordinates of multiple channels, and we get Get rid of waste plastic. This method increases the calculation angle information, thereby increasing the success rate of large-angle waste plastic sorting; and reduces the traversal range of the optimal threshold when performing threshold segmentation, thereby reducing the calculation amount of threshold segmentation and reducing the consumption of image recognition. time.

Description

A method for identifying the pose of waste plastics based on machine vision

Technical field

The invention relates to the field of machine vision, and in particular to a method for identifying the pose of waste plastics based on machine vision.

Background technique

The beverage industry uses lightweight plastic bottles as packaging containers on a large scale. These plastic bottles are made from non-renewable petroleum through polymerization and are difficult to degrade in nature. If waste plastics are not recycled and processed, the resulting garbage will be buried, burned or discarded at will, which will bring great harm to the ecological environment and cause a lot of waste of resources. Sorting of waste plastics And recycling is the key to solving this problem. During the recycling process, waste plastics come in different colors, qualities and sizes and need to be sorted. At present, the sorting of waste plastics at home and abroad is mainly based on manual sorting. Manual sorting not only requires high work intensity, low sorting efficiency and poor working environment, so there is an urgent need for a fast and stable automatic sorting equipment to replace manual sorting. .

The machine vision system can capture, process and analyze images at extremely fast speeds, improving processing efficiency and processing capabilities; it can deal with different types and shapes of waste plastics; it can seamlessly connect with other equipment such as conveyor belts and pneumatic devices to achieve comprehensive Automatic waste plastic sorting production line. Generally speaking, the nozzles of the pneumatic sorting part at the end of the waste plastic sorting system are discrete, and the position of the center of mass of the plastic is often inconsistent with the position of the spray head. When sorting waste plastics with large shapes and postures, the air pressure and flow rate of a single spray head are insufficient, resulting in sorting failure. Therefore, when sorting waste plastics, multiple nozzles may need to spray airflow at different times, which requires not only sensing the center of mass position of the waste plastics, but also obtaining the posture information of the waste plastics. The following are some research progress on position and attitude recognition algorithms.

Patent CN109190493A discloses an image recognition method that uses grayscale processing to obtain a grayscale image, and then conducts a multi-threshold search on the grayscale image to obtain the optimal harmonic threshold solution set for segmentation, which can accurately locate the specific parts of the apple. Position. Although this method uses threshold segmentation to accurately locate the position of the apple, it does not calculate its posture and is not suitable for sorting waste plastics at large angles.

Patent CN115082560A discloses a material pose recognition method that obtains the original image containing the part, generates the minimum circumscribed rectangle of the part in the original image, obtains the position data and main direction angle of the part based on the minimum circumscribed rectangle, and combines the position data and rotation The angle is mapped to the world coordinate system to obtain the pose of the part; patent CN113269835A provides a method for identifying the pose of industrial parts based on contour features, by obtaining the image of the industrial part, and then obtaining the edge pixels of the industrial part based on the image point coordinates, and then obtain the corner point coordinates of the four corner points of the minimum circumscribed rectangle of the industrial part based on the edge pixel point coordinates, and finally calculate the center point coordinates of the minimum circumscribed rectangle as the position coordinates of the industrial part, and then calculate Regarding the attitude angle of the industrial parts, both patents used the minimum circumscribed rectangle method to calculate the position information and angle information of the part. However, experiments show that calculations are required when using the minimum circumscribed rectangle method for threshold segmentation to traverse the optimal threshold. The information entropy of 256 gray levels has high requirements on the computer hardware system.

Contents of the invention

In response to the above existing problems, the present invention proposes a method for identifying the posture of waste plastics based on machine vision. This method can quickly identify the posture of waste plastics on the conveyor belt and calculate the position information and angle information of the waste plastics, which not only reduces the The time consumed by the algorithm also improves the effect of large-angle waste plastic sorting.

The purpose of the present invention is achieved through the following technical solutions:

A method for identifying the pose of waste plastics based on machine vision, including the following steps:

Step S1: Install the industrial color camera above the conveyor belt. When the waste plastic appears within the field of view of the industrial color camera, the industrial color camera collects images of the waste plastic in real time;

Step S2: Use the weighted average method to grayscale the image obtained in step S1 to separate the waste plastic in the original image from the background;

Step S3: Use the horizontal average cutting method to segment the grayscale processed image obtained in step S2 into multiple horizontal areas corresponding to the camera field of view, and then set a fixed range threshold for each area to perform empirical threshold segmentation processing. , the binary image is obtained after threshold segmentation;

Step S4: Use the Suzuki algorithm to extract the contours of multiple regions on the binary image obtained by threshold segmentation in step S3, then use the contour moments to calculate the positions of the centroids of multiple regions, and finally calculate the waste according to the formula based on the centroid position of each region. The center of mass and angle of the entire plastic finally realize the position and posture identification of waste plastic.

Beneficial effects of the present invention: the empirical threshold segmentation of sub-regions is used to divide the image into N horizontal regions corresponding to the camera field of view using transverse cutting, and then the empirical threshold segmentation is performed, and the centroid position of each region is used to calculate the overall waste. The angle of the plastic can accurately control the number of nozzles used when sorting large-angle waste plastics and the action time of each nozzle. It also reduces the calculation time of the algorithm, thus improving the efficiency and success rate of waste plastic sorting.

Description of the drawings

Figure 1 is a flow chart of the method of the present invention.

Figure 2 is a flow chart of the empirical threshold segmentation calculation of sub-regions.

Figure 3 shows the original image of waste plastic.

Figure 4 is a grayscale image of waste plastic.

Figure 5 is a grayscale image of waste plastic segmented by area.

Figure 6 Topological relationship diagram of contour boundary.

Figure 7 shows the result of contour extraction of waste plastics by area.

Detailed ways

The present invention will be described in detail below based on the accompanying drawings and examples, and the purpose and effect of the present invention will become clearer. The specific embodiments described here are only used to explain the present invention and are not intended to limit the present invention.

As shown in Figures 1 and 2, the present invention is a pose recognition method for real-time sorting of waste plastics, which specifically includes the following steps:

Step S1: Install the industrial color camera above the conveyor belt. When the waste plastic appears within the field of view of the industrial color camera, the industrial color camera will acquire the image of the waste plastic in real time and transmit it to the computer;

Step S2: Use the weighted average method to grayscale the image obtained in step S1 to separate the waste plastic in the original image from the background;

Step S3: Use the horizontal average cutting method to segment the grayscale processed image obtained in step S2 into N horizontal areas corresponding to the camera field of view, and then set a fixed range threshold for each area to perform empirical threshold segmentation processing. , the binary image is obtained after threshold segmentation;

Step S4: Use the Suzuki algorithm to extract the contours of N regions from the binarized image obtained after the empirical threshold segmentation in step S3. Then use the contour moments to calculate the positions of the centroids of these N regions. Finally, based on the centroid position of each region Calculate the center of mass and angle of the entire waste plastic through formulas, and finally realize the pose identification of waste plastic;

The described method for identifying the pose of waste plastics based on machine vision, the specific method of step S2 is: according to the importance of the three different color channels of RGB, the image acquired by the industrial color camera is given the color value of the three RGB channels respectively. After multiplying by different weights, a grayscale value is obtained, and finally the RGB image is converted into a grayscale image. The specific calculation formula is as shown in formula (1):

Grey＝αR+βG+γB#(1)

Among them, α, β, and γ are respectively obtained according to the corresponding weights based on the brightness contribution of three different color channels. After experimental comparison, it is demonstrated that the best effect is obtained when α = 0.07, β = 0.72, and γ = 0.21 for grayscale processing, as follows Figures 3 and 4 are respectively the original color image of waste plastic and the result image after grayscale processing.

The described method for identifying the pose of waste plastics based on machine vision, the specific method of step S3 is: in order to shorten the traversal search time of the segmentation threshold during the threshold segmentation process, determine the segmentation threshold in advance instead of traversing the threshold for each frame of image. , this fixed threshold segmentation method is more suitable for occasions where the background is relatively single and the segmentation threshold fluctuates less, and is more suitable for the working scenario of the present invention. The black and gray conveyor belt background used in the work of the present invention has little local pixel value change during the movement, but the color difference between different areas is large. In the grayscale mode in the range of 0-255, the gray value difference reaches 50 If equal fixed thresholds are used for segmentation in different areas, large errors will occur. Therefore, consider partitioning the image range captured by the camera, setting the number of regions to N, and setting a fixed range of empirical thresholds in each region for threshold segmentation processing, and calculating the height of each region based on the height of the image. height, and then slice the image horizontally according to the predefined number of rows. The empirical average gray value of the segmented N strip areas can be calculated as the basis for calculating the fixed threshold of the area.

Taking N=12 as an example, Figure 5 is the rendering after grayscale image segmentation. Some of the data obtained from the test are listed in Table 1 below. The values in Table 1 represent the average grayscale value of the channel when there is only the background, and the spaces represent the Data points containing plastics to be identified are not included in the calculation.

Table 1 Statistical table of average gray value of each channel

The grayscale empirical value of the background of each channel when N=12 is calculated from the data in Table 1. The results are as follows in Table 2. The empirical grayscale average value in Table 2 plus a specific adjustment value, and the empirical value of the adjustment value is set to 50, can be used as the maximum entropy threshold traversal range to distinguish the background and foreground.

Table 2 Grayscale statistical average of each channel

When performing threshold segmentation processing on an image, first segment the entire image into N horizontal areas, and then perform separate maximum entropy threshold segmentation on each area. Assume that the statistical average of the background grayscale of this channel is x, then At this time, the traversal range of the maximum entropy segmentation threshold is not [0, 255] but [x, x+50]. If the threshold that maximizes image entropy appears between x and x+50, then the threshold is the maximum entropy segmentation threshold. , if the entropy value of the image increases monotonically in the range of [x, x+50], then x+50 is selected as the segmentation threshold. Although this threshold is not the maximum entropy segmentation threshold in the full grayscale range of [0, 255], it is sufficient. The plastic and conveyor belt background in the image are distinguished, so there is no need to continue traversing to find the maximum entropy segmentation threshold. In this way, the segmentation threshold traversal range of each channel is reduced from 0-255, a total of 256 values to less than 50, which greatly compresses the graphics. processing time.

The described method for identifying the pose of waste plastics based on machine vision, the specific method of step S4 is: use the Suzuki algorithm to extract the contours of the binary image after completing the empirical threshold segmentation of the sub-region, and the Suzuki algorithm performs contour extraction on the digital binary image. Topological analysis, and creatively use the outer boundary of the image, the hole boundary and their hierarchical relationship to express the outline of the digital image. The definition of the outer boundary and the hole boundary is: if there are 1 connected domain S1 and 0 connected domain S2, if S2 directly surrounds S1, the boundary between S2 and S1 is called the outer boundary; if S1 directly surrounds S2, then S2 and The edge between S1 is called the hole boundary, and both the outer boundary and the hole boundary are composed of 1 pixel. The surround of a connected area refers to the connection to two adjacent connected areas S1 and S2. If S2 can be reached from any 4 directions of any point on S1, then S2 surrounds S1. The definition of the hierarchical relationship between contours is: Assume that there are existing 1-connected domains S1 and S3, and 0-connected domain S2; S2 directly surrounds S1, S3 directly surrounds S2, the boundaries between S1 and S2 are B1, and the boundaries between S2 and S3 The boundary is B2, then B2 is the parent boundary of B1. If S2 is the background, then the parent boundary of B1 is the image border. The topological relationship between the image and the border is shown in Figure 6 below. The Suzuki algorithm contour extraction method is a method similar to raster scanning, that is, scanning the image pixels from left to right and from top to bottom. Using the boundary tracking algorithm, multiple contours of the image can be obtained, and each contour is given a unique number. , during the scanning process, NBD is used to represent the number of the currently tracked boundary, and LNBD is used to represent the number of the previous saved boundary.

The specific process of extracting the original outline of the Suzuki algorithm is as follows: if the input image is F = {f (i, j)}, where i and j are the abscissa and ordinate of the pixel respectively, set the initial NBD to 1, and change the image The border is regarded as the first contour boundary, the image is scanned, and when a certain pixel point f(i, j)≠0 is scanned, the following steps are performed:

(1) Which of the following situations is determined?

(a) If f(i,j)=1 and f(i,j-1)=0, then (i,j) is the starting point of the outer boundary, let NBD=NBD+1, and save (i ₂ , j ₂ ) The coordinates are equal to (i, j-1).

(b) If f(i,j)≥1 and f(i,j+1)=0, then (i,j) is the starting point of the hole boundary, let NBD=NBD+1, and save (i ₂ , j ₂ ) The coordinates are equal to (i, j+1).

(c) In other cases, jump to (4)

(2) Based on the saved previous boundary and the currently encountered boundary, query the parent boundary of the current boundary from Table 3, where B’ is the previous boundary and B is the new boundary;

Table 3 Parent boundary of current boundary

(3) Starting from the boundary starting point (i, j), perform boundary tracking according to the following algorithm

(3.1) With (i, j) as the center and (i ₂ , j ₂ ) as the starting point, search the eight neighborhoods of (i, j) in the clockwise direction to determine whether there are non-zero pixels. If a non-zero pixel is found, let the coordinates of the first non-zero pixel in the clockwise direction be (i ₁ , j ₁ ) and go to (3.2); otherwise, let f (i, j) = -NBD, and Go to(4);

(3.2) Update the coordinates of (i ₂ , j ₂ ) to (i ₁ , j ₁ ), and the coordinates of (i ₃ , j ₃ ) to (i, j);

(3.3) Taking (i ₃ , j ₃ ) as the center, starting from (i ₂ , j ₂ ), search in the counterclockwise direction whether there are non-zero pixels in the eight neighborhoods of (i ₃ , j ₃ ), and let ( i ₄ , j ₄ ) are the coordinates of the first non-zero pixel encountered, where the eight areas refer to the eight adjacent pixels around a pixel, that is, for a given pixel (i, j), its The position of the eight neighbors can be expressed as (i-1, j-1), (i-1, j), (i-1, j+1), (i, j-1), (i, j), (i, j+1), (i+1, j-1), (i+1, j), (i+1, j+1). In contrast, if only four neighborhoods (above) are considered , bottom, left, right), the boundary information in the diagonal direction may be ignored, resulting in incomplete or inaccurate extracted contours;

(3.4) If (i ₃ , j ₃ +1) is the pixel checked in (3.3) and is a zero pixel, then let f (i ₃ , j ₃ )=-NBD; if (i ₃ , j ₃ +1) If it is not a pixel checked in (3.3) and is a non-zero pixel, then f(i ₃ , j ₃ )=NBD;

(3.5) If (i ₄ , j ₄ ) = (i, j) and (i ₃ , j ₃ ) = (i ₂ , j ₂ ) means that it has returned to the starting point, jump to (4); otherwise update (i ₂ The coordinates of , j ₂ ) are (i ₃ , j ₃ ), the coordinates of (i ₃ , j ₃ ) are (i ₄ , j ₄ ), and go to (3.3);

(4) If f(i,j)≠1, then LNBD=|f(i,j)|, continue scanning from point (i,j+1), and the algorithm ends when scanning to the lower right corner vertex of the picture.

The original Suzuki algorithm process is improved and simplified. The present invention uses the Suzuki algorithm process as follows: use f(i, j) to represent the pixel value of point (i, j), and scan the waste plastic binarized image line by line until encountering a connected area. When the pixel value of a point (m, n) appears f (m-1, n) = 0 and f (m, n) = 255, this point is the first point on the contour. Taking point (m-1, n) as the starting point, traverse the eight neighborhoods of point (m, n) counterclockwise, and traverse point (m, n) counterclockwise using the first discovered point that is not 0 as the starting point. In the eight-neighborhood, the first point found that is not 0 is the next point of the contour. Follow this method until the outline is completely closed. Mark pixels on the boundary of this connected domain and assign a unique identifier to this boundary. After discovering a new connected domain again, add 1 to the identifier and trace the contour in the same way. After getting all the contours of the waste plastic, select the largest one and export it.

After obtaining the contour of each area of the waste plastic, the present invention uses the moments of the contour to obtain the contour centroid position of each area of the waste plastic. The (p+q)-order contour moment m(pq) _i of the i-th region is defined as follows:

Among them, x and y are the abscissa and ordinate values of the pixel point on the contour, I(x, y) is the pixel value of the pixel coordinate point (x, y) (the value is 0 or 1), n _i is the i-th The number of pixels on the area contour; when solving the centroid, the first-order contour moment is used, that is, the values of the coefficients p and q are 0 or 1 respectively, but the sum of the two is 1, that is, p+q=1.

In the contour image, the pixel values of the points on the contour are all 1, so the zero-order moment m(00) _i is the number of points on the contour of the i-th area, and the first-order contour moment m(10) _i , m(01) _i is respectively the accumulation of the x-coordinate value and y-coordinate value of each pixel on the i-th area outline. The centroid position (x _i , y _i ) corresponding to the i-th region outline can be calculated by equation (3).

Extract the contour of the binary image and obtain the centroid position according to the above method. The result is shown in Figure 7 below. After calculating the coordinates of the center of mass of each area, use the following formulas to calculate the total center of mass (x, y) of the waste plastic and the angle θ of the waste plastic.

Where n is the number of connected domains, (xi _, y _i ) is the contour centroid of the i-th connected domain.

The centroid, angle and calculation time (when N=12) calculated for the waste plastic bottles in Figure 7 using the above formula are shown in Table 4 below.

Table 4 Calculation results of waste plastic poses

It can be seen from the above table that the use of regional empirical threshold segmentation method can effectively reduce the calculation amount of threshold segmentation, and the calculation time is reduced to 23ms, which greatly reduces the time consumed in image recognition.

The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those of ordinary skill in the art, other modifications can be made without departing from the spirit of the present invention. Various changes.

Claims (5)

1. The waste plastic material position and posture identification method based on machine vision is characterized by comprising the following steps of:

step S1: the industrial color camera is arranged above the conveyor belt, and when the waste plastic is in the visual range of the industrial color camera, the industrial color camera acquires the image of the waste plastic in real time;

step S2: carrying out graying treatment on the image obtained in the step S1 by using a weighted average method, and distinguishing waste plastics from the background in the original image;

step S3: dividing the gray-level processed image obtained in the step S2 into a plurality of transverse areas corresponding to the camera view field by using a transverse average cutting method, setting a threshold value of a fixed range for each area, and performing empirical threshold segmentation processing, wherein the threshold value is segmented to obtain a binarized image;

step S4: and (3) carrying out contour extraction on the plurality of areas by using a Suzuki algorithm on the binarized image obtained by threshold segmentation in the step (S3), calculating the positions of centroids of the plurality of areas by using contour moments, calculating the integral centroid and angle of the waste plastic according to the centroid position of each area through a formula, and finally realizing pose recognition of the waste plastic.

2. The machine vision-based waste plastic material position and orientation recognition method according to claim 1 is characterized in that the step S2 of weighted average gray scale processing specifically comprises the steps of multiplying color values of three RGB channels according to the importance of the three RGB channels by different weights respectively to obtain a gray scale value, and finally converting an RGB image into a gray scale image.

3. The machine vision-based waste plastic material position and orientation recognition method according to claim 1, wherein the specific method for transverse average cutting and empirical threshold segmentation in the step S3 is as follows: firstly defining a cutting line, calculating the height of each region according to the height of an image, then cutting the image into a plurality of regions in the horizontal direction according to a predefined line number, counting the average gray average value of the background region of the conveyor belt for each region, setting a threshold traversing range for distinguishing the background from the foreground according to the average gray value, and finally carrying out independent threshold segmentation on each region to generate a corresponding binarized image.

4. The machine vision-based waste plastic material position and orientation recognition method according to claim 1, wherein the specific method for extracting the outline by using the Suzuki algorithm in the step S4 is as follows: initializing a binary image into a mark image with the same size, initializing all pixels into 0, and using f (i, j) to represent the pixel value of a point (i, j), scanning the waste plastic binary image in a row until encountering a point (m, n) of a connected area, wherein f (m-1, n) =0, and f (m, n) =255, wherein the point is the first point on the outline; traversing the eight neighborhood of the point (m, n) anticlockwise by taking the point (m-1, n) as a starting point, traversing the eight neighborhood of the point (m, n) anticlockwise by taking the first point which is not 0 as the starting point, and taking the first point which is not 0 as the next point of the contour; according to this method, until the profile is completely closed; marking pixels on the boundary of the connected domain and assigning a unique identifier to the boundary; after finding a new connected domain again, adding 1 to the identifier, and tracking the outline according to the same method; and (3) after obtaining all the outlines of the waste plastics, selecting the largest outline and outputting the outline, and finally obtaining a complete outline image.

5. The machine vision-based waste plastic position and orientation recognition method according to claim 1, wherein the calculation formulas of the mass center and the angle of the whole waste plastic are respectively as follows:

wherein (x, y) is the barycenter coordinate of the whole waste plastic, n is the number of areas, (x) _i ，y _i ) Is the centroid of the contour in the i-th region; θ is the angle of the whole waste plastic;

the centroid calculation formula of the contour in the i-th region is:

wherein m (00) _i The number of points on the contour of the ith area, m (10) _i 、m(01) _i The x coordinate value and the y coordinate value of each pixel point on the ith area outline are respectively accumulated.