CN110378970B - Monocular vision deviation detection method and device for AGV - Google Patents

Abstract

The invention provides a monocular vision deviation detection method and device for an AGV (automatic guided vehicle). Gray processing and sampling are carried out on collected image data, a dynamic segmentation threshold value is calculated according to sample image data, threshold value segmentation is carried out on the gray data, and a binary image is obtained; screening the binary image, performing morphological filtering on an object which is probably foreground noise, and performing threshold segmentation on the filtered pixel points again; segmenting the acquired image by using the grids, traversing all grid units, and forming a set by the grid units containing the foreground colors; clustering elements in the set, and separating positioning blocks; calculating the coordinates of the central point of the positioning block by adopting an average value method; and establishing a space conversion relation model between the image coordinate system and the pixel coordinate system according to the central point coordinates of the positioning blocks, and calculating the deviation. The invention can improve the anti-interference capability of the AGV while meeting the use precision.

Description

A monocular vision deviation detection method and device for AGV

technical field

The invention belongs to the technical field of logistics automation, and in particular relates to a monocular vision deviation detection method and device for AGV.

Background technique

In recent years, with the rise of smart logistics, unmanned warehouses have emerged to meet the requirements of "unmanned operations". AGV is widely used in unmanned warehouses as the carrier for goods storage, sorting, and outbound operations in unmanned warehouses. However, inertial navigation AGV has a cumulative effect due to sensor errors. Therefore, a deviation detection device with accurate results, high stability and strong anti-interference ability is needed to eliminate accumulated errors.

In order to eliminate the cumulative error, the currently used two-dimensional code deviation detection method has the advantage of rich information, and can also realize positioning while realizing deviation detection; but also because of its rich information, its anti-interference ability is insufficient. A large amount of useful information will be lost when the border of the QR code is blurred due to strong light interference, and useful information cannot be extracted when it is interfered by stains, resulting in the failure of deviation detection. At the same time, this positioning method also has the disadvantages of difficult equipment maintenance, high cost, and is not conducive to saving the production cost of the enterprise.

Contents of the invention

The technical problem to be solved by the present invention is to provide a monocular vision deviation detection method and device for AGV, which can improve the anti-interference ability of the AGV while satisfying the use accuracy.

The technical solution adopted by the present invention to solve the above-mentioned technical problems is: a monocular visual deviation detection method for AGV, characterized in that it includes the following steps:

S1. Perform grayscale processing on the collected image data to obtain grayscale data; the image data is obtained by photographing the beacon, which contains three positioning blocks, and each positioning block has at least one set The graph of the vertical axis of symmetry, and the center points of the three positioning blocks can form a Cartesian coordinate system;

S2. Sampling the collected image data to obtain sample image data, calculating a dynamic segmentation threshold T with the sample image data, performing threshold segmentation on the grayscale data, and obtaining a binary image;

S3. Screen the binary image, perform morphological filtering on objects that may be foreground noise, and re-threshold the filtered pixels; the object that may be foreground noise refers to pixels that meet a certain range of pixel values point;

S4. Segment the image acquired by S3 using a grid, traverse all the grid units, and form a set S of grid units containing the foreground color;

S5. Cluster the elements in the set S, and separate the positioning blocks;

S6, using the average value method to obtain the coordinates of the center point of the positioning block;

S7. Establish a spatial conversion relationship model between the image coordinate system and the pixel coordinate system according to the coordinates of the center point of the positioning block, and calculate the deviation.

According to the above method, the S1 specifically performs grayscale processing on each pixel row by row, and the grayscale processing method is:

g(x, y) = af _R (x, y) + bf _G (x, y) + cf _B (x, y)

The constraints are: a, b, c are all positive integers;

f _R (x, y), f _G (x, y), f _B (x, y) are the R, G, and B components of the pixel f (x, y), and g (x, y) is the grayscale processed The data.

According to the above method, the S2 specifically includes:

2.1, according to the data obtained in step S1, use the maximum internal variance method to calculate the dynamic segmentation threshold T, the calculation method is:

各像素值出现的概率P_i满足公式：

选取一个整数K将像素分为两组，C₀＝{1，2，……，k}，C₁＝{k+1，k+2，……，m}，则两组之间的方差公式：σ²(k)＝ω₀ω₁(μ₁-μ₀)² Assuming that the sample pixels used are divided into 1, 2, ..., m levels, and the number of pixels with gray value i is n, the total number of pixels N can be expressed by the formula:

The probability P _i of the occurrence of each pixel value satisfies the formula:

Choose an integer K to divide the pixels into two groups, C ₀ ={1, 2,...,k}, C ₁ ={k+1,k+2,...,m}, then the variance between the two groups Formula: σ ² (k) = ω ₀ ω ₁ (μ ₁ -μ ₀ ) ²

μ₀为C₀的均值，

ω₁为C₁发生的概率，

μ₁为C₁的均值，

In the formula: ω ₀ is the probability of C ₀ occurring,

μ ₀ is the mean value of C ₀ ,

ω ₁ is the probability of C ₁ occurring,

μ ₁ is the mean value of C ₁ ,

Change K from (1, 2, ..., m) to find K when the variance is at its maximum value, and the K value when maxσ ² (k) is obtained is the dynamic segmentation threshold T;

2.2. According to the data obtained in step S1 and the dynamic segmentation threshold T obtained in 2.1, obtain a binary image as follows:

In the formula: 1 represents white, 0 represents black, g(x, y) is the gray value of the pixel in column x and row y, and h(x, y) is the value of the pixel in column x and row y after segmentation .

According to the above method, the S3 specifically includes:

To meet:

的像素点使用形态学滤波方法：

The pixels use the morphological filtering method:

In the formula: Med represents the intermediate value after sorting the gray values of all pixels in the filtering window A in ascending order, and A is the filtering window;

Then re-threshold segmentation:

According to the above method, the S5 is clustered according to the following method:

5.1. Traverse all grid units, take the grid coordinates containing the positioning block image as the elements of the set S, and the number of positioning blocks N=0;

5.2. N=N+1, establish a positioning block set A _N ; move the first element in the set S into the set A _N , and move the element out of the set at the same time;

5.3. Sequentially compare the remaining elements in the set S with the grid positions represented by all the elements in the set A _N ; judge whether there are two adjacent grids, and if so, compare the elements corresponding to the grid in the set S Move into the set A _N , and move this element out of the set S at the same time;

5.4. Obtain the final positioning block set A _N .

According to the above method, described S6 uses the following method to obtain the center point coordinates:

For a graph with two vertical symmetry axes, use the average value algorithm to calculate the center point of the graph, and the coordinates of the center point of the graph satisfy the formula:

In the formula: x _t is the abscissa of the center point of the t th positioning block, y _t is the ordinate of the center point of the t th positioning block, n is the total number of black pixels in the t th positioning block, x _i is The abscissa of the i-th black pixel in the t-th positioning block, and y _i is the ordinate of the i-th black pixel in the t-th positioning block.

According to the above-mentioned method, described S7 specifically comprises:

Establish a pixel coordinate system by judging the relative positional relationship between the center point coordinates of the positioning blocks;

Calculate which quadrant the center point of the image is in the pixel coordinate system;

Calculate the distance between the center point of the image and the pixel coordinates X-axis and Y-axis;

Correct the positive and negative relationship of the deviation value according to the quadrant of the image center point;

According to the coordinate axis rotation relationship between the pixel coordinate system and the image coordinate system, the included angle between the corresponding coordinate axes of the two coordinate systems is calculated.

A monocular visual deviation detection device for AGV, characterized in that: the device includes an image acquisition device, a memory and a data processor arranged on the AGV; wherein the image acquisition device is used to collect image data, and the memory stores The computer program is called by the data processor to complete the monocular vision deviation detection method for AGV.

The beneficial effects of the present invention are:

1. After obtaining the beacon image, by extracting the feature points in the beacon, the relative positional relationship between the image sensor center point and the beacon positioning point can be established, and the absolute coordinate system of the image sensor center point relative to the beacon positioning point can be calculated. The offset and included angle of the lower X-axis and Y-axis; the automatic threshold segmentation method used can obtain clearer binary images under different lighting conditions, and has strong adaptability; the average value method is used to calculate the center of the positioning block Point coordinates, strong resistance to strong light interference. As shown in Figure 6, the effect comparison of different degrees of strong light interference at the same position, in which (a), (b), and (c) the intensity of light received by the three increases in sequence.

2. The clustering algorithm has a strong ability to filter out the black stains contaminated by beacons, and can automatically eliminate this interference. Although changes in the lighting environment will lead to changes in image grayscale processing, image binary segmentation, and noise-filtered black and white images, the use of image gathering point algorithms can effectively minimize this external influence, and the calculated reference object center The point coordinates fluctuate within the allowable range, and the final result will not fluctuate greatly due to changes in the lighting environment.

Description of drawings

FIG. 1 is a flowchart of a method according to an embodiment of the present invention.

Fig. 2 is a relationship model among the absolute coordinate system, the image coordinate system and the pixel coordinate system.

Figure 3 is a binary image obtained under different lighting conditions using dynamic threshold segmentation. Where (a) is normal light conditions, and (b) is low light conditions.

Figure 4 shows the effect of using a grid to segment an image.

Figure 5 shows the location block extracted by clustering method.

Figure 6 shows the effect when disturbed by strong light. (a), (b), (c) received gradually increased light.

Figure 7 is a flow chart of clustering.

Detailed ways

The present invention will be further described below in conjunction with specific examples and accompanying drawings.

The present invention provides a kind of monocular visual deviation detection method for AGV, as shown in Figure 1, it comprises the following steps:

S0. Start the camera and take a frame of pictures. The image data is obtained by photographing the beacon, which contains three positioning blocks, each positioning block is a figure with at least one set of vertical symmetry axes, such as a circle, a square, a rectangle or a rhombus, and three The center point of the positioning block can constitute a Cartesian coordinate system.

S1. Perform grayscale processing on the image data collected by the image sensor. The weighted average grayscale processing algorithm based on R, G, and B components is used to grayscale the image to enlarge the difference between the foreground color and the background color. Specifically include:

g(x, y) = af _R (x, y) + bf _G (x, y) + cf _B (x, y)

The constraints are:

a, b, c are all positive integers.

f _R (x, y), f _G (x, y), and f _B (x, y) are the R, G, and B components of the pixel f (x, y).

f(x, y) is the color value of the pixel point (x, y) in the image coordinate system.

g(x, y) is the grayscale processed data of the pixel point (x, y).

S2. By sampling the data in S1, calculate the dynamic segmentation threshold T with the sample image data, perform threshold segmentation on the grayscale data in S1, and obtain a binary image. The result is shown in Figure 3, where (a) is the integer An image is the threshold segmentation result of the data source, (b) is the threshold segmentation result of the sample image data as the data source. Specifically include:

The sampling principle is:

1. Calculate the gray level difference between the first pixel in each row of pixels and the rest of the pixels row by row. If the difference is greater than the set value, it is considered that this row may contain both foreground and background colors; otherwise, it will be detected one line.

2. For a line that may contain foreground color and background color, detect the number of foreground color pixels in the line, if the number of foreground color is greater than the set value, then the required sample for this line; otherwise, detect the next line.

The method to calculate the dynamic threshold is:

各像素值出现的概率P_i满足公式：

选取一个整数K将像素分为两组，C₀＝{1，2，……，k}，C₁＝{k+1，k+2，……，m}，则两组之间的方差公式：σ²(k)＝ω₀ω₁(μ₁-μ₀)² Assuming that the sample pixels used are divided into 1, 2, ..., m levels, and the number of pixels with gray value i is n, the total number of pixels N can be expressed by the formula:

The probability P _i of the appearance of each pixel value satisfies the formula:

Choose an integer K to divide the pixels into two groups, C ₀ ={1, 2,...,k}, C ₁ ={k+1,k+2,...,m}, then the variance between the two groups Formula: σ ² (k) = ω ₀ ω ₁ (μ ₁ -μ ₀ ) ²

μ₀为C₀的均值，

ω₁为C₁发生的概率，

μ₁为C₁的均值，

In the formula: ω ₀ is the probability of C ₀ occurring,

μ ₀ is the mean value of C ₀ ,

ω ₁ is the probability of C ₁ occurring,

μ ₁ is the mean value of C ₁ ,

Change K from (1, 2, ..., m) to find K when the variance is maximized, and get the K value when maxσ ² (k) is the optimal threshold T.

After the dynamic segmentation threshold T is calculated, the grayscale image can be binary segmented to obtain a binary image.

The method to obtain a binary image is:

In the formula: 1 represents white. 0 represents black. g(x, y) is the gray value of the pixel in column x and row y. h(x, y) is the pixel-divided value of column x and row y.

S3. Filter the binary image data obtained in S2, perform morphological filtering on objects that appear in the background color and may be foreground noise, and perform threshold segmentation again on the filtered pixels. The noise in the foreground color area does not affect the image clustering, and the extraction of the coordinates of the center point of the positioning block is very small, and does not affect the accuracy of the result.

S4. Segment the image obtained in S3 by using a grid, and the segmentation effect is shown in FIG. 4 . Traversing all grid cells, the grid cells containing the foreground color form a set S. The size of the grid unit should ensure that in any case, there is at least one grid unit containing only the background color between any two positioning blocks. The binary image acquired by S2 contains foreground noise and target foreground color, S3 removes the foreground noise, leaving the target foreground color, and S4 segments the target foreground color.

S5. Cluster the elements in the set S, and separate the positioning blocks. The clustering process is shown in Figure 7, 5.1, traverse all grid units, use the grid coordinates containing the positioning block image as the elements of the set S, and the number of positioning blocks N=0; 5.2, N=N+1, establish positioning block set A _N ; move the first element in the set S into the set A _N , and move this element out of the set; 5.3. Compare the grid positions represented by the remaining elements in the set S and all the elements in the set A _N in sequence Relationship; judge whether there are two adjacent grids, if so, move the element corresponding to the grid in the set S into the set A _N , and move this element out of the set S; 5.4, get the final positioning block set A _N . The clustering results are shown in Figure 5.

S6. Calculating the coordinates of the center point of the positioning block separated in S5. The coordinates of the center point of the positioning block satisfy:

For any figure with at least one set of mutually perpendicular symmetry axes, the coordinates of its center point satisfy:

In the formula: x _t is the abscissa of the center point of the t th positioning block, y _t is the ordinate of the center point of the t th positioning block, n is the total number of black pixels in the t th positioning block, x _i is The abscissa of the i-th black pixel in the t-th positioning block, and y _i is the ordinate of the i-th black pixel in the t-th positioning block.

S7. Establish a spatial conversion relationship model between the image coordinate system and the pixel coordinate system according to the coordinates of the center point of the positioning block, and calculate the deviation, as shown in FIG. 2 .

Specific steps include:

1. Determine the relative positional relationship between the three center points. By calculating the pixel distance between the three center points, the two end points of the farthest distance are positioning block B and positioning block C (at this time, positioning block B and positioning block C cannot be judged), and the remaining one is positioning block A.

2. Determine the positions of positioning block B and positioning block C. Take any one of the two endpoints with the farthest distance in 1, record it as B', and form a vector with positioning block A starting from the center point of positioning block A and pointing to B', and judge the center point of the third positioning block The positional relationship with this vector. If it is on the left, then B' corresponds to the positioning block B, otherwise, B' corresponds to the positioning block C.

3. Establish a pixel coordinate system. In 2, the vector from the center point of positioning block A to the center point of positioning block B coincides with the X-axis of the pixel coordinate system and points to the positive half axis of the X-axis; the vector from the center point of positioning block A to the center point of positioning block B Coincident with the Y-axis of the pixel coordinate system and point to the positive half-axis of Y.

4. Calculate the deviation.

1. Determine which quadrant of the pixel coordinate system the center point of the image is in.

The formula is:

t ₁ <0, t ₂ <0, located in the first quadrant, dx>0, dy>0.

t ₁ >0, t ₂ <0, located in the second quadrant, dx<0, dy>0.

t ₁ >0, t ₂ >0, located in the third quadrant, dx<0, dy<0.

t ₁ <0, t ₂ >0, located in the fourth quadrant, dx>0, dy<0.

The formula for calculating the distance dy between the center of the image sensor and the beacon reference point is:

The formula for calculating the distance dx between the center of the image sensor and the beacon reference point is:

The formula for calculating the angle between the image coordinate system and the pixel coordinate system in the Y-axis direction is:

The formula for calculating the included angle between the image coordinate system and the pixel coordinate system in the X-axis direction is:

The angle θy or θx is the angle between the coordinate system X ₃ O ₃ Y ₃ and the corresponding x-coordinate axis or y-coordinate axis of the coordinate system X ₂ O ₂ Y ₂ , clockwise rotation is negative, counterclockwise rotation is positive, and finally The result is in the interval [-180°—180°).

Where: f is the focal length of the camera, and h is the installation height.

The present invention also provides a monocular visual deviation detection device for AGV, including an image acquisition device, a memory and a data processor arranged on the AGV; wherein, the image acquisition device is used to collect image data, and a computer program is stored in the memory , to be called by the data processor to complete the above-mentioned monocular vision deviation detection method for AGV. The hardware system includes a power supply circuit, a camera, an RS232 module, a CAN bus transceiver, an embedded microcontroller, a static random access memory, a clock circuit, a 5V to 3.3V circuit, a PCB base board and a PCB core board; PCB base board Connect and couple with the PCB core board through the female terminal and the male terminal; the power circuit is welded on the front of the PCB base plate, and the 12V DC power is stepped down to 5V and 3.3V DC power; the camera is installed on the back of the PCB base plate, through a The double-row female pin seat is connected to the circuit of the PCB bottom board and the center of the COMS image sensor of the camera coincides with the center of the PCB bottom board; the RS232 module is installed on the front of the PCB bottom board, and its VCC pin and GND pin are connected to the power supply circuit through the PCB bottom board VCC3.3 and GND, TX pin and RX pin are connected to the corresponding pins of the female terminal through the PCB backplane; the CAN bus transceiver module is installed on the front of the PCB backplane, and its VCC pin and GND pin are connected to the The VCC3.3 and GND of the power circuit, the CAN TX pin and the CAN RX pin are connected to the corresponding pins of the female terminal through the PCB bottom plate; the embedded microcontroller is welded on the front of the PCB core board for grayscale processing of images, Image binary segmentation, noise filtering, image gathering points, calculating the coordinates of the center of the reference object, calculating the distance and angle between the X-axis and the Y-axis with the target reference object; SRAM is used to store image information.

The method of the invention can automatically set the segmentation threshold according to the lighting conditions of the working environment where the camera is located, automatically turn on the LED light to supplement the light under weak lighting conditions, and can accurately extract the coordinates of the target point under strong lighting environments, and has strong adaptability to the environment. The detection accuracy meets the requirements of use, the calculation speed is fast, and the real-time performance is good.

The above embodiments are only used to illustrate the design concept and characteristics of the present invention, and its purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.

Claims (6)

1. A monocular vision deviation detection method for an AGV is characterized in that: it comprises the following steps:

s1, carrying out gray processing on acquired image data to obtain gray data; the image data is obtained by shooting a beacon, the beacon comprises three positioning blocks, each positioning block is a graph at least provided with a group of vertical symmetry axes, and the central points of the three positioning blocks can form a Cartesian coordinate system;

s2, sampling the acquired image data to obtain sample image data, calculating a dynamic segmentation threshold T according to the sample image data, and performing threshold segmentation on gray data to obtain a binary image;

s3, screening the binary image, performing morphological filtering on an object which may be foreground noise, and performing threshold segmentation on the filtered pixel points again; the object which is probably foreground noise refers to a pixel point which meets a certain pixel value range;

s4, segmenting the image obtained in the S3 by using the grid, traversing all grid units, and forming a set S by the grid units containing the foreground;

s5, clustering the elements in the set S, and separating positioning blocks;

s6, solving the coordinates of the central point of the positioning block by adopting an average value method;

s7, establishing a space conversion relation model between an image coordinate system and a pixel coordinate system according to the central point coordinates of the positioning blocks, and calculating deviation;

s5, clustering is carried out according to the following method:

5.1, setting the number of bit blocks N =0 for the set S obtained by S4;

5.2, N = N +1, establishing a positioning block set A _N (ii) a Moving the first element in set S into set A _N While moving the element out of the set;

5.3, comparing the remaining elements in the set S with the set A in sequence _N The grid position relationship represented by all the elements in (1); judging whether two grids are adjacent or not, if so, moving the elements corresponding to the grids in the set S into the set A _N Simultaneously moving this element out of set S;

5.4, obtaining a final positioning block set A _N ；

The S7 specifically includes:

establishing a pixel coordinate system by judging the relative position relation between the coordinates of the central points of the positioning blocks;

calculating the quadrant of the pixel coordinate system in which the central point of the image is positioned;

calculating the distance between the center point of the image and the X axis and the Y axis of the pixel coordinate;

correcting the positive and negative relation of the deviation value according to the quadrant of the central point of the image;

and calculating an included angle between the coordinate axes corresponding to the two coordinate systems according to the coordinate axis rotation relationship between the pixel coordinate system and the image coordinate system.

2. The monocular visual deviation detecting method of claim 1, wherein: specifically, in S1, the gray processing is performed on each pixel row by row and column by column, and the gray processing method includes:

g(x，y)＝af _R (x，y)+bf _G (x，y)+cf _B (x，y)

the constraint conditions are as follows: a, b and c are positive integers;

f _R (x，y)、f _G (x，y)、f _B the (x, y) is the R, G, B component of the pixel f (x, y), and the G (x, y) is the data after the gradation processing.

3. The monocular visual deviation detecting method of claim 1, wherein: the S2 specifically comprises:

2.1, calculating a dynamic segmentation threshold T by using a maximum inter-variance method according to the data obtained in the step S1, wherein the calculation method comprises the following steps:

assume that the sample pixel employed is divided into 1,2, \8230;, m-level, gray-value i with the number of pixels n _i Then, the total pixel number M can be formulated as:

probability P of occurrence of each pixel value _i Satisfies the formula:

an integer k is selected to divide the pixels into two groups, C ₀ ＝{1，2，……，k}，C ₁ = k +1, k +2, \8230 \ 8230 \, m }, then the variance formula between the two groups: sigma ² (k)＝ω ₀ ω ₁ (μ ₁ -μ ₀ ) ²

In the formula: omega ₀ Is C ₀ The probability of the occurrence of the event is,

μ ₀ is C ₀ The average value of (a) of (b),

ω ₁ is C ₁ The probability of occurrence of the event is determined,

μ ₁ is C ₁ The average value of (a) is calculated,

by changing k from (1, 2, \8230;, m), k at which the variance is maximized is obtained, and max σ is obtained ² (k) The k value is the dynamic segmentation threshold value T;

2.2, acquiring a binary image according to the data obtained in the step S1 and the dynamic segmentation threshold T obtained in the step 2.1 according to the following method:

in the formula: 1 represents white, 0 represents black, g (x, y) is the gray scale value of the pixel in the x-th column and the y-th row, and h (x, y) is the value obtained by dividing the pixel in the x-th column and the y-th row.

4. The monocular visual deviation detecting method of claim 1, wherein: the S3 specifically comprises:

for the following requirements:

the pixel point of (2) uses a morphological filtering method:

in the formula: med represents a middle value obtained by sequencing all pixel gray values in a filtering window A in an ascending order, wherein A is the filtering window;

then re-thresholding:

5. the monocular vision deviation detecting method for an AGV of claim 1, characterized in that: s6, the coordinates of the central point are obtained by using the following method:

for a graph with two perpendicular symmetry axes, calculating a graph center point by using an average value algorithm, wherein the graph center point coordinates satisfy the formula:

in the formula: x is the number of _t Is the abscissa of the center point of the t-th positioning block, y _t Is the longitudinal coordinate of the central point of the tth positioning block, is the total number of black pixel points in the tth positioning block, x _i Is the abscissa, y, of the ith black pixel in the tth positioning block _i Is the ordinate of the ith black pixel point in the tth positioning block.

6. The utility model provides a monocular vision deviation detection device for AGV which characterized in that: the device comprises an image acquisition device, a memory and a data processor which are arranged on the AGV; the image acquisition device is used for acquiring image data, and a computer program is stored in a memory and is called by the data processor to complete the monocular vision deviation detecting method for the AGV according to any one of claims 1 to 5.

Images