CN110147748A - A kind of mobile robot obstacle recognition method based on road-edge detection - Google Patents

Abstract

The present invention relates to artificial intelligence field, specifically a kind of mobile robot obstacle recognition method based on road-edge detection, the specific steps of which are as follows: S1: obtaining boundary line；S2: edge detection；S3: target detection；S4: coordinate value；S5: road barrier domain of the existence calculates；S6: road barrier domain of the existence judgement；There is object in front of the mobile robot that can only be detected compared to traditional obstacle recognition, and for having intersection that can not accurately identify, barrier domain of the existence Ω is found using road-edge detection algorithm, target detection is carried out to the object in image using deep learning network frame, through the invention can be more accurate identify the barrier detected in target in the realtime graphic, intelligence degree is high.

Description

An obstacle recognition method for mobile robots based on road edge detection

technical field

The invention relates to the field of artificial intelligence, in particular to a mobile robot obstacle identification method based on road edge detection.

Background technique

Mobile robot technology and industry have been developing rapidly in recent years, and mobile robot application scenarios have become more and more extensive. Its research and application have also expanded from military and industrial fields to agriculture, household, service and security industries. In the field of mobile robot research, obstacle detection is an important direction. When the mobile robot runs on the road, there will be obstacles in the direction of the road. Obstacles will hinder the advancement of the mobile robot, collide with the mobile robot, etc., causing damage to the mobile robot or damage to the obstacles. The mobile robot can only reach its destination by continuously avoiding obstacles. Therefore, it is very necessary to complete the robot obstacle detection when the mobile robot is traveling.

At present, there are robot obstacle detection methods based on lidar information. Although lidar has good adaptability to the environment, lidar can only detect objects and cannot identify objects, so it is impossible to judge the behavior and type of objects, and multi-threaded lidar Prices are higher. For the detection of obstacles, ultrasonic and millimeter-wave radars can only detect the distance of obstacles.

SUMMARY OF THE INVENTION

In order to solve the above problems, the present invention proposes an obstacle identification method for a mobile robot based on road edge detection.

A mobile robot obstacle recognition method based on road edge detection, the specific steps are as follows:

S1: Obtain boundary line: the image acquisition device installed on the mobile robot obtains a real-time image, including the upper boundary line l _up of the real-time image and the lower boundary line l _down of the real-time image;

S2: edge detection: obtain the road edge line l _road through the edge detection algorithm, and then determine the equation y _l =F _l (x) of the road edge line l _road ;

S3: target detection: use the deep learning network framework to perform target detection on real-time images, and obtain the bounding box of each detection target and its characterization function: f=(x,y,w,h,c);

S4: Coordinate value: Using the characterization function f, the coordinate values of the lower left corner and the lower right corner of each detection target bounding box can be obtained as (x _L , y _L ) and (x _R , y _R ) respectively;

S5: road obstacle existence area calculation: calculate the road edge line l _road , the real-time image upper boundary line l _up and the real-time image lower boundary line l _down to enclose the road obstacle existence area Ω;

S6: Judgment of road obstacle existence area: judge whether the coordinate values (x _L , y _L ) and (x _R , y _R ) of the lower left corner and lower right corner of each detection target bounding box are included in the road obstacle existence area Ω And complete obstacle identification.

The equation y _l =F _l (x _l ) of the road edge line l _road in step 1, wherein F _l is a piecewise function:

The units of y _l and x _l are both px, and the units of x, y, w, and h in the characterization function f=(x, y, w, h, c) are all px, and the px is a pixel unit, the x is the abscissa value of the upper left corner of the detection target bounding box, the y is the ordinate value of the upper left corner of the detection target bounding box, and the w is the The horizontal width of the detection target bounding box, the h is the vertical height of the detection target bounding box, and the c is the recognition rate of the detection target bounding box.

The size of the real-time image in the step 1 is a*b, the units of a and b are px, the a is the length of the real-time image, and the b is the real-time image. , the value range of x in the road edge equation F=f(x, y) is 0≤x≤a, 0≤x≤b.

The edge detection algorithm of the step 2 satisfies that the complete detection of the road edge can be realized, the detection time of the edge detection algorithm is less than the braking time of the mobile robot, and the target detection time of the real-time image is less than the braking time of the mobile robot. .

Each detection target bounding box representation function f=(x, y, w, h, c) in the step 4 and (x _L , y _L ) and (x _R , y _R ) on each detection target bounding box satisfy The following relationship:

and

The road obstacle existing area Ω in the step 5 is the two-dimensional point set in the real-time image, which satisfies the following conditions:

Ω={(x,y)q ₁ ≤x<q ₂ , q ₃ ≤x<q ₄ ,...,q _n-1 ≤x<q _n ,q _n ≤a; 0≤y≤b}.

In the step 6, if the (x _L , y _L ) and the (x _R , y _R ) are both included in the road obstacle existing area Ω, then it is determined that the (x _L , y R ) y _L ) and the target object selected by the detection target bounding box to which the (x _R , y _R ) belongs is an obstacle;

In the step 6, if the (x _L , y _L ) is included in the road obstacle existing area Ω, the (x _R , y _R ) is not included in the road obstacle existing area Ω, then it is also determined that the target object selected by the detection target bounding box to which the (x _L , y _L ) and the (x _R , y _R ) belong are obstacles;

In the step 6, if the (x _R , y _R ) is included in the road obstacle existing area Ω, the (x _L , y _L ) is not included in the road obstacle existing area Ω, then it is also determined that the target object selected by the detection target bounding box to which the (x _L , y _L ) and the (x _R , y _R ) belong are obstacles;

In the step 6, if both the (x _L , y _L ) and the (x _R , y _R ) are not included in the road obstacle existing area Ω, then it is determined that the (x _L , y R ) , y _L ) and the target object selected by the detection target bounding box to which the (x _R , y _R ) belongs is not an obstacle.

The beneficial effects of the present invention are: compared with the traditional obstacle recognition that can only detect objects in front of the mobile robot, and there are overlapping parts that cannot be accurately recognized, the road edge detection algorithm is used to find the obstacle existing area Ω, and the depth is used to find the obstacle. The learning network framework performs target detection on the objects in the image, and the present invention can more accurately identify the obstacles in the detected target in the real-time image, and has a high degree of intelligence.

Description of drawings

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

Fig. 1 is the real-time image schematic diagram of the present invention;

2 is a schematic diagram of a target detection bounding box of the present invention;

3 is a schematic diagram of real-time image target detection and road edge detection according to the present invention;

FIG. 4 is a schematic diagram of the distribution of the road obstacle existing area Ω in the real-time image according to the present invention.

Detailed ways

In order to make it easy to understand the technical means, creative features, achieved goals and effects of the present invention, the present invention is further described below.

As shown in Figure 1 to Figure 4, an obstacle recognition method for mobile robots based on road edge detection. Real-time image, select the trained deep learning neural network framework as the image software detection tool, run under the VS2015 environment, the real-time image includes the real-time image upper boundary line l _up and the real-time image lower boundary line l _down , and the coordinate origin is 0 ( 0,0) at the top left corner of the real-time image, the specific steps are as follows:

S1: Obtain the boundary line: the image acquisition device installed on the mobile robot, use the image acquisition device to obtain the real-time image of the scene in front of the mobile robot's movement, and the real-time image includes the real-time image upper boundary line l _up and the real-time image lower side Boundary l _down :

S2: edge detection: detect the road edge in the real-time image through the edge detection algorithm in the real-time image described in step 1, obtain the road edge line l _road , and then determine the equation of the road edge line l _road y _l =F _l (x );

S3: Target detection: As shown in Figure 2, use the deep learning network framework to perform target detection on the real-time image described in step 1, and obtain the bounding box of each detection target and its characterization function: f=(x, y, w, h , c), in this embodiment, the four values of x, y, w, and h in the characterization function of each detection target are, from left to right,

S4: Coordinate values: Using the characterization function f described in step 3, the coordinate values of the lower left corner and the lower right corner of each detection target bounding box can be obtained as (x _L , y _L ) and (x _R , y _R ), respectively, as shown in The coordinate values of the lower left corner and the lower right corner of each detection target bounding box shown in Figure 3 are from left to right:

S5: Calculation of road obstacle existence area: Calculate the road edge line l _road described in step 2, the real-time image upper boundary line l _up and the real-time image lower boundary line l _down to enclose the road obstacle existence area Ω, as shown in the figure 4 shows that l _up and l _down in this embodiment satisfy the following equations:

Then the road obstacle existing area Ω in this embodiment satisfies the following equation

S6: Judging the existence area of road obstacles: judging whether the coordinate values (x _L , y _L ) and (x _R , y _R ) of the lower left corner and the lower right corner of each detection target bounding box described in step 4 are included in the Road obstacles exist in the area Ω and complete obstacle identification.

Determine whether the coordinate values (x _L , y _L ) and (x _R , y _R ) of the lower left corner and the lower right corner of each detection target bounding box are included in the road obstacle existence area, specifically:

a: For the detection target tree of No. 1, the coordinate values (135, 450) and (230, 450) of the lower left corner and the lower right corner of the bounding box are brought into the area equation of Ω described in step 5, and it is judged that neither of the two coordinates is in the area where road obstacles exist , then it is determined that the target tree detected by No. 1 is not an obstacle;

b: For the detection target tree of No. 2, the coordinate values (360, 199) and (430, 199) of the lower left corner and lower right corner of the bounding box are brought into the area equation of Ω described in step 5, and it is judged that neither of the two coordinates exists in the road obstacle area, then it is determined that the target tree detected by No. 2 is not an obstacle;

c: For the detection target Person No. 3, the coordinate values (390, 480) and (470, 480) of the lower left corner and lower right corner of the bounding box are brought into the area equation of Ω in step 5, and it is judged that both coordinates are in the area where road obstacles exist , then the number 3 detection target Person in the judgment is an obstacle;

d: For the detection target bicycle of No. 4, the coordinate values (512, 290) and (605, 290) of the lower left corner and the lower right corner of the bounding box are brought into the area equation of Ω described in step 5, and it is judged that both coordinates are in the existence of road obstacles area, the number 4 detection target bicycle in the judgment is an obstacle;

e: For the detection target bus of No. 5, the coordinate values (590, 180) and (670, 180) of the lower left corner and the lower right corner of the bounding box are brought into the area equation of Ω described in step 5, and it is judged that both coordinates are in the existence of road obstacles. area, then the number 5 detection target bus in the judgment is an obstacle;

f: For the detection target Person No. 6, the coordinate values (610, 435) and (685, 435) of the lower left corner and the lower right corner of the bounding box are brought into the area equation of Ω in step 5, and it is judged that both coordinates are in the area where road obstacles exist , then it is determined that the detection target Person of No. 6 is an obstacle.

The equation y _l =F _l (x _l ) of the road edge line l _road in step 1, wherein F _l is a piecewise function:

The units of y _l and x _l are both px, and the units of x, y, w, and h in the characterization function f=(x, y, w, h, c) are all px, and the px is a pixel unit, the x is the abscissa value of the upper left corner of the detection target bounding box, the y is the ordinate value of the upper left corner of the detection target bounding box, and the w is the The horizontal width of the detection target bounding box, the h is the vertical height of the detection target bounding box, and the c is the recognition rate of the detection target bounding box.

The road edge line l _road may be multiple, and the road edge line l _road may be either a straight line or a curved line.

Compared with the traditional obstacle recognition that can only detect objects in front of the mobile robot, and there are overlapping parts, it cannot be accurately recognized. The road edge detection algorithm is used to find the obstacle existing area Ω, and the deep learning network framework is used to detect the objects in the image. The object is detected by the target, and the present invention can more accurately identify the obstacles in the detected target in the real-time image, and the degree of intelligence is high.

The mobile robot in step 1 is provided with an image processing platform, and the image processing platform includes a hardware part and a software part; the image acquisition device is a calibrated monocular camera, and the coordinate values in the image are all The coordinate value in pixel px, the origin of the coordinate is the upper left corner of the real-time image.

The size of the real-time image is a*b, the units of a and b are px, the a is the length of the real-time image, and the b is the height of the real-time image, The value range of x in the road edge equation F=f(x, y) is 0≤x≤a, 0≤x≤b.

The edge detection algorithm can achieve complete detection of road edges, the detection time of the edge detection algorithm is shorter than the braking time of the mobile robot, and the target detection time of the real-time image is shorter than the braking time of the mobile robot.

Each detection target bounding box representation function f=(x, y, w, h, c) in the step 4 and (x _L , y _L ) and (x _R , y _R ) on each detection target bounding box satisfy The following relationship:

and

The road obstacle existing area Ω in the step 5 is the two-dimensional point set in the real-time image, which satisfies the following conditions:

Ω={(x,y)q ₁ ≤x<q ₂ , q ₃ ≤x<q ₄ ,...,q _n-1 ≤x<q _n ,q _n ≤a; 0≤y≤b}.

The real-time image detected by the edge detection algorithm and the real-time image using the deep learning network framework for target detection are the same frame image or the real-time image using the deep learning network framework for target detection. The real-time image detected by the edge detection algorithm.

In the step 6, if the (x _L , y _L ) and the (x _R , y _R ) are both included in the road obstacle existing area Ω, then it is determined that the (x _L , y R ) y _L ) and the target object selected by the detection target bounding box to which the (x _R , y _R ) belongs is an obstacle;

In the step 6, if the (x _L , y _L ) is included in the road obstacle existing area Ω, the (x _R , y _R ) is not included in the road obstacle existing area Ω, then it is also determined that the target object selected by the detection target bounding box to which the (x _L , y _L ) and the (x _R , y _R ) belong are obstacles;

In the step 6, if the (x _R , y _R ) is included in the road obstacle existing area Ω, the (x _L , y _L ) is not included in the road obstacle existing area Ω, then it is also determined that the target object selected by the detection target bounding box to which the (x _L , y _L ) and the (x _R , y _R ) belong are obstacles;

In the step 6, if both the (x _L , y _L ) and the (x _R , y _R ) are not included in the road obstacle existing area Ω, then it is determined that the (x _L , y R ) , y _L ) and the target object selected by the detection target bounding box to which the (x _R , y _R ) belongs is not an obstacle.

The foregoing has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above-mentioned embodiments, and the above-mentioned embodiments and descriptions describe only the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will also have various Such changes and improvements fall within the scope of the claimed invention. The claimed scope of the present invention is defined by the appended claims and their equivalents.

Claims (10)

1. a kind of mobile robot obstacle recognition method based on road-edge detection, it is characterised in that: its specific steps is such as Under:

S1: obtain boundary line: the image capture device installed in mobile robot obtains realtime graphic, includes realtime graphic top Boundary line l_upWith the following boundary line l of realtime graphic_down；

S2: road edge line l edge detection: is obtained by edge detection algorithm_road, and then determine road edge line l_roadSide Journey y_l=F_l(x)；

S3: target detection: target detection is carried out using deep learning network frame realtime graphic, obtains each detection object boundary frame And its characterization function: f=(x, y, w, h, c)；

S4: coordinate value: it is respectively using the coordinate value that characterization function f can acquire each detection object boundary frame lower left corner and the lower right corner (x_L,y_L) and (x_R,y_R)；

S5: road barrier domain of the existence calculates: calculating road edge line l_roadWith the realtime graphic upper border line l_upWith The following boundary line l of realtime graphic_downSurround road barrier domain of the existence Ω；

S6: the judgement of road barrier domain of the existence: judge the coordinate value (x in each detection the object boundary frame lower left corner and the lower right corner_L, y_L) and (x_R,y_R) whether be contained in the road barrier domain of the existence Ω and complete obstacle recognition.

2. a kind of mobile robot obstacle recognition method based on road-edge detection according to claim 1, special Sign is: the road edge line l of the step 1_roadEquation y_l=F_l(x_l), wherein F_lFor piecewise function:

Wherein, y_lAnd x_lUnit be px, the x in the characterization function f=(x, y, w, h, c), the unit of y, w, h is Px, the px are pixel unit, and the x is the upper left corner abscissa value of the detection object boundary frame, and the y is The upper left corner ordinate value of the detection object boundary frame, the w are the transverse widths of the detection object boundary frame, The h is the vertical height of the detection object boundary frame, and the c is the identification of the detection object boundary frame Rate.

3. a kind of mobile robot obstacle recognition method based on road-edge detection according to claim 1, special Sign is: the size of the realtime graphic of the step 1 is that the unit of a*b, a and b are px, and a is institute The length for the realtime graphic stated, the b are the height of the realtime graphic, the road edge equation F=f (x, y) The value range of middle x is 0≤x≤a, 0≤x≤b.

4. a kind of mobile robot obstacle recognition method based on road-edge detection according to claim 1, special Sign is: the edge detection algorithm satisfaction of the step 2 is able to achieve road edge and completely detects, the edge detection algorithm Detection time be less than the braking time of mobile robot, target detection time of the realtime graphic is less than mobile robot Braking time.

5. a kind of mobile robot obstacle recognition method based on road-edge detection according to claim 1, special Sign is: each detection object boundary frame characterization function f=(x, y, w, h, c) of the step 4 and each detection object boundary frame On (x_L,y_L) and (x_R,y_R) meet following relationship:

With

6. a kind of mobile robot obstacle recognition method based on road-edge detection according to claim 1, special Sign is: the road barrier domain of the existence Ω of the step 5 is the two-dimentional point set in the realtime graphic, is met as follows Condition:

Ω=(x, y) | q₁≤ x < q₂,q₃≤ x < q₄,…,q_n-1≤ x < q_n,q_n≤a；0≤y≤b}.

7. a kind of mobile robot obstacle recognition method based on road-edge detection according to claim 1, special Sign is: if (x described in the step 6_L,y_L) and (x_R,y_R) it is both contained in the road barrier presence Region Ω then determines the (x_L,y_L) and (x_R,y_R) belonging to the mesh chosen of the detection object boundary frame Mark object is barrier.

8. a kind of mobile robot obstacle recognition method based on road-edge detection according to claim 1, special Sign is: if (x described in the step 6_L,y_L) it is contained in road barrier the domain of the existence the Ω, (x_R, y_R) be not included in the road barrier domain of the existence Ω, then also determine the (x_L,y_L) and (x_R,y_R) belonging to In the target object chosen of the detection object boundary frame be barrier.

9. a kind of mobile robot obstacle recognition method based on road-edge detection according to claim 1, special Sign is: if (x described in the step 6_R,y_R) it is contained in road barrier the domain of the existence the Ω, (x_L, y_L) be not included in the road barrier domain of the existence Ω, then also determine the (x_L,y_L) and (x_R,y_R) belonging to In the target object chosen of the detection object boundary frame be barrier.

10. a kind of mobile robot obstacle recognition method based on road-edge detection according to claim 1, special Sign is: if (x described in the step 6_L,y_L) and (x_R,y_R) be not included in the road barrier and deposit In region Ω, then (the x is determined_L,y_L) and (x_R,y_R) belonging to the detection object boundary frame choose Target object is not barrier.