WO2019119752A1

WO2019119752A1 - Obstacle recognition method and terminal

Info

Publication number: WO2019119752A1
Application number: PCT/CN2018/091638
Authority: WO
Inventors: 曲磊
Original assignee: 海信集团有限公司
Priority date: 2017-12-21
Filing date: 2018-06-15
Publication date: 2019-06-27
Also published as: CN107977649B; CN107977649A

Abstract

An obstacle recognition method, device, and terminal, relating to the technical field of driver assistance. The method comprises: extracting an inclined line in a V disparity map of an image on which recognition is to be performed (S201); determining a first starting pixel on the inclined line (S202), wherein the sum of the values of pixels in a pixel set corresponding to the first starting pixel is greater than a first threshold, and the pixel set corresponding to the first starting pixel comprises a pixel located on the inclined line and in a column where the first starting pixel is located; determining, in the V disparity map, a detection window corresponding to the first starting pixel according to the disparity of the first starting pixel and a window parameter corresponding to a preset obstacle type (S203); and determining, according to the recognition degree of the image in the detection window, the type of an obstacle identified in the image on which recognition is to be performed (S204). The method is used for improving the obstacle recognition accuracy.

Description

Obstacle recognition method and terminal

Technical field

The embodiments of the present invention relate to the field of assisted driving technologies, and in particular, to an obstacle recognition method and a terminal.

Background technique

In the image recognition technology, high-resolution image information and depth information of an image can be acquired by binocular stereo vision technology. Therefore, binocular stereo vision technology is widely used in image recognition.

In the obstacle recognition process based on binocular stereo vision, the binocular stereo vision technology is usually used to acquire the disparity map of the image to be recognized, and the obstacle recognition is performed according to the lines in the disparity map. However, this conventional obstacle recognition method may recognize the same obstacle as a plurality of obstacles when performing obstacle recognition based on the lines in the parallax map, resulting in a lower accuracy of obstacle recognition.

Summary of the invention

In order to solve the problem that the same obstacle is identified as a plurality of obstacles when the obstacle is recognized due to the discontinuity of the envelope line in the parallax map, the embodiment of the present invention provides an obstacle recognition method and terminal. Improves the accuracy of obstacle recognition.

In a first aspect, an embodiment of the present invention provides an obstacle recognition method, including:

Extracting a diagonal line in the V disparity map of the image to be identified;

Determining, on the oblique line, a first start pixel, where a sum of values of pixels in the set of pixels corresponding to the first start pixel is greater than a first threshold, and a set of pixels corresponding to the first start pixel includes the first start pixel a pixel in the column above the slash;

Determining, in the V disparity map, a detection window corresponding to the first start pixel according to a parallax of the first start pixel and a window parameter corresponding to the preset obstacle type;

Determining an obstacle type identified in the image to be identified according to the recognition degree of the image in the detection window.

In an embodiment, determining the first starting pixel on the oblique line comprises:

Determining, from the one end of the oblique line, the pixels on the oblique line as the first pixel along the extending direction of the oblique line, and acquiring the value of the pixel in the pixel set corresponding to the first pixel And determining, when the sum of the values of the pixels in the pixel set corresponding to the first pixel obtained on the oblique line is greater than the first threshold, determining the first pixel as the first start pixel;

The pixel set corresponding to the first pixel includes a pixel located above the oblique line in the column of the first pixel.

In another embodiment, the preset obstacle type includes a first obstacle type, the first obstacle type corresponds to a first window parameter; according to a disparity of the first starting pixel and the first window a parameter, in the V disparity map, determining a first detection window corresponding to the first start pixel, including:

Obtaining a first window parameter corresponding to the first obstacle type;

Determining a size of the first detection window according to an obstacle recognition direction, a parallax of the first start pixel, the first window parameter, and a camera parameter that captures the image to be recognized;

Determining, in the V disparity map, the first detection window according to a size of the first detection window, where the first starting pixel is a pixel of a corner of the first detection window.

In another embodiment, determining the first detection window in the V disparity map according to the size of the first detection window comprises:

Determining a position of the first start pixel in the first detection window according to the obstacle recognition direction;

The first detection window is determined in the V disparity map according to a position of the first start pixel in the first detection window and a size of the first detection window.

In another embodiment, determining the type of the obstacle identified in the image to be identified according to the recognition degree of the image in the detection window includes:

Detecting an image in the detection window according to a preset obstacle type corresponding to the detection window, to obtain an image recognition degree in the detection window;

Determining a target detection window according to the recognition degree of the image in the detection window, wherein the recognition degree of the image in the target detection window is the highest;

The obstacle type corresponding to the target window is determined as the obstacle type identified in the image to be identified.

In another embodiment, the detection window includes a second detection window, the second detection window corresponding to the second obstacle type; and the image in the second detection window according to the second obstacle type Performing detection to obtain the recognition degree of the image in the second detection window, including:

Obtaining a standard image corresponding to the second obstacle type;

Determining a similarity between the image in the second detection window and the standard image;

The similarity is determined as the degree of recognition of the image in the second detection window.

In another embodiment, the oblique line includes a straight line of the lower edge of the obstacle in the V-disparity map; or

The oblique line includes a straight line where the lower edge of the obstacle in the V-disparity map is located, and a portion of the upper edge line and/or the lower edge line of the V-disparity map.

In a second aspect, an embodiment of the present invention provides an obstacle recognition apparatus, including an extraction module, a first determining module, a second determining module, and a third determining module, where

The extracting module is configured to extract a diagonal line in the V disparity map of the image to be identified;

The first determining module is configured to determine, on the oblique line, a first starting pixel, where a sum of values of pixels in the pixel set corresponding to the first starting pixel is greater than a first threshold, where the first starting pixel corresponds The set of pixels includes pixels in the column in which the first start pixel is located above the oblique line;

The second determining module is configured to determine, according to a parallax of the first starting pixel and a window parameter corresponding to the preset obstacle type, a detection window corresponding to the first starting pixel in the V disparity map;

The third determining module is configured to determine an obstacle type identified in the image to be identified according to the recognition degree of the image in the detection window.

In an embodiment, the first determining module is specifically configured to:

In another embodiment, the preset obstacle type includes a first obstacle type, and the first obstacle type corresponds to a first window parameter; and the second determining module is specifically configured to:

Obtaining a first window parameter corresponding to the first obstacle type;

In another implementation, the second determining module is specifically configured to:

In another implementation, the third determining module is specifically configured to:

In another embodiment, the detection window includes a second detection window, and the second detection window corresponds to a second obstacle type; the third determining module is specifically configured to:

Obtaining a standard image corresponding to the second obstacle type;

In a third aspect, an embodiment of the present invention provides an obstacle recognition terminal, including a processor, a memory, a camera component, and a communication bus, where the communication bus is used to implement connection between components, and the memory is used to store a computer. An instruction, the processor configured to execute the computer instructions to cause the terminal to execute:

In a fourth aspect, an embodiment of the present invention provides a computer readable storage medium storing computer executable instructions for causing the computer to perform any of the foregoing Methods.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief description of the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description It is a certain embodiment of the present invention, and other drawings can be obtained from those skilled in the art without any inventive labor.

1 is a schematic diagram of an application scenario of an obstacle recognition method according to an embodiment of the present invention;

2 is a schematic flowchart 1 of an obstacle recognition method according to an embodiment of the present invention;

3A is a schematic diagram 1 of a diagonal line according to an embodiment of the present invention;

FIG. 3B is a schematic diagram 2 of a diagonal line according to an embodiment of the present invention; FIG.

3C is a schematic diagram 3 of a diagonal line according to an embodiment of the present invention;

FIG. 3D is a schematic diagram 4 of a diagonal line according to an embodiment of the present invention; FIG.

4 is a schematic diagram of a process of determining a first start pixel according to an embodiment of the present invention;

FIG. 5 is a schematic flowchart of a method for determining a first detection window according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of an obstacle recognition process according to an embodiment of the present invention; FIG.

FIG. 7 is a schematic structural diagram of an obstacle recognition apparatus according to an embodiment of the present invention;

FIG. 8 is a schematic structural diagram of an obstacle recognition terminal according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. It is a partial embodiment of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

In the related art, in the obstacle recognition process based on binocular stereo vision, a binocular stereoscopic vision technique is generally used to acquire a disparity map of an image to be recognized, and an obstacle recognition is performed according to a line in the disparity map. However, in the disparity map, the edge line of the obstacle can usually be preserved normally, while the part of the homogenous weak texture usually disappears, and the continuous envelope of the obstacle in the disparity map may be divided into discrete segments. Happening. For example, for a bus in an image, the overall line of the bus in the disparity map is divided into two parts (head line and tail line). In the identification process, the front and rear of the bus are often judged as Two objects, which in turn cause the bus identification in the image to fail. Then, when the obstacle recognition is performed according to the line in the parallax map, the same obstacle may be recognized as a plurality of obstacles, resulting in a lower accuracy of the obstacle recognition.

FIG. 1 is a schematic diagram of an application scenario of an obstacle recognition method according to an embodiment of the present invention. Referring to FIG. 1, a V-disparity map 101 and an identification model 102 including an image to be identified are included. Wherein, the image to be identified in the present application may include a plurality of obstacles, and correspondingly, the V-disparity map 101 includes an envelope map corresponding to each obstacle (a line drawing of the obstacle), and the obstacle in the V-disparity map 101 The line in the envelope map may be partially missing. For example, the V-disparity map 101 of the image to be recognized shown in FIG. 1 includes the obstacle 101-1 and the obstacle 101-2 in the envelope diagram of the obstacle 101-1. Part of the line of obstacle 101-1 is missing. The recognition model 102 is obtained by learning a large number of samples. Optionally, the identification module 102 may include multiple obstacle types and window parameters corresponding to each obstacle type. For example, the window parameters may be used to indicate the window. Length, width, etc.

In the present application, when identifying an obstacle in the image to be identified, a starting pixel of the obstacle may be first determined in the V disparity map of the image to be identified, and determined according to a window parameter corresponding to the obstacle type in the recognition model. The detection window identifies the image in the determined detection window according to the obstacle type corresponding to the detection window to determine the recognition degree of the image in the detection window, wherein the recognition degree of the image in the detection window is: detecting the image in the window The similarity between the preset images corresponding to the detection window, and the type of the obstacle corresponding to the detection window having the highest recognition is determined as the obstacle type of the obstacle.

For example, referring to Fig. 1, when the obstacle 101-1 is identified, the detection window A, the detection window B, and the detection window C corresponding to the recognition obstacle 101-1 can be determined based on the recognition model. It is assumed that the obstacle types corresponding to the detection window A, the detection window B, and the detection window C are pedestrians, small cars, and buses, respectively, and the recognition degree of the image in the detection window A is determined (the similarity between the image in the detection window A and the preset pedestrian) Degree is the first degree of recognition, determining the degree of recognition of the image in the detection window B (detecting the similarity between the image in the window B and the preset small car) as the second degree of recognition, determining the degree of recognition of the image in the detection window C ( The similarity between the image in the detection window C and the preset bus is the third recognition degree. Assuming that the second degree of recognition is the highest, it is possible to determine that the obstacle 101-1 is a small car.

The technical solutions shown in the present application are described in detail below through specific embodiments. It should be noted that the following specific embodiments may be combined with each other, and the same or similar content will not be repeatedly described in different embodiments.

FIG. 2 is a schematic flowchart 1 of an obstacle recognition method according to an embodiment of the present invention. Referring to FIG. 2, the method may include:

S201. Extract a diagonal line in the V-disparity map of the image to be identified.

The execution subject of the embodiment of the present invention may be an obstacle recognition device. Optionally, the obstacle recognition device may be implemented by software, or the obstacle recognition device may also be implemented by a combination of software and hardware.

Optionally, the image to be recognized may be an image taken by the imaging device.

Optionally, the disparity map of the image to be identified may be acquired first, the V disparity map corresponding to the disparity map is obtained, and the oblique line is extracted in the V disparity map. It should be noted that the process of acquiring the disparity map of the image to be identified and the process of obtaining the V disparity map of the disparity map may be referred to the prior art, which is not specifically limited in the embodiment of the present invention.

Optionally, the value of each pixel in the disparity map is the parallax of the pixel.

Optionally, the number of rows of the V disparity map is the same as the number of rows of the disparity map. The number of columns of the V-disparity map is the same as the range of the parallax in the parallax map. For example, if the disparity range in the disparity map is 0-5 (a total of 6 disparity), the number of columns of the V disparity map is 6 columns.

Optionally, the value of the pixel of the pixel (i, j) in the V disparity map is the number of pixels in the i-th row of the disparity map and having a disparity of j. For example, if the number of pixels having a parallax of 3 in the second row of the parallax map is three, the value of the pixel in the second row and the third column in the V-disparity map is three.

The existing theory proves that the locally flat ground has an inclined straight line in the V parallax map. The oblique line in the embodiment of the present invention includes a straight line where the ground is located, for example, the inclined straight line may be a part of a diagonal line or a diagonal line.

Optionally, when the interception of the V-disparity map is different, the shape of the obstacle, the angle, the position, and the like of the obstacle in the V-disparity map are different, and the oblique line and the V-disparity map are shown in the embodiment of the present invention. The shape of the obstacle, the angle at which the obstacle is placed (lateral, positive, etc.), and the position of the obstacle in the V-disparity map are related. For example, the oblique line may include a straight line where the lower edge of the obstacle in the V-disparity map is located; or the oblique line may include a straight line where the lower edge of the obstacle in the V-disparity map is located and a partial upper edge line of the V-disparity map; or, oblique The line may include a line where the lower edge of the obstacle in the V-disparity map is located and a part of the lower edge line of the V-disparity map; or, the oblique line may include a line on the lower edge of the obstacle in the V-disparity map, and a portion of the V-disparity map The edge line and part of the lower edge line of the V disparity map.

Alternatively, a straight line fit may be performed in the V disparity map to obtain an initial oblique line, which may be a diagonal line in the embodiment of the present invention, or a part of the oblique line in the embodiment of the present invention. It is determined whether the two ends of the initial oblique line can extend to the left and right edges of the V disparity map. If yes, it can be determined that the initial oblique line is the oblique line in the embodiment of the present invention, and if not, the upper edge of the V disparity map can be followed. And/or the lower edge, the initial oblique line is extended such that the initial oblique line can extend to the left and right edges of the V disparity map. Specifically, the following four cases may be included. Hereinafter, oblique lines will be described in detail with reference to FIGS. 3A to 3D.

FIG. 3A is a schematic diagram 1 of a diagonal line according to an embodiment of the present invention. FIG. 3B is a schematic diagram 2 of a diagonal line according to an embodiment of the present invention. FIG. 3C is a third schematic diagram of a diagonal line according to an embodiment of the present invention. FIG. 3D is a schematic diagram 4 of a diagonal line according to an embodiment of the present invention. Here, in FIGS. 3A to 3D, a V-disparity map and oblique lines extracted in the V-disparity map are included.

Referring to FIG. 3A, the initial oblique line obtained by fitting in the V disparity map may extend to the left and right edges of the V disparity map, and therefore, the initial oblique line may be directly determined as the oblique line S1. In this case, the oblique line S1 includes a straight line where the lower edge of the obstacle in the V-disparity map is located.

Referring to FIG. 3B, the left end of the initial oblique line fitted in the V disparity map may extend to the right edge of the V disparity map, but the right end of the initial oblique line may only extend to the M point of the V disparity map, and may not extend to V. The left edge of the disparity map. Therefore, the initial oblique line can be extended to the left along the upper edge of the V-disparity map to obtain the oblique line S2. In this case, the oblique line S2 includes a straight line where the lower edge of the obstacle in the V-disparity map is located and a partial upper edge line of the V-disparity map.

Referring to FIG. 3C, the right end of the initial oblique line fitted in the V disparity map may extend to the left edge of the V disparity map, but the left end of the initial oblique line may only extend to the N point of the V disparity map, and may not extend to V. The right edge of the disparity map. Therefore, the initial oblique line can be extended to the right along the lower edge of the V-disparity map to obtain the oblique line S3. In this case, the oblique line S3 includes a straight line where the lower edge of the obstacle in the V-disparity map is located and a partial lower edge line of the V-disparity map.

Referring to FIG. 3D, the left end of the initial oblique line fitted in the V disparity map can only extend to the P point of the V disparity map, and cannot extend to the left edge of the V disparity map, and the right end of the initial oblique line can only extend to V. The Q point of the disparity map cannot be extended to the right edge of the V disparity map. Therefore, the initial oblique line can be extended to the left along the upper edge of the V-disparity map, and the initial oblique line can be extended to the right along the lower edge of the V-disparity map to obtain the oblique line S4. In this case, the oblique line S4 includes a straight line where the lower edge of the obstacle in the V-disparity map, a partial upper edge line of the V-disparity map, and a partial lower edge line of the V-disparity map.

Optionally, the slanting line may be extracted in the V disparity map by an algorithm such as Hough transform, RANSAC, and least squares, and is not described herein again in the embodiment of the present invention.

S202. Determine a first starting pixel on a diagonal line.

The sum of the values of the pixels in the set of pixels corresponding to the first start pixel is greater than the first threshold, and the set of pixels corresponding to the first start pixel includes the pixels above the oblique line in the column of the first start pixel.

Optionally, in the process of identifying the obstacle in the V disparity map, the obstacle may be identified from left to right, or the obstacle may be identified from right to left.

Optionally, according to the direction of the obstacle recognition (from left to right or from right to left), the pixel on the oblique line is sequentially determined as the first pixel from the end of the oblique line along the extending direction of the oblique line. And obtaining a sum of values of pixels in the set of pixels corresponding to the first pixel, until the sum of the values of the pixels in the set of pixels corresponding to the first pixel obtained on the oblique line is greater than the first threshold, determining the first pixel as The first starting pixel.

Optionally, the sum of the values of the pixels in the set of pixels corresponding to the first pixel is used to indicate the number of pixels in the obstacle included in the column of the first pixel in the V disparity map. The larger the sum of the values, the more the number of pixels of the obstacle included in the column in which the first pixel is located.

Optionally, one end of the slash can be an end point of the slash, or the end of a detection window in the slash. For example, when the obstacle recognition is started on the V-disparity map, one end of the oblique line is an end point of the oblique line. When a plurality of obstacles are included in the V-disparity map and an obstacle has been identified, and then the obstacle recognition is continued, one end of the oblique line is the end point of the most recently determined detection window in the oblique line.

Next, the process of determining the first start pixel will be described in detail with reference to FIG.

FIG. 4 is a schematic diagram of a process of determining a first start pixel according to an embodiment of the present invention. Referring to FIG. 4, an obstacle 401, an obstacle 402, and a diagonal S are included in the V-disparity map.

Assume that the direction of obstacle recognition is from right to left. Initially, starting from the rightmost side of the oblique line S, the pixel A is first determined as the first pixel, and the sum of the values of the pixels located above the oblique line S in the column of the pixel A is obtained, and the value is determined. And if it is greater than the first threshold. As can be seen from FIG. 4, the pixel in the obstacle 401 is not included in the column of the pixel A. Therefore, the sum of the values of the pixels located above the oblique line S in the column of the pixel A is smaller than the first threshold.

Further, the pixel B is determined as the first pixel, and it is judged that the pixel B does not satisfy the condition as the first start pixel. Further, the pixel C is determined as the first pixel, and it is judged that the pixel C does not satisfy the condition as the first start pixel. Further, the pixel D is determined as the first pixel, and the pixel D is determined to be the first starting pixel, and the pixel D is determined as the first starting pixel.

Assuming that the obstacle 401 is identified in the V disparity map, it is necessary to continue to determine the new first starting pixel. specific:

According to the extending direction of the oblique line, the pixel E is first determined as the first pixel, and it is judged that the pixel E does not satisfy the condition as the first starting pixel. Further, the pixel F is determined as the first pixel, and it is judged that the pixel F does not satisfy the condition as the first start pixel. Further, the pixel G is determined as the first pixel, and the pixel G is determined to be the first starting pixel, and the pixel G is determined as the first starting pixel.

S203. Determine, according to a parallax of the first starting pixel and a window parameter corresponding to the preset obstacle type, a detection window corresponding to the first starting pixel in the V disparity map.

Optionally, the number of the preset obstacle types may be one, or may be multiple. In the actual application process, the number of the preset obstacle types may be set according to actual needs, which is not used in the embodiment of the present invention. Specifically limited.

In the embodiment of the present invention, a preset obstacle type corresponds to a window parameter, and the preset obstacle type and the corresponding window parameter are obtained by learning a large number of samples in advance.

For example, the correspondence between the preset obstacle type and the window parameter can be as shown in Table 1.

Optionally, when the obstacle recognition direction is detected from right to left, the first start pixel is used as the lower right corner of the detection window, and each preset is determined according to the window parameter corresponding to each preset obstacle type. The detection window corresponding to the obstacle type.

Table 1

预设障碍物类型Preset obstacle type	窗口参数Window parameter
公交车bus	长：Z1，高：Y1Length: Z1, height: Y1
货车truck	长：Z2，高：Y2Length: Z2, height: Y2
小型车Small car	长：Z3，高：Y3Length: Z3, high: Y3
非机动车non-motor vehicle	长：Z4，高：Y4Length: Z4, high: Y4
……......	……......

Optionally, when the obstacle recognition direction is detected from left to right, the first start pixel is used as the lower left corner of the detection window, and each preset is determined according to the window parameter corresponding to each preset obstacle type. The detection window corresponding to the obstacle type.

Wherein, it is determined that the number of detection windows is the same as the number of preset obstacle types. For example, if five preset obstacle types are pre-set, it can be determined that five detection windows corresponding to the first start pixel are obtained.

It should be noted that the method for determining the detection window is described in detail in the embodiment shown in FIG. 5, and will not be described here.

S204. Determine, according to the recognition degree of the image in the detection window, the type of the obstacle identified in the image to be identified.

Optionally, after determining the detection window, the detection window may be pre-processed to extract an image in the detection window.

For example, the pre-processing may include: culling the slash in the detection window, randomly adding or deleting points on the oblique line in the detection window, performing appropriate enlargement or reduction processing on the detection window, and the like. Of course, in the actual application process, the preset processing may also include other processing, which is not specifically limited in this embodiment of the present invention.

Optionally, the image in the detection window may be detected according to the preset obstacle type corresponding to the detection window, to obtain the recognition degree of the image in the detection window, and the target detection window is determined according to the recognition degree of the image in the detection window. The image in the detection window has the highest degree of recognition, and the type of obstacle corresponding to the target window is determined as the type of obstacle identified in the image to be recognized.

It is assumed that the detection window includes a second detection window, and the second detection window corresponds to the second obstacle type. Correspondingly, the recognition degree of the image in the second detection window is obtained according to the following feasible implementation manner, including: acquiring the second obstacle type corresponding The standard image determines the similarity between the image in the second detection window and the standard image, and determines the similarity as the recognition degree of the image in the second detection window.

Optionally, multiple features of the image in the second detection window may be extracted, multiple features of the standard image are extracted, and multiple features of the image in the second detection window are matched with multiple features of the standard image to determine The similarity between the image in the second detection window and the standard image. Wherein, the more the number of features of the image in the second detection window matches the features of the standard image, the higher the similarity.

Optionally, the feature extraction method may include HOG, LBP, DPM, and the like.

For example, if the preset obstacle type is a bus, and the detection window determined according to the window parameter corresponding to the preset obstacle type “bus” is the detection window 1, when the image in the detection window 1 is detected, Comparing the image in the detection window 1 with the preset bus image to obtain the similarity between the image in the detection window 1 and the bus, and determining the similarity between the image in the detection window 1 and the bus as detection The degree of recognition of the image in window 1.

Optionally, when performing image recognition on the image in the detection window, the feature of the image in the detection window may be extracted first, and the obstacle recognition may be performed according to the feature of the image in the detection window. For example, the feature extraction method may include HOG, LBP, DPM, etc. Obstacle recognition can also be performed by convolutional neural networks.

It should be noted that, in the actual application process, when multiple obstacles are included in the image to be identified, by repeatedly performing the embodiment shown in FIG. 2, each obstacle can be identified in the image to be identified, correspondingly The recognition result may include an obstacle type included in the image to be detected, and a position of the obstacle type in the image to be detected.

The obstacle recognition method provided by the embodiment of the present invention first extracts a diagonal line in the V-disparity map of the image to be recognized when the obstacle recognition is required for the image to be recognized, and determines the first start pixel on the oblique line according to the first start. a parallax corresponding to the pixel and a window parameter corresponding to the preset obstacle type, determining a detection window corresponding to the first starting pixel in the V disparity map, and determining an obstacle recognized in the image to be identified according to the recognition degree of the image in the detection window Types of.

In the above process, only one of the preset obstacle types is the obstacle type corresponding to the obstacle. Therefore, only one detection window in the determined detection window is the window in which the obstacle is located (hereinafter referred to as the correct one). Detection window), other detection windows are not the window where the obstacle is located (hereinafter referred to as the error detection window). Since the type of the preset obstacle corresponding to the erroneous detection window is different from the type of the obstacle in the detection window, the image recognition degree in the erroneous detection window is usually lower than the preset threshold. The correct detection window corresponds to the type of the preset obstacle and the type of the obstacle in the detection window. Even if the obstacle envelope line in the V disparity map is partially missing, the image recognition in the correct detection window will still be caused. The degree is greater than the preset threshold.

It can be seen from the above that even if the obstacle envelope line in the V disparity map is partially missing, the recognition degree of the image in the correct detection window is much higher than the recognition degree of the image in the erroneous detection window, and therefore, even if it is to be identified When the envelope diagram of the obstacle in the V-disparity map of the image is partially missing, the type of the obstacle can still be accurately identified in the image to be recognized, thereby improving the accuracy of the obstacle recognition.

On the basis of any of the foregoing embodiments, optionally, the first start pixel corresponding to the window parameter corresponding to the preset obstacle type and the window parameter corresponding to the preset obstacle type may be determined according to the feasible implementation manner. The detection window (S203 in the embodiment shown in FIG. 2), specifically, the embodiment shown in FIG.

It should be noted that the process of determining each detection window in the detection window is the same. Below, the preset obstacle type is the first obstacle type, the window parameter is the first window parameter, and the determined detection window is determined as the first detection. The window is explained as an example.

FIG. 5 is a schematic flowchart of a method for determining a first detection window according to an embodiment of the present invention. Referring to FIG. 5, the method may include:

S501. Acquire a first window parameter corresponding to the first obstacle type.

Optionally, the first window parameter may include a length, a width, a height, and the like of the obstacle corresponding to the first obstacle type. Of course, in the actual application process, the content included in the first window parameter may be set according to actual needs, which is not specifically limited in this embodiment of the present invention.

S502. Determine a size of the first detection window according to the obstacle recognition direction, the parallax of the first start pixel, the first window parameter, and the camera parameter that captures the image to be recognized.

Optionally, the obstacle recognition direction includes left to right and right to left.

Optionally, the camera shown in the embodiment of the present invention is usually a binocular camera. Correspondingly, the camera parameters generally include the baseline length, focal length, and the like of the binocular camera.

Optionally, the size of the first detection window generally includes the length and width of the first detection window.

Optionally, when the obstacle recognition direction is from right to left, the width of the first detection window may be determined by the following formula 1, wherein the width of the first detection window refers to the lateral length.

Where x is the width of the first detection window, d is the parallax of the first starting pixel, Δz is the preset width in the first window parameter, B is the baseline length of the binocular camera, and f is the focal length of the binocular camera.

Optionally, when the obstacle recognition direction is from left to right, the width of the first detection window may be determined by the following formula 2, wherein the width of the first detection window refers to the lateral length.

Alternatively, the height of the first detection window may be determined by the following formula 3, wherein the height of the first detection window refers to the longitudinal length.

Where y is the height of the first detection window, d is the parallax of the first starting pixel, Δy is the preset height in the first window parameter, and B is the baseline length of the binocular camera.

It should be noted that the above is only a schematic manner for determining the size of the first detection window in an exemplary manner. Of course, in the actual application process, the size of the first detection window may be determined according to actual needs, which is not used by the embodiment of the present invention. Specifically limited.

S503. Determine a first detection window in the V disparity map according to the size of the first detection window.

Optionally, the first detection window may also be determined according to the preset window shape. For example, the preset window shape may be a polygon such as a rectangle or a trapezoid. Of course, in the actual application process, the preset window shape may be set according to actual needs, which is not specifically limited in the embodiment of the present invention.

Optionally, the position of the first start pixel in the first detection window may be determined according to the obstacle recognition direction, according to the position of the first start pixel in the first detection window and the size of the first detection window, in the V disparity map The first detection window is determined.

Optionally, when the obstacle recognition direction is from right to left, the first detection window is determined in the V disparity map according to the size of the first detection window with the first starting pixel as the lower right corner of the first detection window.

Optionally, when the obstacle recognition direction is from left to right, the first detection window is determined in the V disparity map according to the size of the first detection window with the first starting pixel as the lower left corner of the first detection window.

In the embodiment shown in FIG. 5, it is determined that the size of the obtained first detection window matches the size of the obstacle corresponding to the first obstacle type.

Hereinafter, the above method embodiment will be further described in detail by way of specific examples in conjunction with FIG. 6.

FIG. 6 is a schematic diagram of an obstacle recognition process according to an embodiment of the present invention. Referring to FIG. 6, the scene 601 to the scene 603 are included.

Referring to the scenario 601, after obtaining the V disparity map of the image to be identified, the oblique line S is extracted in the V disparity map. Referring to the scene 601, the oblique line S includes a line where the lower edge of the obstacle in the V-disparity map is located and a portion of the lower edge line of the disparity map.

Referring to the scenario 602, assuming that the obstacle recognition direction is from right to left, from the rightmost end of the oblique line, the pixel A is now determined as the first pixel, and it is judged that the pixel A does not satisfy the condition as the first start pixel. The pixel B is further determined as the first pixel, and it is judged that the pixel B does not satisfy the condition as the first start pixel. The pixel C is further determined as the first pixel, and it is judged that the pixel C does not satisfy the condition as the first start pixel. Further determining the pixel D as the first pixel and determining that the pixel D satisfies the condition as the first starting pixel, the pixel D is determined as the first starting pixel.

Please refer to scenario 603, assuming that there are four types of obstacles, namely pedestrians, non-motor vehicles, small cars and buses. Then, the position of the pixel D is taken as the lower right corner, and the detection window K1 is determined according to the window parameter corresponding to the pedestrian. Taking the position of the pixel D as the lower right corner, the detection window K2 is determined according to the window parameter corresponding to the non-motor vehicle. Taking the position of the pixel D as the lower right corner, the detection window K3 is determined according to the window parameter corresponding to the small car. Taking the position of the pixel D as the lower right corner, the detection window K4 is determined according to the window parameter corresponding to the bus.

Comparing the image in the detection window K1 with the standard pedestrian image, obtaining the similarity between the detection window K2 and the standard pedestrian image, and determining the similarity as the recognition degree of the image in the detection window K1, which is recorded as the recognition degree 1 . Comparing the image in the detection window K2 with the standard non-motor vehicle image, obtaining the similarity between the detection window K2 and the standard non-motor vehicle image, and determining the similarity as the recognition degree of the image in the detection window K2, To identify the degree 2. Comparing the image in the detection window K3 with the standard small car image, obtaining the similarity between the detection window K3 and the standard small car image, and determining the similarity as the recognition degree of the image in the detection window K3, which is recorded as identification Degree 3. Comparing the image in the detection window K4 with the standard bus image, obtaining the similarity between the detection window K4 and the standard bus image, and determining the similarity as the recognition degree of the image in the detection window K4, which is recorded as identification Degree 4. Specifically, the image to be compared and the degree of recognition can be as shown in Table 2:

Table 2

Comparing the degree of recognition 1 to the degree of recognition 4, assuming that the recognition degree 3 is determined to be the largest, determining that the detection window 3 is the target detection window, and determining the obstacle type (small car) corresponding to the detection window 3 as the recognition in the image to be recognized Type of obstacle.

FIG. 7 is a schematic structural diagram of an obstacle recognition apparatus according to an embodiment of the present invention. Referring to FIG. 7, an extraction module 11, a first determining module 12, a second determining module 13, and a third determining module 14 are included, where

The extraction module 11 is configured to extract a diagonal line in the V disparity map of the image to be identified;

The first determining module 12 is configured to determine, on the oblique line, a first starting pixel, where a sum of values of pixels in the pixel set corresponding to the first starting pixel is greater than a first threshold, where the first starting pixel corresponds to a set of pixels includes pixels in the column in which the first start pixel is located above the oblique line;

The second determining module 13 is configured to determine, according to the parallax of the first starting pixel and the window parameter corresponding to the preset obstacle type, a detection window corresponding to the first starting pixel in the V disparity map;

The third determining module 14 is configured to determine an obstacle type identified in the image to be identified according to the recognition degree of the image in the detection window.

The obstacle recognition device provided in the embodiment of the present invention can perform the technical solution shown in the foregoing method embodiment, and the implementation principle and the beneficial effects are similar, and details are not described herein.

In an embodiment, the first determining module 12 is specifically configured to:

In another embodiment, the preset obstacle type includes a first obstacle type, and the first obstacle type corresponds to a first window parameter; and the second determining module 13 is specifically configured to:

Obtaining a first window parameter corresponding to the first obstacle type;

In another implementation, the second determining module 13 is specifically configured to:

In another implementation, the third determining module 14 is specifically configured to:

In another embodiment, the detection window includes a second detection window, and the second detection window corresponds to a second obstacle type; the third determining module 14 is specifically configured to:

Obtaining a standard image corresponding to the second obstacle type;

FIG. 8 is a schematic structural diagram of an obstacle recognition terminal according to an embodiment of the present invention. Referring to FIG. 8, a processor 21, a memory 22, a camera component 23, and a communication bus 24 are provided. The communication bus 24 is used to implement a connection between components.

The processor 21 is the control center of the obstacle recognition terminal, and connects various parts of the terminal using various interfaces and lines, by running or executing software programs and/or modules stored in the memory 22, and calling the memory 22 in the memory. The internal data performs various functions and processing data of the obstacle recognition terminal, thereby performing overall monitoring of the terminal.

The memory 22 can be used to store software programs and modules, and the processor 21 executes various functional applications and data processing by running software programs and modules stored in the memory 22. The memory 22 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function, and the like; the storage data area may store data created according to the use of the obstacle recognition terminal ( For example, the acquired image, the calculated parallax image, or the processed grayscale image, etc.).

The camera assembly 23 is for acquiring images and transmitting the images to the memory 21 and/or the processor 22; the memory 21 is for storing computer instructions; the processor 22 is configured to perform the Computer instructions for causing the terminal to perform: extracting a diagonal line in a V-disparity map of the image to be identified; determining a first starting pixel on the oblique line, a sum of values of pixels in a pixel set corresponding to the first starting pixel The pixel set corresponding to the first start pixel includes a pixel located above the oblique line in the column of the first start pixel; the parallax according to the first start pixel and the preset obstacle type And corresponding to the window parameter, determining, in the V disparity map, a detection window corresponding to the first starting pixel; and determining, according to the recognition degree of the image in the detection window, an obstacle type identified in the to-be-identified image.

In some embodiments, the processor 22 determines the first starting pixel on the oblique line by performing the following process:

Determining, from the one end of the oblique line, the pixels on the oblique line as the first pixel along the extending direction of the oblique line, and acquiring the value of the pixel in the pixel set corresponding to the first pixel And determining, when the sum of the values of the pixels in the pixel set corresponding to the first pixel obtained on the oblique line is greater than the first threshold, determining the first pixel as the first start pixel; The set of pixels corresponding to the first pixel includes pixels located above the oblique line in the column of the first pixel.

In some embodiments, the preset obstacle type includes a first obstacle type, the first obstacle type corresponding to a first window parameter; the processor 22 performs according to the first start by performing processing described below Determining a parallax of the pixel and the first window parameter, determining, in the V disparity map, a first detection window corresponding to the first starting pixel:

Obtaining a first window parameter corresponding to the first obstacle type; determining, according to an obstacle recognition direction, a parallax of the first starting pixel, the first window parameter, and a camera parameter that captures the image to be recognized a size of the first detection window; determining the first detection window in the V disparity map according to the size of the first detection window, where the first starting pixel is a pixel of a corner of the first detection window .

In some embodiments, the processor 22 determines the first detection window in the V disparity map according to the size of the first detection window by performing a process of:

Determining, according to the obstacle recognition direction, a position of the first start pixel in the first detection window; according to a position of the first start pixel in the first detection window and the first detection window The size of the first detection window is determined in the V disparity map.

In some embodiments, the processor 22 determines the type of obstacle identified in the image to be identified according to the recognition degree of the image in the detection window by performing the following processing:

And detecting, according to the preset obstacle type corresponding to the detection window, the image in the detection window to obtain the recognition degree of the image in the detection window; determining the target detection window according to the recognition degree of the image in the detection window, the target detection The image in the window has the highest degree of recognition; the obstacle type corresponding to the target window is determined as the type of obstacle identified in the image to be recognized.

In some embodiments, the detection window includes a second detection window, the second detection window corresponding to a second obstacle type; the processor 22 performs, according to the second obstacle type, by performing the following processing, The image in the second detection window is detected to obtain the recognition degree of the image in the second detection window:

Obtaining a standard image corresponding to the second obstacle type; determining a similarity between the image in the second detection window and the standard image; determining the similarity as an image recognition degree in the second detection window .

Optionally, the oblique line includes a straight line where the lower edge of the obstacle in the V-disparity map is located; or the oblique line includes a straight line where the lower edge of the obstacle in the V-disparity map is located, and the V The portion of the upper edge line and/or the lower edge line of the disparity map.

Optionally, the number of camera components may be one or more. In the actual application process, the number of the camera components may be set according to actual needs, which is not specifically limited in the embodiment of the present invention.

An embodiment of the present invention provides a computer readable storage medium storing computer executable instructions for causing the computer to execute the method described in any of the above embodiments.

One of ordinary skill in the art will appreciate that all or part of the steps to implement the various method embodiments described above may be accomplished by hardware associated with the program instructions. The aforementioned program may be stored in a computer readable storage medium storing computer executable instructions for causing the computer to perform any of the methods described above. The program, when executed, performs the steps including the foregoing method embodiments; and the foregoing storage medium includes various media that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

It should be noted that the above embodiments are only used to explain the technical solutions of the embodiments of the present invention, and are not limited thereto; although the embodiments of the present invention are described in detail with reference to the foregoing embodiments, those skilled in the art It should be understood that the technical solutions described in the foregoing embodiments may be modified, or some or all of the technical features may be equivalently replaced; and the modifications or substitutions do not deviate from the essence of the corresponding technical solutions. The scope of the program.

Claims

An obstacle recognition method, comprising:

Extracting a diagonal line in the V disparity map of the image to be identified;

Determining, on the oblique line, a first start pixel, where a sum of values of pixels in the set of pixels corresponding to the first start pixel is greater than a first threshold, and a set of pixels corresponding to the first start pixel includes the first start pixel a pixel in the column above the slash;

Determining, in the V disparity map, a detection window corresponding to the first start pixel according to a parallax of the first start pixel and a window parameter corresponding to the preset obstacle type;

Determining an obstacle type identified in the image to be identified according to the recognition degree of the image in the detection window.
The method of claim 1 wherein determining the first starting pixel on the oblique line comprises:

Determining, from the one end of the oblique line, the pixels on the oblique line as the first pixel along the extending direction of the oblique line, and acquiring the value of the pixel in the pixel set corresponding to the first pixel And determining, when the sum of the values of the pixels in the pixel set corresponding to the first pixel obtained on the oblique line is greater than the first threshold, determining the first pixel as the first start pixel;

The pixel set corresponding to the first pixel includes a pixel located above the oblique line in the column of the first pixel.
The method according to claim 1 or 2, wherein the preset obstacle type comprises a first obstacle type, the first obstacle type corresponding to a first window parameter; according to the first starting pixel Determining, by the disparity and the first window parameter, determining, in the V disparity map, a first detection window corresponding to the first start pixel, including:

Obtaining a first window parameter corresponding to the first obstacle type;

Determining a size of the first detection window according to an obstacle recognition direction, a parallax of the first start pixel, the first window parameter, and a camera parameter that captures the image to be recognized;

Determining, in the V disparity map, the first detection window according to a size of the first detection window, where the first starting pixel is a pixel of a corner of the first detection window.
The method according to claim 3, wherein determining the first detection window in the V disparity map according to the size of the first detection window comprises:

Determining a position of the first start pixel in the first detection window according to the obstacle recognition direction;

The first detection window is determined in the V disparity map according to a position of the first start pixel in the first detection window and a size of the first detection window.
The method according to claim 1 or 2, wherein determining the type of the obstacle identified in the image to be identified according to the degree of recognition of the image in the detection window comprises:

Detecting an image in the detection window according to a preset obstacle type corresponding to the detection window, to obtain an image recognition degree in the detection window;

Determining a target detection window according to the recognition degree of the image in the detection window, wherein the recognition degree of the image in the target detection window is the highest;

The obstacle type corresponding to the target window is determined as the obstacle type identified in the image to be identified.
The method according to claim 5, wherein the detection window comprises a second detection window, the second detection window corresponds to a second obstacle type; and according to the second obstacle type, the second Detecting an image in the window to detect the image in the second detection window, including:

Obtaining a standard image corresponding to the second obstacle type;

Determining a similarity between the image in the second detection window and the standard image;

The similarity is determined as the degree of recognition of the image in the second detection window.
Method according to claim 1 or 2, characterized in that

The oblique line includes a straight line where the lower edge of the obstacle in the V-disparity map is located; or

The oblique line includes a straight line where the lower edge of the obstacle in the V-disparity map is located, and a portion of the upper edge line and/or the lower edge line of the V-disparity map.
An obstacle recognition terminal, comprising: a processor, a memory, a camera assembly, and a communication bus, wherein the communication bus is used to implement a connection between components, and the memory is configured to store computer instructions, and the processing The processor is configured to execute the computer instructions to cause the terminal to execute:

Extracting a diagonal line in the V disparity map of the image to be identified;

Determining, on the oblique line, a first start pixel, where a sum of values of pixels in the set of pixels corresponding to the first start pixel is greater than a first threshold, and a set of pixels corresponding to the first start pixel includes the first start pixel a pixel in the column above the slash;

Determining, in the V disparity map, a detection window corresponding to the first start pixel according to a parallax of the first start pixel and a window parameter corresponding to the preset obstacle type;

Determining an obstacle type identified in the image to be identified according to the recognition degree of the image in the detection window.
The terminal according to claim 8, wherein said processor determines a first start pixel on said oblique line by performing processing described below:

Determining, from the one end of the oblique line, the pixels on the oblique line as the first pixel along the extending direction of the oblique line, and acquiring the value of the pixel in the pixel set corresponding to the first pixel And determining, when the sum of the values of the pixels in the pixel set corresponding to the first pixel obtained on the oblique line is greater than the first threshold, determining the first pixel as the first start pixel;

The pixel set corresponding to the first pixel includes a pixel located above the oblique line in the column of the first pixel.
The terminal according to claim 8 or 9, wherein the preset obstacle type includes a first obstacle type, the first obstacle type corresponds to a first window parameter; and the processor performs the following Processing, according to the disparity of the first starting pixel and the first window parameter, determining, in the V disparity map, a first detection window corresponding to the first starting pixel:

Obtaining a first window parameter corresponding to the first obstacle type;

Determining a size of the first detection window according to an obstacle recognition direction, a parallax of the first start pixel, the first window parameter, and a camera parameter that captures the image to be recognized;

Determining, in the V disparity map, the first detection window according to a size of the first detection window, where the first starting pixel is a pixel of a corner of the first detection window.
The terminal according to claim 10, wherein the processor determines the first detection window in the V-disparity map according to a size of the first detection window by performing a process of:

Determining a position of the first start pixel in the first detection window according to the obstacle recognition direction;

The first detection window is determined in the V disparity map according to a position of the first start pixel in the first detection window and a size of the first detection window.
The terminal according to claim 8 or 9, wherein the processor determines the type of the obstacle recognized in the image to be recognized based on the degree of recognition of the image in the detection window by performing the following processing:

Detecting an image in the detection window according to a preset obstacle type corresponding to the detection window, to obtain an image recognition degree in the detection window;

Determining a target detection window according to the recognition degree of the image in the detection window, wherein the recognition degree of the image in the target detection window is the highest;

The obstacle type corresponding to the target window is determined as the obstacle type identified in the image to be identified.
The terminal according to claim 12, wherein the detection window includes a second detection window, the second detection window corresponds to a second obstacle type; and the processor performs the following processing by performing the following processing a second obstacle type, detecting an image in the second detection window to obtain an image recognition degree in the second detection window:

Obtaining a standard image corresponding to the second obstacle type;

Determining a similarity between the image in the second detection window and the standard image;

The similarity is determined as the degree of recognition of the image in the second detection window.
A terminal according to claim 8 or 9, wherein

The oblique line includes a straight line where the lower edge of the obstacle in the V-disparity map is located; or

The oblique line includes a straight line where the lower edge of the obstacle in the V-disparity map is located, and a portion of the upper edge line and/or the lower edge line of the V-disparity map.
A computer readable storage medium storing computer executable instructions for causing the computer to perform the method of any of claims 1-7.