WO2021093418A1

WO2021093418A1 - Ground obstacle detection method and device, and computer-readable storage medium

Info

Publication number: WO2021093418A1
Application number: PCT/CN2020/112132
Authority: WO
Inventors: 赵健章; 邹振华
Original assignee: 深圳创维数字技术有限公司
Priority date: 2019-11-12
Filing date: 2020-08-28
Publication date: 2021-05-20
Also published as: CN110826512B; CN110826512A

Abstract

Disclosed are a ground obstacle detection method and device, and a computer-readable storage medium. The method comprises: when obstacles are detected on the basis of a photographing apparatus mounted on a vehicle body, performing projection on each of the obstacles in a preset direction, so as to generate a projection height of each of the obstacles; according to the projection heights, classifying the obstacles into a first obstacle and a second obstacle, and identifying a first contour of the first obstacle in the preset direction; performing texture edge detection on the second obstacle to generate a second contour of the second obstacle; and according to the first contour and the second contour, determining a target obstacle closest to the vehicle body. According to the present application, with regard to a high obstacle, image processing may not be interfered with due to wobbling of the vehicle body, thereby ensuring the precision of identification; and with regard to a short obstacle, detection can be realized without the need to reduce the ground precision, various obstacles in a transport environment can be comprehensively identified, and a strong environment compatibility is provided.

Description

Ground obstacle detection method, equipment and computer readable storage medium

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office, the application number is 201911103707.4, and the invention title is "ground obstacle detection method, equipment and computer-readable storage medium" on November 12, 2019, and the entire content of it is approved The reference is incorporated in the application.

Technical field

This application relates to the field of intelligent driving technology, and in particular to a ground obstacle detection method, device, and computer-readable storage medium.

Background technique

With the development of smart technology, the application of smart warehouses has become more and more extensive; at present, smart warehouses use navigation to guide vehicles to drive automatically and realize the transportation of goods. If there are obstacles on the transportation path and its surroundings during transportation, the distance of the obstacles needs to be detected to avoid the danger of collision.

Although traditional laser radar SLAM (simultaneous localization and mapping, real-time positioning and map construction) can be used directly for navigation; however, in the process of processing obstacles, the posture of the vehicle may shake, which affects the camera installed on the vehicle. The actual measurement of the angle causes the ground to remove a lot of interference, and even false obstacle interference; and after reducing the accuracy on the ground, some very low obstacles will not be recognized, resulting in incomplete obstacle detection.

Summary of the invention

The main purpose of this application is to provide a ground obstacle detection method, equipment, and computer readable storage medium, aiming to solve the existing technical problems of inaccurate and incomplete ground obstacle detection.

To achieve the above objective, the present application provides a ground obstacle detection method, the ground obstacle detection method includes the steps:

When an obstacle is detected based on the camera device installed on the vehicle body, projection of each of the obstacles in a preset direction is performed to generate the projection height of each of the obstacles;

Dividing each of the obstacles into a first obstacle and a second obstacle according to each of the projection heights, and identifying a first contour of the first obstacle in a preset direction;

Performing texture edge detection on the second obstacle to generate a second contour of the second obstacle;

According to the first contour and the second contour, the target obstacle closest to the position of the vehicle body is determined.

In addition, in order to achieve the above object, the present application also provides a ground obstacle detection device, the ground obstacle detection device includes a memory, a processor, and a ground obstacle stored in the memory and capable of running on the processor An object detection program, which implements the steps of the above-mentioned ground obstacle detection method when the ground obstacle detection program is executed by the processor.

In addition, in order to achieve the above-mentioned object, the present application also provides a computer-readable storage medium having a ground obstacle detection program stored on the computer-readable storage medium, and the ground obstacle detection program is executed by a processor to achieve the above The steps of the ground obstacle detection method described.

When an obstacle is detected by the camera device installed on the vehicle body, this application first performs projection processing on each obstacle in a preset direction, and divides each obstacle into the height of each obstacle represented by the projection height. The first obstacle and the second obstacle; thereafter, the first obstacle is identified in the preset direction for the first obstacle, and the texture edge detection of the second obstacle is performed to obtain the second contour; and then based on the first contour and The position of the second contour is to determine the target obstacle closest to the position of the vehicle body. Since the recognized first contour has little correlation with the angle installed by the camera, vehicle body shaking will not interfere with image processing, ensuring the accuracy of recognition; while the second contour is generated based on texture edge detection without lowering the ground The accuracy can realize the detection of low obstacles, can fully identify various obstacles in the transportation environment, and have strong environmental compatibility.

Description of the drawings

FIG. 1 is a schematic structural diagram of a hardware operating environment involved in a solution of an embodiment of the present application;

2 is a schematic flowchart of a first embodiment of a ground obstacle detection method according to the present application;

FIG. 3 is a schematic flowchart of a second embodiment of a ground obstacle detection method according to the present application;

4 is a schematic flowchart of a third embodiment of a ground obstacle detection method according to the present application;

FIG. 5 is a schematic diagram of generating a first contour in the ground obstacle detection method of the present application;

Fig. 6 is a schematic diagram of contour line generation in the polar coordinate mode of the first contour and the second contour in the ground obstacle detection method of the present application.

The realization, functional characteristics, and advantages of the purpose of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Detailed ways

It should be understood that the specific embodiments described here are only used to explain the application, and not used to limit the application.

As shown in FIG. 1, FIG. 1 is a schematic structural diagram of a hardware operating environment involved in a solution of an embodiment of the present application.

As shown in FIG. 1, the ground obstacle detection may include: a processor 1001, such as a CPU, a user interface 1003, a network interface 1004, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to implement connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as a magnetic disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

Those skilled in the art can understand that the ground obstacle detection structure shown in FIG. 1 does not constitute a limitation on ground obstacle detection, and may include more or less components than shown in the figure, or a combination of certain components, or different components. The layout of the components.

As shown in FIG. 1, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a ground obstacle detection program. Among them, the operating system is a program that manages and controls ground obstacle detection hardware and software resources, and supports the operation of ground obstacle detection programs and other software or programs.

In the ground obstacle detection shown in FIG. 1, the user interface 1003 is mainly used to connect to the client (user side) and communicate with the client; the network interface 1004 is mainly used to connect to the background server and communicate with the background server; The processor 1001 may be used to call the ground obstacle detection program stored in the memory 1005, and perform the following operations:

Further, the step of determining the target obstacle closest to the position of the vehicle body according to the first contour and the second contour includes:

Converting the first contour to first polar coordinates, converting the second contour to second polar coordinates, and determining whether the first polar coordinates and the second polar coordinates are within a preset angle range;

If it is within the preset angle range, combining the first polar coordinate and the second polar coordinate to generate a polar coordinate set;

According to the set of polar coordinates, a target obstacle that is closest to the position of the vehicle body is determined.

Further, the step of converting the first contour into a first polar coordinate includes:

Reading the installation height, installation angle, vertical field of view, horizontal field of view, number of effective pixel rows and effective pixel columns of the camera device;

Read all the pixels in the first contour as measurement points, and perform the following steps for each measurement point one by one:

Detecting the depth value between the measurement point and the camera device, and the number of pixel rows and pixel columns of the measurement point;

Determine the polar coordinate modulus of the measurement point according to the installation angle, the vertical field of view angle, the number of pixel rows, the number of effective pixel rows, and the depth value;

Determine the polar coordinate angle of the measurement point according to the horizontal field of view angle, the installation height, the installation angle, the vertical field of view angle, the number of pixel columns, the number of effective pixel columns, the number of pixel rows and the number of effective pixel rows;

The polar coordinate modulus value and the polar coordinate angle are set as the polar coordinates of the measuring point, and after the polar coordinates are generated for each measuring point, each of the polar coordinates is formed as the first polar coordinate.

Further, the step of determining the target obstacle closest to the position of the vehicle body according to the set of polar coordinates includes:

Performing median filtering, denoising, and mean filtering on each element in the polar coordinate set in sequence to generate a processing result;

Combine the elements in the processing result to generate a target element, and calculate a first angle and a first distance corresponding to the target element and the camera device;

According to the first angle and the first distance, the target obstacle closest to the position of the vehicle body is determined.

Further, after the step of judging whether the first polar coordinates and the second polar coordinates are within a preset angle range, the processor 1001 may be used to call a ground obstacle detection program stored in the memory 1005, and perform the following operations :

If the first polar coordinate and the second polar coordinate are not within the preset angle range, the second angle and the second distance between the first polar coordinate and the camera device, and the second polar coordinate are determined A third angle and a third distance between the coordinates and the camera device;

The element group formed between the second angle and the second distance is compared with the element group formed between the third angle and the third distance to generate a comparison result, and the comparison result is determined according to the comparison result. The target obstacle closest to the location of the vehicle body.

Further, the step of projecting each of the obstacles in a preset direction to generate the projection height of each of the obstacles includes:

Read the installation height, installation angle, field of view angle, and number of effective pixel rows of the camera device, and use each obstacle as a point to be measured, and perform the following steps for each point to be measured one by one:

Detecting the measured depth value between the camera device and the point to be measured, and the number of pixel rows where the point of the point to be measured is located;

Determine the deflection angle of the pixel row corresponding to the point to be measured according to the installation angle, the field of view angle, the number of effective pixel rows and the number of pixel rows where the points are located;

Determine the projection median value of the obstacle in the preset direction according to the measured depth value and the deflection angle of the row where the pixel is located;

According to the installation height and the projection intermediate value, the projection height of each of the obstacles is generated.

Further, the step of identifying the first contour of the first obstacle in a preset direction includes:

Read the projection image of the first obstacle in the preset direction, and sequentially perform average filtering, edge extraction, contour search, and polyline fitting processing on the projection image to obtain the first contour.

Further, the step of performing texture edge detection on the second obstacle to generate a second contour of the second obstacle includes:

Acquiring an imaging image corresponding to the second obstacle based on the camera device, and invoking a first preset algorithm to perform fill-in processing on the imaging image;

Calling a second preset algorithm, extracting the initial contour of the imaged image that has been leveled, and determining the second contour of the second obstacle according to the initial contour.

Further, the step of dividing each of the obstacles into a first obstacle and a second obstacle according to each of the projection heights includes:

Comparing the projection height of each obstacle with the projection threshold one by one, and determining the target projection height of each projection height that is greater than the projection threshold;

According to the target projection height, each of the obstacles is divided into the first obstacle and the second obstacle.

Based on the above structure, various embodiments of the ground obstacle detection method are proposed.

Referring to FIG. 2, FIG. 2 is a schematic flowchart of a first embodiment of a ground obstacle detection method according to the present application.

The embodiment of the application provides an embodiment of a ground obstacle detection method. It should be noted that although the logical sequence is shown in the flowchart, in some cases, the sequence shown here may be executed in a different order than here. Or the steps described.

Specifically, the ground obstacle detection method includes:

Step S10, when an obstacle is detected based on the camera device installed on the vehicle body, project each of the obstacles in a preset direction to generate the projection height of each of the obstacles;

The ground obstacle detection method of this embodiment is applied in the process of intelligent automatic driving to detect the obstacles in the driving path and its surroundings to ensure the safety of driving; among them, intelligent automatic driving can be applied to warehouse freight transportation in a closed environment. It can also be applied to road transportation in an open environment. In this embodiment, warehouse freight is taken as an example for illustration. Specifically, a camera device is installed on the body of a vehicle that realizes automatic driving, and the camera device is preferably a stereo camera; while the vehicle is driving, the stereo camera scans the surrounding environment in real time to determine whether there are obstacles in the driving path and its surroundings. Things.

In addition, this embodiment has a pre-established three-dimensional space coordinate system. The three-dimensional space coordinate system uses the position of the stereo camera as the coordinate origin, the plane where the vehicle is located as the XY plane, and the upper space perpendicular to the XY plane is the space where the positive direction of the Y axis is located. ; Among them, for the XY plane, the direction of the Y-axis is the direction directly in front of the vehicle and the direction perpendicular to the Y-axis on the right side of the vehicle is the X-axis direction. Taking the positive direction of the Y axis as the preset direction, once the camera detects an obstacle, it will image each obstacle to form a projection of each obstacle in the preset direction, and detect the projection height of each obstacle.

Step S20, dividing each of the obstacles into a first obstacle and a second obstacle according to each of the projection heights, and identifying a first contour of the first obstacle in a preset direction;

Further, the projection height of each obstacle characterizes the height of each obstacle. In order to distinguish between high obstacles and short obstacles, a projection threshold for dividing high obstacles and low obstacles is preset, based on each obstacle The size relationship between the projection height and the projection threshold can divide each obstacle into a first obstacle and a second obstacle; the first obstacle is a high obstacle, and the second obstacle is a short obstacle. Specifically, the step of dividing each obstacle into a first obstacle and a second obstacle according to each projection height includes:

Step S21, comparing the projection height of each obstacle with the projection threshold one by one, and determining the target projection height of each of the projection heights that is greater than the projection threshold;

Step S22: Divide each of the obstacles into the first obstacle and the second obstacle according to the target projection height.

In order to determine the size relationship between the projection height of each obstacle and the projection threshold, the projection height of each obstacle is compared with the projection threshold one by one, and the target projection height that is greater than the projection threshold is selected from the projection heights; and then the obstacles are selected. The obstacle with the target projection height is determined as the first obstacle, and the other obstacles are determined as the second obstacle.

Further, in order to ensure that the preset projection threshold value can enable the camera device to accurately and comprehensively image various obstacles, the projection threshold value needs to be adjusted multiple times, and the adjustment is carried out before the comparison; specifically, according to the projection height, the obstacles The step of dividing the object into the first obstacle and the second obstacle includes:

Step a1, after receiving the set value in the preset direction, acquire a first camera image corresponding to the set value based on the camera device, and determine whether the first camera image is valid;

Step a2, if the first captured image is valid, adjust the set value, and acquire a second captured image corresponding to the adjusted set value based on the camera device;

Step a3, when the second camera image is valid, update the adjusted setting value to the setting value to be verified according to a preset interference factor;

Step a4, verifying the setting value to be verified, and determining the setting value to be verified as the projection threshold after the verification of the setting value to be verified is successful.

Specifically, after the stereo camera is installed on the vehicle body, the vehicle is driven to a level ground, and the projection threshold is set and adjusted. The installation height of the stereo camera is detected, and the setting value of the vehicle in the preset direction is set to 0, and the error range of the setting value is within plus or minus 5% of the installation height. After receiving the setting value, the ground in front of the vehicle is photographed by a stereo camera, and the photographing range is different depending on the setting value, and the photographed image is taken as the first photographed image corresponding to the setting value. After that, the validity of the first camera image is judged, and the validity judgment is to judge whether all the ground in the front is present in the image; if all of the ground is present in the image, it is judged that the first camera image is valid, and the set value is adjusted. Set to a value greater than 0. If the first captured image is determined to be invalid, the installation height, installation angle, and field of view angle of the stereo camera are adjusted to ensure that all the first captured image is presented in the image.

Further, after the setting value is set to a value greater than 0, the ground in front of the vehicle is also photographed by a stereo camera to obtain a second photographed image corresponding to the adjusted setting value, and the second photographed image is verified The validity of the validity verification is a process of judging whether the projection of the preset direction on the ground in front has completely disappeared. If it disappears completely, it is determined that the second camera image is valid, combined with preset interference factors such as vehicle body tilt and shaking, increase the value of the set value, and update the adjusted set value to the set value to be verified to ensure the prediction of the ground in front. Set the direction projection will not appear in the captured image. After that, verify the set value to be verified, and determine whether the obstacles on the ground ahead that are higher than the set value are all present in the field of view of the stereo camera; if they are all present in the field of view of the stereo camera, the verification is determined to be successful, and the The verification setting is determined as the projection threshold to distinguish between tall obstacles and short obstacles.

Furthermore, after the first obstacle is obtained by dividing, the first contour formed by the first obstacle in the preset direction is identified, and the first contour formed in the preset direction is the first obstacle The outer contour facing the direction of the car body. Wherein, the step of identifying the first contour of the first obstacle in the preset direction includes:

Step S23: Read a projection image of the first obstacle in a preset direction, and sequentially perform mean filtering, edge extraction, contour search, and polyline fitting processing on the projection image to obtain the first contour.

For the first obstacle, read its projection map in the preset direction, and the projection map reflects the contour curve of the high obstacle. In order to extract the contour curve, the projection image is first subjected to mean filtering to filter out the impurity points on the ground that are not obstructed in the stereo camera imaging; after that, the edge extraction of the projection image is performed to obtain the high obstacle in the projection image The formed edge pixels are then contour searched on the edge pixels to obtain contour points of high obstacles. By initially extracting the edge pixels of the high obstacle in the projection image, and then searching for the contour points based on the preliminary extracted edge pixels, the accuracy of the obtained contour points can be ensured. After that, a polyline fitting process is performed on each contour point to generate a first contour of the first obstacle in a preset direction. Please refer to Figure 6. In Figure 6, the number 1 is the origin of the stereo camera, the number 1.3 is the installation height of the stereo camera, the number 2.1 is the projection height of the first obstacle on the Z axis, and the number 2.2 is the second obstacle on the Z axis. The projection height, number 4 is the projection threshold, number 5.1 is the projection contour of the second obstacle on the Z axis, and number 5.2 is the first contour of the first obstacle on the Z axis.

Step S30: Perform texture edge detection on the second obstacle to generate a second contour of the second obstacle;

Further, taking into account the characteristics of the obvious difference between the imaging of the short obstacle and the ground texture, the second obstacle is detected by texture edge detection to generate its second contour; the local Fourier transform is used to find the ground The different positions where the frequency features are distributed are the contours imaged by the ground short obstacles, so as to obtain the second contour of the second short obstacles.

Step S40: Determine the target obstacle closest to the position of the vehicle body according to the first contour and the second contour.

Furthermore, the distances between different obstacles and the vehicle body are different, and the closer the distance is, the more likely a collision risk will occur. The first contour reflects the distance between high ground obstacles and the car body, and the second contour reflects the distance between low ground obstacles and the car body; in order to determine the obstacles that are most likely to collide with the car body, you can first The first contour and the second contour are respectively converted into polar coordinates relative to the origin of the coordinates. The polar coordinates of the two are used to determine the target obstacle that is closest to the position of the vehicle body; and then the distance between the target obstacle and the vehicle body is determined Determine whether obstacle avoidance is required to ensure the safety of the vehicle during automatic driving. Please refer to Figure 6. In Figure 6, number 1 is the coordinate origin of the stereo camera, number 5.1 is the projection profile of the second obstacle on the Z axis, number 5.2 is the first profile of the first obstacle on the Z axis, and number 5.3 is The second contour of the second obstacle, number 5.4 is the projection contour of the first obstacle on the XY plane, the number 6.1 is the contour line of the polar coordinate method of the second contour, and the number 6.2 is the contour line of the polar coordinate method of the first contour .

In this embodiment, when an obstacle is detected by the camera device installed on the vehicle body, the obstacle is first projected in a preset direction, and each obstacle is divided by the height of each obstacle represented by the projection height. Are the first obstacle and the second obstacle; after that, the first obstacle is identified in the preset direction for the first contour, and the texture edge detection of the second obstacle is performed to obtain the second contour; and then based on the first contour And the position of the second contour, determine the target obstacle closest to the position of the vehicle body. Since the recognized first contour has little correlation with the angle installed by the camera, vehicle body shaking will not interfere with image processing, ensuring the accuracy of recognition; while the second contour is generated based on texture edge detection without lowering the ground The accuracy can realize the detection of low obstacles, can fully identify various obstacles in the transportation environment, and have strong environmental compatibility.

Further, a second embodiment of the ground obstacle detection method of the present application is proposed.

Referring to Fig. 3, Fig. 3 is a schematic flowchart of a second embodiment of a ground obstacle detection method according to the present application.

The difference between the second embodiment of the ground obstacle detection method and the first embodiment of the ground obstacle detection method is that the position of the vehicle body is determined according to the first contour and the second contour. The steps to the nearest target obstacle include:

Step 41: Convert the first contour to a first polar coordinate, convert the second contour to a second polar coordinate, and determine whether the first polar coordinate and the second polar coordinate are within a preset angle range;

Furthermore, in this embodiment, in the process of respectively converting the first contour and the second contour into polar coordinates relative to the origin of the coordinates, the first contour is converted into the first polar coordinate, and the second contour is converted into the second pole. coordinate. Since both the first contour and the second contour are composed of a plurality of pixel points, the converted first polar coordinate and the second polar coordinate are essentially a polar coordinate set composed of the polar coordinate values of the plurality of pixel points. Wherein, the step of converting the first contour to the first polar coordinate includes:

Step 411: Read the installation height, the installation angle, the vertical field of view, the horizontal field of view, the number of effective pixel rows and the number of effective pixel columns of the camera device;

Step 412: Read all the pixel points in the first contour as measurement points, and perform the following steps for each measurement point one by one:

Step 413: Detect the depth value between the measurement point and the camera device, and the number of pixel rows and pixel columns of the measurement point;

Step 414: Determine the polar coordinate modulus of the measurement point according to the installation angle, the vertical field of view angle, the number of pixel rows, the number of effective pixel rows, and the depth value;

Step 415: Determine the polar coordinates of the measurement point according to the horizontal field of view angle, installation height, installation angle, vertical field of view angle, the number of pixel columns, the number of effective pixel columns, the number of pixel rows and the number of effective pixel rows. angle;

Step 416: Set the polar coordinate modulus and the polar coordinate angle as the polar coordinate of the measuring point, and after generating the polar coordinate for each measuring point, form each of the polar coordinates as a first pole. coordinate.

Further, the installation parameters of the stereo camera are read, and the installation parameters include the installation height H, the installation angle θ, the vertical field of view angle ω _z , the horizontal field of view angle ω _h , the number of effective pixel rows L and the number of effective pixel columns C; The number of effective pixel rows is the maximum imaging pixel value of the stereo camera in the Y-axis direction, and the effective pixel column number is the maximum imaging pixel value of the stereo camera in the X-axis direction. Thereafter, each pixel point contained in the first contour is read as a measurement point, and processing is performed on each measurement point one by one. During processing, first detect the depth value D between the measuring point and the camera device, and the number of pixel rows n and the number of pixel columns m of the measuring point; then the installation angle θ, the vertical field of view angle ω _z , and the number of pixel rows n And the number of effective pixel rows L is transferred to formula (1), and calculated by formula (1), the deflection angle α of the pixel row is obtained; where formula (1) is:

α=θ-(ω _z /2)+(ω _z *n/L) (1);

After the deflection angle α of the pixel row is calculated by formula (1), the deflection angle α and the depth value D are transmitted to formula (2), and the polar coordinate modulus r of the measuring point is calculated by formula (2); The formula (2) is:

r=D*Cos(α) (2).

Further, the absolute value coordinates (|Xmax|, |Ymax|) of the farthest projection point imaged by the stereo camera and the absolute value coordinates (|Xmin|, |Ymin|) of the nearest projection point are calculated; specifically, the horizontal The field of view angle ω _h , the installation height H, the installation angle θ, and the vertical field of view angle ω _{z are} transmitted to formula (3). Through the calculation of formula (3), the absolute value of the coordinates of the farthest projection point is obtained as |Xmax| ; Transmit the installation height H, the installation angle θ, and the vertical field of view angle ω _z into formula (4), and obtain the value of |Ymax| in the absolute coordinates of the farthest projection point through the calculation of formula (4); at the same time, The horizontal field of view angle ω _h , the installation height H, the installation angle θ, and the vertical field of view angle ω _{z are} transmitted to formula (5), and the value of |Xmin| in the absolute coordinates of the most recent projection is obtained through the calculation of formula (5); The installation height H, the installation angle θ, and the vertical field of view angle ω _{z are} transmitted to formula (6), and the value of |Ymin| in the absolute coordinate of the most recent projection is obtained through the calculation of formula (6). The formulas (3), (4), (5), (6) are:

|Xmax|=Tan(0.5*ω _h )*H/Cos(θ-0.5*ω _z ) (3);

|Ymax|=H/Tan(θ-0.5*ω _z ) (4);

|Xmin|=Tan(0.5*ω _h )*H/Cos(θ+0.5*ω _z ) (5);

|Ymin|=H/Tan(θ+0.5*ω _z ) (6).

Furthermore, the absolute value coordinates (|Xc|, |Yc|) of the coordinates of the measuring point are calculated, and the number of pixel columns m, the number of effective pixel columns C, |Xmax| and |Xmin| are transferred to formula (7) In the calculation of formula (7), obtain the value of |Xc| in the absolute value of the coordinate of the measuring point, and transfer the number of pixel rows n, the number of effective pixel rows L, |Ymax| and |Ymin| to formula (8) In, through the calculation of formula (8), the value of |Yc| in the absolute value of the coordinates of the measuring point is obtained; where formulas (7) and (8) are respectively:

|Xc|=m/C*(|Xmax|-|Xmin|)+|Xmin|(7);

|Yc|=n/L*(|Ymax|-|Ymin|)+|Ymin| (8).

After that, the absolute value of the coordinates of the measuring point is transferred to the formula (9), and the polar coordinate angle of the measuring point is obtained through the calculation of the formula (9), where the formula (9) is:

=Tan-1(|Yc|/|Xc|) (9).

Understandably, the polar coordinate modulus and polar coordinate angle of the measuring point calculated by the above formulas (1) to (9) form the polar coordinates of the measuring point; each measuring point passes through formulas (1) to (9) After calculating the polar coordinates of each measuring point, each polar coordinate forms the first polar coordinate, and the first contour is completely converted into the first polar coordinate. It should be noted that the process of converting the second contour into the second polar coordinate is the same as the above-mentioned process of converting the first contour into the first polar coordinate, and will not be repeated here.

Step 42: If it is within a preset angle range, merge the first polar coordinate and the second polar coordinate to generate a polar coordinate set;

Step 43: Determine the target obstacle that is closest to the position of the vehicle body according to the set of polar coordinates.

Considering the uncertainty of the obstacles in the driving process of the vehicle, there may be a situation where multiple obstacles are located very close. At this time, the obstacles with close positions can be combined and processed. Specifically, a preset angle range that characterizes the proximity of the position is preset, such as 5 degrees; it is determined whether the first polar coordinate and the second polar coordinate are both within the preset angle range, and if they are within the preset angle range, the The first polar coordinate and the second polar coordinate are merged, and the first polar coordinate and the second polar coordinate are imaged on the same layer to form a coordinate set. The coordinate set contains each polar coordinate point of the first polar coordinate and each polar coordinate point of the second polar coordinate. It is necessary to denoise the impurity points in it, and merge the denoised effective points to determine the difference with the car body The closest obstacle to the target. Specifically, according to the polar coordinate set, the step of determining the target obstacle closest to the position of the vehicle body includes:

Step 431: Perform median filtering, denoising, and mean filtering on each element in the polar coordinate set in sequence to generate a processing result;

Step 432: Combine the elements in the processing result to generate a target element, and calculate a first angle and a first distance corresponding to the target element and the camera device;

Step 433: Determine the target obstacle that is closest to the position of the vehicle body according to the first angle and the first distance.

Further, each polar coordinate point in the polar coordinate set is taken as each element in the polar coordinate set; the median filtering process is performed on each element first to remove the salt and pepper noise points; and then by setting the minimum value of the distance from the origin of the coordinate, Among the elements, the elements whose distance from the origin of the coordinates are greater than the minimum value are removed; after that, the average filtering process is performed on the processed elements to generate the processing result. Furthermore, the elements in the processing result are merged, and the polar coordinate points as the elements are merged into one polar coordinate point, and the polar coordinate point obtained by the merge is the target element. Thereafter, the first angle and the first distance of the target element relative to the origin of the coordinate are calculated, and the relative distance between the vehicle body and the combined obstacles is represented by the first distance and the first angle; Obstacles and the relative distances between the obstacles and the vehicle body that have not been merged are used to determine the target obstacle that is the closest to the position of the vehicle body among multiple obstacles.

Furthermore, for the situation where the locations of multiple obstacles are far apart, each obstacle needs to be processed separately; specifically, the determining whether the first polar coordinate and the second polar coordinate are located at a preset angle After the steps in the scope include:

Step 44: If the first polar coordinate and the second polar coordinate are not within a preset angle range, determine the second angle and the second distance between the first polar coordinate and the camera device, and the A third angle and a third distance between the second polar coordinate and the camera device;

Step 45: Compare the element group formed between the second angle and the second distance with the element group formed between the third angle and the third distance to generate a comparison result, and compare the result according to the comparison As a result, the target obstacle closest to the position of the vehicle body is determined.

When it is determined that the first polar coordinate and the second polar coordinate are not within the preset angle range, and it is necessary to deal with a single obstacle, each element in the first polar coordinate and each element in the second polar coordinate are performed separately. Median filtering, denoising and mean filtering, and merge the processed elements in the first polar coordinate to generate the first element point, and merge the processed elements in the second polar coordinate to generate the second element point. Thereafter, calculate the angle and distance between the first element point and the origin of coordinates, that is, the second angle and the second distance between the first polar coordinate and the camera device, and calculate the sum of the angles between the second element point and the origin of coordinates The distance is the third angle and the third distance between the second polar coordinate and the camera.

Further, the second angle and the second distance are formed into an element group, and the third angle and the third distance are formed into an element group, and the two elements are compared to generate the respective distances representing the first polar coordinates and the second polar coordinates. The comparison result of the distance of the vehicle body is used to determine the target obstacle that is closest to the position of the vehicle body based on the comparison result.

In this embodiment, the obstacles that are close in position are processed together, and the obstacles that are far apart are processed separately; it avoids that each obstacle is processed separately to increase the amount of calculation, but does not contribute much to the resolution of obstacle avoidance; While the obstacles are comprehensively detected, the detection efficiency is improved. In addition, the processing methods of converting the contour to polar coordinates, filtering, denoising and calculating, improve the accuracy of calculating the distance between each obstacle and the vehicle body, thereby making the detection of target obstacles more accurate.

Further, a third embodiment of the ground obstacle detection method of the present application is proposed.

Referring to Fig. 4, Fig. 4 is a schematic flowchart of a third embodiment of a ground obstacle detection method according to the present application.

The difference between the third embodiment of the ground obstacle detection method and the first or second embodiment of the ground obstacle detection method is that the projection of each of the obstacles in a preset direction is used to generate each The steps of the projection height of the obstacle include:

Step S11: Read the installation height, installation angle, field of view angle and number of effective pixel rows of the camera device, and use each obstacle as a point to be measured, and perform the following steps for each point to be measured one by one:

Step S12, detecting the measured depth value between the camera device and the point to be measured, and the number of pixel rows where the point of the point to be measured is located;

Step S13: Determine the deflection angle of the pixel row corresponding to the point to be measured according to the installation angle, the field of view angle, the number of effective pixel rows, and the number of pixel rows where the point is located;

Step S14: Determine the projection intermediate value of the obstacle in the preset direction according to the measured depth value and the deflection angle of the row where the pixel is located;

Step S15: Generate the projection height of each obstacle according to the installation height and the projection intermediate value.

This embodiment realizes that each obstacle is projected in a preset direction, and the projection height of each obstacle is obtained, so that each obstacle is divided into a first obstacle and a second obstacle through each projection height. First, read the installation height H, installation angle θ, field of view angle ω, and number of effective pixel rows L of the stereo camera; then each obstacle is taken as the point to be measured, and each point to be measured is processed one by one. When processing, first detect the measured depth value D'between the point to be measured and the camera device, and the number of pixel rows of the point to be measured n'; then the installation angle θ, the angle of view ω, and the number of pixel rows of the point n' And the number of effective pixel rows L is transferred to formula (10), and calculated by formula (10), the deflection angle α'of the pixel row corresponding to the point to be measured is obtained; where formula (10) is:

α'=θ-(ω/2)+(ω*n'/L) (10);

After the deflection angle α'of the row of pixels corresponding to the point to be measured is calculated by formula (10), the deflection angle α'and the measured depth value D'are transmitted to formula (11), which is calculated by formula (11), Obtain the median projection h _{c of the} obstacle in the preset direction; where the formula (11) is:

h _c =D'*Sin(α') (11);

After that, a difference is made between the installation height H and the projection intermediate value h _c , and the result of the difference is the projection height h _{z of} the obstacle. Each point to be measured is processed and calculated as described above, and each calculation result obtained is the projection height of each obstacle, and then each obstacle is divided into a first obstacle and a second obstacle according to each projection height.

In this embodiment, the height of each obstacle is characterized by calculating the projection height of each obstacle in a preset direction, and then each obstacle is divided into a first obstacle and a second obstacle, so as to target different types of obstacles. Different methods are adopted to obtain the outline of the obstacle, so that the detection of the obstacle is more comprehensive and accurate.

Further, a fourth embodiment of the ground obstacle detection method of the present application is proposed.

The difference between the fourth embodiment of the ground obstacle detection method and the first, second, or third embodiment of the ground obstacle detection method is that the texture edge detection is performed on the second obstacle to generate the The steps of the second contour of the second obstacle include:

Step S31: Acquire an imaging image corresponding to the second obstacle based on the camera device, and call a first preset algorithm to perform fill-in processing on the imaging image;

Step S32: Invoking a second preset algorithm, extracting the initial contour of the imaged image after the leveling process, and determining the second contour of the second obstacle according to the initial contour.

In this embodiment, texture edge detection is performed on the second obstacle to obtain the second contour of the second obstacle. A first preset algorithm and a second preset algorithm are preset. The first preset algorithm is preferably a data filling method, and the second preset algorithm is a local Fourier transform to pass the first preset algorithm and the second preset algorithm. The algorithm processes the imaging image of the second obstacle. Specifically, first use a stereo camera to shoot the second obstacle to obtain an imaged image; considering the effects of various factors such as the stereo camera and ambient light in the imaged image, there may be some pixels that have no data; this category has no data Pixels will cause the calculation of the second preset algorithm to fail, and need to be processed by the first preset algorithm. The data filling method as the first preset algorithm includes expansion and median filtering. The imaging image is expanded by the expansion algorithm, the hole data in the imaging image is filled in, and the impurities in the imaging image are eliminated by median filtering. Remove. After that, the second preset algorithm is called to process the imaged image after the fill-in process, and the 16*16 pixel recognition area is used as the smallest recognition unit to segment the imaged image to extract the target pixel where the contour edge of the imaged image is located. Area; Fourier transform is performed for the target pixel identification area by formulas (12) and (13), and then the complex number is converted into amplitude value by formula (14), and then the amplitude value is switched to logarithmic scale by formula (15), To scale the image on a logarithmic scale. The formulas (12), (13), (14) and (15) are:

e ^ix = cos x+i sin x (13);

M1=log ^(1+M) (15);

Among them, F(k,l) is the value of the target pixel recognition area of the imaged image in the frequency domain, f(i,j) is the value of the target pixel recognition area of the imaged image in the space domain, and M is the value of F(k,l) Amplitude; the above formulas (12) to (15) are often used to implement Fourier transform, so I won’t repeat them here.

Further, cut and redistribute the amplitude image, rearrange the quadrants of the result so that the origin of the coordinate corresponds to the center of the image, and then perform normalization processing to process the values beyond the display range to facilitate classification judgment, and then to identify edges that contain Zone classification, and regenerate contours, so as to extract the initial contour of the second obstacle from the ground texture.

Furthermore, the color block expansion process is performed on the initially extracted contour, and the edge is extracted from the original imaging image according to the processing result, and then the contour is accurately found from the extracted edge to obtain the second contour of the second obstacle .

In this embodiment, for the low second obstacle, the contour is extracted according to the characteristics of the obvious difference between it and the ground texture, and then the recognition and detection are performed. High obstacles and low obstacles are processed in different ways. While identifying high obstacles, it realizes the detection of low obstacles, ensuring the comprehensiveness and accuracy of the detection of various obstacles in the vehicle body driving environment. .

It should be noted that those of ordinary skill in the art can understand that all or part of the steps in the above-mentioned embodiments can be completed by hardware, or by a program to instruct related hardware, and the program can be stored in a computer readable Among the storage media, the aforementioned storage media may be read-only memory, magnetic disks, or optical disks.

In addition, an embodiment of the present application also proposes a computer-readable storage medium having a ground obstacle detection program stored on the computer-readable storage medium, and when the ground obstacle detection program is executed by a processor, the above-mentioned ground obstacle detection program is implemented. Steps of obstacle detection method.

The specific implementation of the computer-readable storage medium of the present application is basically the same as the foregoing embodiments of the ground obstacle detection method, and will not be repeated here.

It should be noted that in this article, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, It also includes other elements not explicitly listed, or elements inherent to the process, method, article, or device. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or device that includes the element.

The serial numbers of the foregoing embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

Through the description of the above implementation manners, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。 Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, The optical disc) includes several instructions to enable a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the method described in each embodiment of the present application.

The above are only the preferred embodiments of the application, and do not limit the scope of the patent for this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of the application, or directly or indirectly applied to other related technical fields , The same reason is included in the scope of patent protection of this application.

Claims

A ground obstacle detection method, wherein the ground obstacle detection method includes the following steps:

When an obstacle is detected based on the camera device installed on the vehicle body, projection of each of the obstacles in a preset direction is performed to generate the projection height of each of the obstacles;

Dividing each of the obstacles into a first obstacle and a second obstacle according to each of the projection heights, and identifying a first contour of the first obstacle in a preset direction;

Performing texture edge detection on the second obstacle to generate a second contour of the second obstacle;

According to the first contour and the second contour, the target obstacle closest to the position of the vehicle body is determined.
The ground obstacle detection method according to claim 1, wherein the step of determining the target obstacle closest to the position of the vehicle body according to the first contour and the second contour comprises:

Converting the first contour to first polar coordinates, converting the second contour to second polar coordinates, and determining whether the first polar coordinates and the second polar coordinates are within a preset angle range;

If it is within the preset angle range, combining the first polar coordinate and the second polar coordinate to generate a polar coordinate set;

According to the set of polar coordinates, a target obstacle that is closest to the position of the vehicle body is determined.
3. The ground obstacle detection method according to claim 2, wherein the step of converting the first contour to the first polar coordinate comprises:

Reading the installation height, installation angle, vertical field of view, horizontal field of view, number of effective pixel rows and effective pixel columns of the camera device;

Read all the pixels in the first contour as measurement points, and perform the following steps for each measurement point one by one:

Detecting the depth value between the measurement point and the camera device, and the number of pixel rows and pixel columns of the measurement point;

Determine the polar coordinate modulus of the measurement point according to the installation angle, the vertical field of view angle, the number of pixel rows, the number of effective pixel rows, and the depth value;

Determine the polar coordinate angle of the measurement point according to the horizontal field of view angle, the installation height, the installation angle, the vertical field of view angle, the number of pixel columns, the number of effective pixel columns, the number of pixel rows and the number of effective pixel rows;

The polar coordinate modulus value and the polar coordinate angle are set as the polar coordinates of the measuring point, and after the polar coordinates are generated for each measuring point, each of the polar coordinates is formed as the first polar coordinate.
The method for detecting ground obstacles according to claim 2, wherein the step of determining the target obstacle closest to the position of the vehicle body according to the set of polar coordinates comprises:

Performing median filtering, denoising, and mean filtering on each element in the polar coordinate set in sequence to generate a processing result;

Combine the elements in the processing result to generate a target element, and calculate a first angle and a first distance corresponding to the target element and the camera device;

According to the first angle and the first distance, the target obstacle closest to the position of the vehicle body is determined.
3. The ground obstacle detection method according to claim 2, wherein after the step of judging whether the first polar coordinate and the second polar coordinate are within a preset angle range comprises:

If the first polar coordinate and the second polar coordinate are not within the preset angle range, the second angle and the second distance between the first polar coordinate and the camera device, and the second polar coordinate are determined A third angle and a third distance between the coordinates and the camera device;

The element group formed between the second angle and the second distance is compared with the element group formed between the third angle and the third distance to generate a comparison result, and the comparison result is determined according to the comparison result. The target obstacle closest to the location of the vehicle body.
5. The ground obstacle detection method according to claim 1, wherein the step of projecting each of the obstacles in a preset direction to generate the projection height of each of the obstacles comprises:

Read the installation height, installation angle, field of view angle, and number of effective pixel rows of the camera device, and use each obstacle as a point to be measured, and perform the following steps for each point to be measured one by one:

Detecting the measured depth value between the camera device and the point to be measured, and the number of pixel rows where the point of the point to be measured is located;

Determine the deflection angle of the pixel row corresponding to the point to be measured according to the installation angle, the field of view angle, the number of effective pixel rows, and the number of pixel rows where the points are located;

Determine the projection median value of the obstacle in the preset direction according to the measured depth value and the deflection angle of the row where the pixel is located;

According to the installation height and the projection intermediate value, the projection height of each of the obstacles is generated.
The method for detecting ground obstacles according to claim 1, wherein the step of identifying the first contour of the first obstacle in a preset direction comprises:

Read the projection image of the first obstacle in the preset direction, and sequentially perform average filtering, edge extraction, contour search, and polyline fitting processing on the projection image to obtain the first contour.
5. The ground obstacle detection method according to claim 1, wherein the step of performing texture edge detection on the second obstacle to generate a second contour of the second obstacle comprises:

Acquiring an imaging image corresponding to the second obstacle based on the camera device, and invoking a first preset algorithm to perform fill-in processing on the imaging image;

Calling a second preset algorithm, extracting the initial contour of the imaged image that has been leveled, and determining the second contour of the second obstacle according to the initial contour.
5. The ground obstacle detection method according to claim 1, wherein the step of dividing each of the obstacles into a first obstacle and a second obstacle according to each of the projection heights comprises:

Comparing the projection height of each obstacle with the projection threshold one by one, and determining the target projection height of each projection height that is greater than the projection threshold;

According to the target projection height, each of the obstacles is divided into the first obstacle and the second obstacle.
A ground obstacle detection device, wherein the ground obstacle detection device includes a memory, a processor, and a ground obstacle detection program stored on the memory and running on the processor, and the ground obstacle When the detection program is executed by the processor, the following steps are implemented:

When an obstacle is detected based on the camera device installed on the vehicle body, projection of each of the obstacles in a preset direction is performed to generate the projection height of each of the obstacles;

Dividing each of the obstacles into a first obstacle and a second obstacle according to each of the projection heights, and identifying a first contour of the first obstacle in a preset direction;

Performing texture edge detection on the second obstacle to generate a second contour of the second obstacle;

According to the first contour and the second contour, the target obstacle closest to the position of the vehicle body is determined.
The ground obstacle detection device according to claim 10, wherein the step of determining the target obstacle closest to the position of the vehicle body according to the first contour and the second contour comprises:

Converting the first contour to first polar coordinates, converting the second contour to second polar coordinates, and determining whether the first polar coordinates and the second polar coordinates are within a preset angle range;

If it is within the preset angle range, combining the first polar coordinate and the second polar coordinate to generate a polar coordinate set;

According to the set of polar coordinates, a target obstacle that is closest to the position of the vehicle body is determined.
The ground obstacle detection device according to claim 11, wherein the step of converting the first contour into a first polar coordinate comprises:

Reading the installation height, installation angle, vertical field of view, horizontal field of view, number of effective pixel rows and effective pixel columns of the camera device;

Read all the pixels in the first contour as measurement points, and perform the following steps for each measurement point one by one:

Detecting the depth value between the measurement point and the camera device, and the number of pixel rows and pixel columns of the measurement point;

Determine the polar coordinate modulus of the measurement point according to the installation angle, the vertical field of view angle, the number of pixel rows, the number of effective pixel rows, and the depth value;

Determine the polar coordinate angle of the measurement point according to the horizontal field of view angle, the installation height, the installation angle, the vertical field of view angle, the number of pixel columns, the number of effective pixel columns, the number of pixel rows and the number of effective pixel rows;

The polar coordinate modulus value and the polar coordinate angle are set as the polar coordinates of the measuring point, and after the polar coordinates are generated for each measuring point, each of the polar coordinates is formed as the first polar coordinate.
The ground obstacle detection device according to claim 11, wherein the step of determining the target obstacle closest to the position of the vehicle body according to the set of polar coordinates comprises:

Performing median filtering, denoising, and mean filtering on each element in the polar coordinate set in sequence to generate a processing result;

Combine the elements in the processing result to generate a target element, and calculate a first angle and a first distance corresponding to the target element and the camera device;

According to the first angle and the first distance, the target obstacle closest to the position of the vehicle body is determined.
11. The ground obstacle detection device according to claim 11, wherein after the step of judging whether the first polar coordinate and the second polar coordinate are within a preset angle range, comprises:

If the first polar coordinate and the second polar coordinate are not within the preset angle range, the second angle and the second distance between the first polar coordinate and the camera device, and the second polar coordinate are determined A third angle and a third distance between the coordinates and the camera device;

The element group formed between the second angle and the second distance is compared with the element group formed between the third angle and the third distance to generate a comparison result, and the comparison result is determined according to the comparison result. The target obstacle closest to the location of the vehicle body.
A computer-readable storage medium, wherein a ground obstacle detection program is stored on the computer-readable storage medium, and the following steps are implemented when the ground obstacle detection program is executed by a processor:

When an obstacle is detected based on the camera device installed on the vehicle body, projection of each of the obstacles in a preset direction is performed to generate the projection height of each of the obstacles;

Dividing each of the obstacles into a first obstacle and a second obstacle according to each of the projection heights, and identifying a first contour of the first obstacle in a preset direction;

Performing texture edge detection on the second obstacle to generate a second contour of the second obstacle;

According to the first contour and the second contour, the target obstacle closest to the position of the vehicle body is determined.