WO2023050679A1

WO2023050679A1 - Obstacle detection method and apparatus, and computer device, storage medium, computer program and computer program product

Info

Publication number: WO2023050679A1
Application number: PCT/CN2022/075423
Authority: WO
Inventors: 俞煌颖; 傅东旭; 王哲
Original assignee: 上海商汤智能科技有限公司
Priority date: 2021-09-30
Filing date: 2022-02-07
Publication date: 2023-04-06
Also published as: CN113887433A

Abstract

Provided in the embodiments of the present disclosure are an obstacle detection method and apparatus, and a computer device, a storage medium, a computer program and a computer program product. The obstacle detection method comprises: on the basis of the current frame of radar scanning data obtained by means of scanning a target scenario, determining a first position point which corresponds to an object to be recognized in the target scenario; on the basis of historical position information corresponding to a historical candidate deletion object in a historical frame of radar scanning data, and the first position point, determining whether the object to be recognized is a target object to be deleted; and in response to the fact that the object to be recognized is not the target object to be deleted, determining, as an obstacle in the target scenario, the object to be recognized.

Description

Obstacle detection method, device, computer equipment, storage medium, computer program and computer program product

Cross References to Related Applications

This disclosure is based on the Chinese patent application with the application number 202111165461.0, the application date is September 30, 2021, and the application name is "A method, device, computer equipment and storage medium for detecting obstacles", and requires that the Chinese patent application The entire content of this Chinese patent application is incorporated by reference in this disclosure.

technical field

Embodiments of the present disclosure relate to, but are not limited to, the technical field of radar detection, and in particular, relate to an obstacle detection method, device, computer equipment, storage medium, computer program, and computer program product.

Background technique

When using lidar to detect the driving area, it is usually carried out by emitting laser light to the driving area and receiving the laser light reflected by the ground and obstacles. In the related art, the accuracy of using laser to detect obstacles is low.

Contents of the invention

Embodiments of the present disclosure at least provide an obstacle detection method, device, computer equipment, storage medium, computer program, and computer program product.

An embodiment of the present disclosure provides an obstacle detection method, including: based on the current frame radar scan data obtained by scanning the target scene, determining the first position point corresponding to the object to be identified in the target scene; The historical location information corresponding to the historical candidate deletion object in the radar scanning data, and the first location point, determine whether the object to be identified is the target object to be deleted; in response to the object to be identified is not the target object to be deleted The object to be recognized is determined as an obstacle in the target scene.

In this way, by combining the historical position information of historical candidate deletion objects in the historical frame radar data, it is determined whether to delete the object to be identified in the current frame radar scan data, so that the space domain and the time domain are combined to comprehensively judge the object to be identified Whether the object is the target object to be deleted is used to judge whether the object to be recognized is an obstacle in the target scene, which has higher detection accuracy.

In some implementations, the determining whether the object to be identified is the target object to be deleted based on the historical position information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first position point includes: Based on the first position point corresponding to the object to be identified, determine whether the object to be identified is a current candidate for deletion; The historical location information corresponding to the candidate deletion object and the first location point determine whether the object to be identified is the target object to be deleted. In this way, it can first be judged whether the object to be identified is the current candidate deletion object through the first location point; when it can be determined that the object to be identified is the current candidate deletion object, based on the historical location information and the first location point, a more accurate A judgment is made as to whether the object to be identified is the target object to be deleted.

In some implementations, in response to the fact that the object to be identified is not the current candidate object for deletion, the object to be identified is determined as an obstacle in the target scene. In this way, when it is judged that the object to be identified is not the current candidate for deletion, the object to be identified can be directly determined as an obstacle, which is more efficient; since the detection method provided by the embodiment of the present disclosure can more accurately determine whether the object to be identified is Delete the object for the current candidate, so it is more accurate when judging whether the object to be recognized is an obstacle in the target scene.

In some embodiments, the determining the first position point corresponding to the object to be identified in the target scene based on the current frame radar scanning data obtained by scanning the target scene includes: for each of the objects to be identified, from Determining the point cloud point corresponding to the object to be identified in the current frame radar scan data; based on the three-dimensional position information of the point cloud point corresponding to the object to be identified in the target scene, determine the corresponding the contour information; based on the contour information, determine the first location point corresponding to the object to be recognized. In this way, since the first position point is obtained by using the radar scanning data, it can not only retain the information contained in the original point cloud point, but also represent more information of the object to be identified; therefore, using the first position point instead of passing through the radar Scanning the determined point cloud points requires less calculation, consumes less computing power, and is more efficient.

In some embodiments, the determining the contour information corresponding to the object to be recognized based on the three-dimensional position information of the point cloud point corresponding to the object to be recognized in the target scene includes: corresponding to the object to be recognized The point cloud points of is projected onto a preset plane to obtain a first projection point; based on the two-dimensional position information of the first projection point in the preset plane, the contour information of the object to be recognized is determined. In this way, using the first projection point to determine the contour information of the object to be recognized, compared with the method of directly determining the contour information by using point cloud points, the calculation using two-dimensional position information is simpler, and the determined contour information of the object to be recognized Also more accurate.

In some embodiments, the determining the first position point corresponding to the object to be recognized based on the contour information includes: using the contour information of the object to be recognized to determine that the object to be recognized is in a preset plane The projection area; based on the area of the projection area, determine the first location point corresponding to the object to be identified.

In some implementations, based on the area of the projection area, determining the first position point corresponding to the object to be identified includes: comparing the area of the projection area with a preset area threshold; The area is greater than the area threshold, based on the projection area, determine the minimum bounding box of the projection area; based on the first area corresponding to the minimum bounding box and the preset first interval step, in the second Determining a plurality of candidate position points in an area; determining the candidate position point located in the projected area as the first position point. In this way, for an object to be recognized whose area of the projected area is greater than the area threshold, the method of determining its first position point using the first area determined by the minimum bounding box corresponding to its projected area can better retain the projected The location points in the area make it more accurate when using the first location point to determine whether the object to be recognized is a target object that can be deleted.

In some implementations, the determining the first location point corresponding to the object to be identified based on the area of the projection area further includes: comparing the area of the projection area with a preset area threshold; responding When the area is less than or equal to the area threshold, determine the center point located in the projection area; based on the center point and the preset radius length, determine the center point as the center, the preset radius A second area whose length is a radius; based on the second area and a preset second interval step, the first location point is determined within the second area. In this way, since the to-be-recognized object whose projected area area is smaller than the area threshold is usually small in size, when the first position point is directly determined for it, the number of determined position points may also be small. Therefore, using this method of determining the first position point, a plurality of position points related to the object to be identified can be determined; by increasing the number of position points, it is also possible to use the first position point to determine whether the object to be identified is Improves accuracy when objects are deletable.

In some implementations, the judging whether the object to be identified is a current candidate for deletion based on the first position point corresponding to the object to be identified includes: acquiring a current frame image obtained by scanning the target scene; Projecting the first position point into the current frame image to obtain a second projection point; based on the position information of the second projection point in the current frame image and the obstacles included in the current frame image At a position in the current frame image, determine whether the object to be identified is the current candidate object for deletion. In this way, using the second projection point of the first position point in the current frame image, it is relatively easy to first determine whether the object to be identified can be used as the current candidate for deletion; if it cannot be used as the current candidate for deletion, no further Handle judgment. This can further improve the detection efficiency.

In some embodiments, the number of the first position point is at least one; the projecting the first position point into the current frame image to obtain the second projected point includes: adding at least one of the first position points A position point is projected into the current frame image to obtain at least one second projection point; the position information based on the second projection point in the current frame image and the obstacles included in the current frame image At the position in the current frame image, determining whether the object to be identified is the current candidate deletion object includes: for each of the second projection points, based on the second projection point in the current frame image The position information in the current frame image and the position of the obstacle included in the current frame image, predict the obstacle prediction result corresponding to the second projection point; the obstacle prediction result includes: There is an obstacle at the position corresponding to the second projection point, or there is no obstacle; based on the obstacle prediction results corresponding to each of the second projection points, it is determined whether the object to be identified is the current candidate deletion object. In this way, due to the position information of the second projection point in the current frame image and the position of the obstacle included in the current frame image in the current frame image, it is relatively easy to obtain; and the prediction of the obstacle corresponding to the second projection point is predicted As a result, the data represented by each second projection point can be gradually fused while retaining the position information of the second projection point, so the accuracy of the obtained obstacle prediction result is also higher.

In some implementation manners, the determining whether the object to be identified is the current candidate object for deletion based on the obstacle prediction results corresponding to the second projection points respectively includes: Based on the corresponding obstacle prediction result, determine the confidence that the object to be identified is an obstacle; determine whether the object to be identified is the current candidate for deletion based on the confidence and a preset confidence threshold.

In some embodiments, there are n second projection points, and n is an integer greater than 1; based on the obstacle prediction results corresponding to each second projection point, the confidence that the object to be recognized is an obstacle is determined degree, including: traversing the 2nd to nth second projection points; for the traversed ith second projection point, based on the obstacle prediction result of the ith second projection point, determine the Criterion function corresponding to two projection points; wherein, i is a positive integer greater than 1; based on the criterion function corresponding to the i-th second projection point and the fusion criterion of the 1st to i-1th second projection points As a result, a fusion criterion result corresponding to the ith second projection point is determined; based on the fusion criterion result corresponding to the ith second projection point, a confidence degree for determining that the object to be recognized is an obstacle is obtained.

In some implementations, the determining whether the object to be identified is the target object to be deleted based on the historical position information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first position point includes: Based on the historical position information corresponding to the historical candidate object in the historical frame radar scanning data and the first position point, determine the target candidate object from the historical candidate object; based on the first position of the target candidate object in the preset plane A projection area, and a second projection area of the object to be identified on the preset plane, determine whether the object to be identified is a target object to be deleted. In this way, it is relatively simple to use the first projection area and the second projection area to judge whether the object to be recognized is the target object to be deleted, so the detection efficiency can also be improved.

In some implementations, the historical frame radar scan data includes at least one historical candidate object; based on the historical position information corresponding to the historical candidate object in the historical frame radar scan data and the first position point, from Determining target candidate objects in the historical candidate objects includes: based on the historical position information corresponding to each historical candidate object in the historical frame radar scanning data and the first position point, determining that each historical candidate object is respectively related to the target candidate object Recognizing the distance of the object: based on the distances between each historical candidate object and the object to be recognized, determine the target candidate object with the closest distance to the object to be recognized from each of the historical candidate objects. In this way, by using the distances between the historical candidate objects and the object to be recognized, the target candidate object can be determined more quickly and accurately, and further judge whether the object to be recognized is the target object to be deleted according to the determined target candidate object.

In some implementations, based on the first projection area of the target candidate object on the preset plane and the second projection area of the object to be identified on the preset plane, it is determined whether the object to be identified is The target object to be deleted includes: determining whether there is an overlapping area between the first projection area and the second projection area; in response to no overlapping area between the first projection area and the second projection area, determining the The object to be identified is the target object to be deleted.

In some implementations, based on the first projection area of the target candidate object on the preset plane and the second projection area of the object to be identified on the preset plane, it is determined whether the object to be identified is The target object to be deleted further includes: in response to the fact that there is an overlapping area between the first projection area and the second projection area in the current frame radar scan data, using the object to be identified as a new historical candidate object until the current In the consecutive N frames of radar scanning data after the frame of radar scanning data, if there is an overlapping area between the first projection area and the second projection area, it is determined that the candidate deletion object is not the target object to be deleted, and the new Delete historical candidates; N is a positive integer. In this way, it is possible to determine whether the object to be recognized is the target object to be deleted in the scan data with a limited number of frames, which is faster and more efficient, and can be applied to scenarios that require prompt and accurate responses to obstacles. Furthermore, the detection of obstacles can be made more timely, thereby further ensuring driving safety.

In some implementations, it further includes: for each of the historical candidate deletion objects, detecting the time difference between the storage time of the historical candidate deletion object and the current time; when the time difference is greater than or equal to a preset time difference threshold, Delete the historical candidate deletion object. In this way, it can also ensure that the objects to be identified to be deleted can be accurately screened out by using the historical candidate deletion objects, and the data to be stored is reduced to a certain extent, and the matching calculation of the historical candidate deletion objects and the current object to be identified is also reduced. Improve the detection efficiency of obstacles.

An embodiment of the present disclosure also provides an obstacle detection device, including:

The first determining part is configured to determine a first position point corresponding to the object to be identified in the target scene based on the current frame radar scanning data obtained by scanning the target scene;

The second determination part is configured to determine whether the object to be identified is a target object to be deleted based on the historical position information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first position point;

The third determination part is configured to determine the object to be identified as an obstacle in the target scene in response to the object to be identified not being the target object to be deleted.

In some implementations, the second determining part determines whether the object to be identified is the one to be deleted based on the historical position information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first position point. In the case of a target object, it is configured to: determine whether the object to be identified is a current candidate for deletion based on the first position point corresponding to the object to be identified; in response to the object to be identified being the current candidate for deletion and determining whether the object to be identified is a target object to be deleted based on the historical position information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first position point.

In some implementations, the second determination part is further configured to: determine the object to be identified as an obstacle in the target scene in response to the object to be identified not being the current candidate object for deletion.

In some implementations, the first determination part is configured to determine the first position point corresponding to the object to be identified in the target scene based on the current frame radar scanning data obtained by scanning the target scene. : For each of the objects to be identified, determine the point cloud points corresponding to the objects to be identified from the current frame radar scan data; based on the three-dimensional position of the point cloud points corresponding to the objects to be identified in the target scene Position information, determining contour information corresponding to the object to be recognized; determining a first position point corresponding to the object to be recognized based on the contour information.

In some implementations, the first determining part is determined based on the three-dimensional position information of the point cloud point corresponding to the object to be identified in the target scene, and determines the contour information corresponding to the object to be identified. The configuration is as follows: project the point cloud point corresponding to the object to be identified onto a preset plane to obtain a first projection point; based on the two-dimensional position information of the first projection point in the preset plane, determine the Contour information of the object to be recognized.

In some implementations, in the case of determining the first position point corresponding to the object to be recognized based on the contour information, the first determining part is configured to: use the contour information of the object to be recognized to determine A projection area of the object to be identified on a preset plane; based on an area of the projection area, a first position point corresponding to the object to be identified is determined.

In some implementations, the first determination part is configured to: compare the area of the projection area with a predetermined Compare the set area threshold; in response to the area being greater than the area threshold, based on the projection area, determine the minimum bounding box of the projection area; based on the first area corresponding to the minimum bounding box, and the preset A first interval step of , determining a plurality of candidate location points in the first area; determining the candidate location points located in the projection area as the first location point.

In some implementations, the first determination part is configured to: compare the area of the projection area with a predetermined The set area threshold is compared; in response to the area being less than or equal to the area threshold, determine the center point located in the projection area; based on the center point and the preset radius length, determine the center point is the center of the circle, and the preset radius length is the second area of the radius; based on the second area and the preset second interval step, the first position point is determined in the second area.

In some implementations, the second determination part is configured to: acquire and collect the The current frame image obtained by the target scene; projecting the first position point into the current frame image to obtain a second projection point; based on the position information of the second projection point in the current frame image, and the The position of the obstacle included in the current frame image in the current frame image is determined to determine whether the object to be identified is the current candidate deletion object.

In some implementations, the number of the first position point is at least one; when the second determining part projects the first position point into the current frame image to obtain a second projected point, It is configured to: project at least one of the first position points into the current frame image to obtain at least one second projection point; the second determination part is based on the second projection point in the current frame image In the case of determining whether the object to be identified is the current candidate deletion object, it is configured to: for each For the second projection point, predict the second projection point based on the position information of the second projection point in the current frame image and the position of obstacles included in the current frame image in the current frame image. The obstacle prediction result corresponding to the projection point; the obstacle prediction result includes: there is an obstacle at the position corresponding to the second projection point, or there is no obstacle; based on the obstacles corresponding to each of the second projection points As a result of the prediction, determine whether the object to be identified is the current candidate object for deletion.

In some implementations, the second determination part is configured to determine whether the object to be identified is the current candidate object for deletion based on the obstacle prediction results corresponding to the second projection points respectively. : Based on the obstacle prediction results corresponding to each of the second projection points, determine the confidence that the object to be recognized is an obstacle; determine the object to be recognized based on the confidence and a preset confidence threshold Whether it is the current candidate deletion object.

In some implementation manners, there are n second projection points, and n is an integer greater than 1; the second determination part determines the object to be identified based on the obstacle prediction results corresponding to each second projection point In the case of the confidence degree of an obstacle, it is configured to: traverse the 2nd to nth second projection points; for the traversed i-th second projection point, based on the obstacle Object prediction results, determine the criterion function corresponding to the i-th second projection point; wherein, i is a positive integer greater than 1; based on the criterion function corresponding to the i-th second projection point, and the first to i-th Based on the fusion criterion result of one second projection point, determine the fusion criterion result corresponding to the i-th second projection point; based on the fusion criterion result corresponding to the i-th second projection point, determine the Confidence that the object to be recognized is an obstacle.

In some implementations, the second determining part determines whether the object to be identified is the one to be deleted based on the historical position information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first position point. In the case of a target object, it is configured to: determine the target candidate object from the historical candidate objects based on the historical position information corresponding to the historical candidate object in the historical frame radar scan data and the first position point; A first projection area of the candidate target object on a preset plane and a second projection area of the object to be identified on the preset plane, and determine whether the object to be identified is a target object to be deleted.

In some implementations, the historical frame radar scan data includes at least one historical candidate object; the second determining part is based on the historical position information corresponding to the historical candidate object in the historical frame radar scan data, and the first A location point, when the target candidate object is determined from the historical candidate objects, is configured to: based on the historical location information corresponding to each historical candidate object in the historical frame radar scan data and the first location point, Determine the distances between each historical candidate object and the object to be identified; based on the distances between each historical candidate object and the object to be identified, determine the historical candidate closest to the object to be identified from each of the historical candidate objects The object is the target candidate object.

In some implementations, the second determination part determines the In the case of whether the object to be identified is the target object to be deleted, it is configured to: determine whether there is an overlapping area between the first projection area and the second projection area; respond to the first projection area and the second projection area There is no overlapping area in the projection area, and it is determined that the object to be recognized is the target object to be deleted.

In some implementations, the second determination part determines the In the case of whether the object to be identified is a target object to be deleted, it is further configured to: in response to an overlapping area between the first projection area and the second projection area in the current frame radar scan data, the object to be identified is taken as A new historical candidate object, until in the continuous N frames of radar scanning data after the current frame of radar scanning data, there is an overlapping area between the first projection area and the second projection area, then it is determined that the candidate deletion object is not the object to be deleted Deleted target object, and delete the new historical candidate object; N is a positive integer.

In some implementations, the detection device further includes: a processing part configured to, for each of the historical candidate deletion objects, detect the time difference between the storage time of the historical candidate deletion object and the current time; when the time difference is greater than or If it is equal to the preset time difference threshold, delete the historical candidate deletion object.

An embodiment of the present disclosure provides a computer device, a processor, and a memory, where the memory stores machine-readable instructions executable by the processor, and the processor is configured to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, when the machine-readable instructions are executed by the processor, the above-mentioned first aspect, or the steps in any possible implementation manner of the first aspect are executed.

An embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed, part or all of the steps in the above method are executed.

An embodiment of the present disclosure provides a computer program, including computer readable codes. When the computer readable codes run in a computer device, a processor in the computer device executes part or all of the steps in the above method.

An embodiment of the present disclosure provides a computer program product. The computer program product includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, the above-mentioned methods are implemented. some or all steps

For the effect description of the above-mentioned obstacle detection device, computer equipment, computer-readable storage medium, computer program and computer program product, please refer to the description of the above-mentioned obstacle detection method.

In order to make the above-mentioned features and advantages of the embodiments of the present disclosure more comprehensible, exemplary embodiments will be described in detail below together with the accompanying drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present disclosure more clearly, the following will briefly introduce the accompanying drawings used in the embodiments. The accompanying drawings here are incorporated into the specification and constitute a part of the specification. The drawings show the embodiments consistent with the present disclosure, and are used together with the description to explain the technical solution of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. For those skilled in the art, they can also make From these drawings other related drawings are obtained.

FIG. 1 is a schematic diagram of an implementation flow of an obstacle detection method provided by an embodiment of the present disclosure;

Fig. 2a is a schematic diagram of a projection area provided by an embodiment of the present disclosure;

Fig. 2b is a schematic diagram of an alternative location point provided by an embodiment of the present disclosure;

Fig. 2c is a schematic diagram of a first location point provided by an embodiment of the present disclosure;

Fig. 3a is a schematic diagram of a projection area provided by an embodiment of the present disclosure;

Fig. 3b is a schematic diagram of a first location point provided by an embodiment of the present disclosure;

Fig. 4a is a schematic diagram of a projection area provided by an embodiment of the present disclosure;

Fig. 4b is a schematic diagram of a projection area provided by an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of an implementation process for detecting an object to be recognized provided by an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of an obstacle detection device provided by an embodiment of the present disclosure;

Fig. 7 is a schematic diagram of a computer device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only It is a part of the embodiments of the present disclosure, but not all of them. The components of the disclosed embodiments generally described and illustrated herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort shall fall within the protection scope of the present disclosure.

After research, it is found that the driving area of the vehicle can be scanned by using the laser radar to determine the obstacles that may exist in the driving area. The laser radar can emit laser light to the driving area and receive the reflected laser light, and determine whether there is an obstacle according to the received laser light, so that it is easy to detect the obstacle normally when the reflected laser light is abnormal after the object is scanned . For example, when there is water on the road surface and lane lines with wet paint, the laser emitted by the lidar will not be able to reflect the laser normally due to mirror reflection, so that these non-obstacle objects are judged as obstacles, and the laser is used to detect obstacles. The detection accuracy is low.

Based on the above research, an embodiment of the present disclosure provides an obstacle detection method, by combining the historical position information of the historical candidate deletion object in the historical frame radar data, it is determined whether to detect the object to be identified in the current frame radar scan data The deletion process combines the space domain and the time domain to comprehensively judge whether the object to be recognized is the target object to be deleted, so as to judge whether the object to be recognized is an obstacle in the target scene, and has higher detection accuracy.

The defects in the above solutions are all the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the technical solutions proposed by the embodiments of the present disclosure below for the above problems should be Contributions made by the inventors to the disclosure during the course of the disclosure.

It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

In order to facilitate the understanding of this embodiment, a method for detecting obstacles disclosed in the embodiments of the present disclosure is firstly introduced in detail. The execution subject of the method for detecting obstacles provided in the embodiments of the present disclosure is generally a computer with certain computing power equipment, the computer equipment includes, for example: terminal equipment or server or other processing equipment, the terminal equipment can be user equipment (User Equipment, UE), mobile equipment, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant) Assistant, PDA), handheld devices, computing devices, vehicle-mounted devices, wearable devices, etc. In some possible implementation manners, the obstacle detection method may be implemented by a processor calling computer-readable instructions stored in a memory.

The method for detecting obstacles provided by the embodiments of the present disclosure will be described below.

Fig. 1 is a schematic diagram of an implementation flow diagram of an obstacle detection method provided by an embodiment of the present disclosure. Referring to Fig. 1 , the method includes steps S101 to S103, wherein:

S101: Based on the current frame radar scan data obtained by scanning the target scene, determine a first position point corresponding to the object to be identified in the target scene;

S102: Determine whether the object to be identified is a target object to be deleted based on the historical location information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first location point;

S103: In response to the fact that the object to be identified is not the target object to be deleted, determine the object to be identified as an obstacle in the target scene.

The embodiment of the present disclosure uses the current frame radar scanning data obtained from the target scene to determine the first position point corresponding to the object to be identified, and the historical position information corresponding to the historical candidate deletion object in the historical frame scanning data to determine whether the object to be identified is The target object to be deleted; when it is determined that the object to be recognized is not the target object to be deleted, the object to be recognized is determined as an obstacle in the target scene. This method can combine the space domain and the time domain to comprehensively judge whether the object to be recognized is the target object to be deleted, so as to judge whether the object to be recognized is an obstacle in the target scene, and has higher detection accuracy.

The above-mentioned S101 to S103 will be described in detail below.

With regard to the above S101, the obstacle detection method provided by the embodiment of the present disclosure may be applied in different scenarios. Exemplarily, in the automatic driving scene, the target scene may include, for example, the space where the self-driving car drives, and the target scene may include, for example, other driving vehicles, lane lines, signboards, green belts, and the like. In the smart storage scenario, the target scene may include, for example, the space where the intelligent robot travels, and the target scene may include, for example, other robots, staff, shelves, containers, positioning signs, and the like.

The method for detecting obstacles provided by the embodiments of the present disclosure will be described below by taking an automatic driving scene as an example.

In an autonomous driving scenario, for example, a laser radar can be installed on the autonomous vehicle to scan and detect the area where the autonomous vehicle is driving. During the driving process of the self-driving car, the laser radar can scan the target scene at an interval of 0.2 seconds, and obtain radar scanning data. Here, the radar scanning data whose scanning time is closest to the current moment is taken as the current frame radar scanning data obtained by scanning the target scene.

After the radar scan data of the current frame is obtained, the multiple objects determined in the target scene can be determined by using the radar scan data of the current frame. Exemplarily, the radar scanning data of the current frame may be processed by means of object detection. Since radar scanning data can reflect the size and shape of objects, objects in some target scenes can be identified by using object detection methods, such as vehicles and signage; these objects can be determined directly because they are objects that need to be avoided. for obstacles. At the same time, there may also be objects that cannot be identified by object detection. After detecting them by object recognition, these unrecognizable objects are used as objects to be identified in the current frame of radar scanning data, that is, the classification label is Object of "unknown (unknown) objects".

When determining whether the object to be identified is the target object to be deleted, for example, the first position point determined for the object to be identified may be implemented by using the radar scan data of the current frame.

Here, the first position point is a criterion or basis for judging whether the object to be recognized can be used as a candidate deletion object. Wherein, the candidate deletion objects are: objects that may belong to obstacles. In the subsequent process of determining the target object, the candidate deletion object can be further judged to determine whether it is indeed an obstacle; if the candidate deletion object is the target object, it indicates that the corresponding object to be recognized is not an obstacle, but a false detection.

In some implementation manners, when using the current frame radar scan data to determine the first position point corresponding to the object to be identified in the target scene, the following method may be adopted: for each of the objects to be identified, the current frame radar scanning Determine the point cloud point corresponding to the object to be identified in the data; determine the contour information corresponding to the object to be identified based on the three-dimensional position information of the point cloud point corresponding to the object to be identified in the target scene; The contour information is used to determine the first position point corresponding to the object to be recognized.

Taking any object to be identified among the determined objects to be identified as an example, if the object to be identified is determined in the current frame of radar scanning data by means of object recognition, it can be determined by using the data of the current frame of radar scanning. The point cloud point corresponding to the object to be recognized.

Since the number of point cloud points corresponding to the object to be recognized may be large, and the point cloud points use a large amount of three-dimensional position information data when representing the position of the object to be recognized. Therefore, for the object to be identified, use the point cloud point corresponding to the object to be identified to re-model the object to be identified, and then judge whether it is an obstacle, or use the neural network to re-identify the object corresponding to the object to be identified In terms of the method of encoding the point cloud points of the specific object and then judging the similarity with the point cloud points of the specific object determined in history, the amount of calculation is large, and it needs to consume more computing power, and the efficiency is also low. Therefore, in some embodiments, after determining the point cloud point corresponding to the object to be recognized, the contour information corresponding to the object to be recognized can be determined based on the three-dimensional position information of the point cloud point corresponding to the object to be recognized in the target scene, and then Then use the contour information to determine the first position point corresponding to the object to be recognized, so as to use the first position point with a small amount of data to determine whether the object to be recognized is the target object to be deleted.

When using the three-dimensional position information of the point cloud points corresponding to the object to be recognized in the target scene to determine the contour information corresponding to the object to be recognized, for example, the following method can be used: project the point cloud points corresponding to the object to be recognized to the preset Obtaining a first projection point in the preset plane; determining contour information of the object to be recognized based on two-dimensional position information of the first projection point in the preset plane. Wherein, the preset plane may be, for example, the plane on which the ground on which the self-driving car is driving is located. After projecting the point cloud point corresponding to the object to be recognized onto the preset plane, the first projection point can be obtained; at this time, the three-dimensional position information corresponding to the point cloud point is transformed into the two-dimensional position information corresponding to the first projection point, and the amount of data reduce. Using the two-dimensional position information of the first projection point in the preset plane, the projection points at the edge positions in the first projection point can be determined, and using these projection points at the edge positions and their corresponding two-dimensional position information, That is, the contour information corresponding to the object to be recognized can be determined.

In some implementations, when using the contour information to determine the first position point corresponding to the object to be recognized, for example, the contour information corresponding to the object to be recognized can be used to determine the projection area to be recognized in the preset plane; based on the area of the projection area , to determine the first location point corresponding to the object to be recognized.

By using the contour information corresponding to the object to be recognized, multiple corner points corresponding to the contour surrounding the object to be recognized can be determined; by using the two-dimensional position information of multiple corner points, it is possible to determine the area occupied by the projection area to be recognized on the preset plane area, so as to determine the first location point corresponding to the object to be recognized. In implementation, when determining the first location point corresponding to the object to be identified based on the area of the projection area, for example, the area of the projection area can be compared with a preset area threshold, and the object to be identified can be determined according to the comparison result corresponding to the first position point. Wherein, the preset area threshold may include, for example, 0.3 square meters, 0.5 square meters, etc.; the preset area threshold may be determined based on experience or actual conditions, for example. Taking the preset area threshold of 0.3 square meters as an example, when it is determined that the area of the projection area is greater than the area threshold, based on the projection area, determine the minimum bounding box of the projection area; An area, and a preset first interval step, a plurality of candidate position points are determined in the first area; and the candidate position points located in the projection area are determined as the first position points.

FIG. 2a to FIG. 2c are schematic diagrams of determining a first location point provided by an embodiment of the present disclosure. As shown in FIG. 2 a , a projection area 21 may be determined for the object to be identified, and the projection area 21 is, for example, an irregular polygon. Using this projected area 21 a corresponding minimum bounding box 22 can be determined. In some embodiments, the minimum bounding box 22 includes a rectangle. After the minimum bounding box 22 is determined, the first area 23 occupied by the minimum bounding box 22 can be determined. Using the preset first interval step, for example, a plurality of candidate position points can be determined within the first area 23 . The preset first interval step may include, for example, 0.2 meters. As shown in Fig. 2b, a plurality of candidate position points 24 can be determined in the first area 23 by using a preset first interval step; For the position point of the selected position point 24, the interval between the candidate position points 24 respectively corresponding to the four directions of up, down, left and right is the preset first interval step.

After a plurality of candidate position points 24 are determined, the candidate position point 24 corresponding to the object to be recognized can be frame-selected as the first position point by using the projection area 21 . As shown in FIG. 2c, using a plurality of candidate position points 24 shown in FIG. 2b and the projection area 21 shown in FIG. One location point 25.

Since the projection area 21 is generally an irregular figure, when the first position point 25 is determined using the projection area 21, there may be alternative position points 24 falling on the boundary of the projection area 21, such as shown in FIG. 2c point a and point b. In view of this situation, in some implementations, in order to keep the maximum number of location points, the candidate location points 24 respectively corresponding to points a and b can be used as the first location points 25 . In some implementations, since the projected area of the object to be identified has a relatively large area, the number of first position points that can be determined is large enough. In order to reduce the amount of data processing and improve the processing accuracy, a Candidate location points 24 corresponding to points and b points are screened. For example, the candidate position point 24 corresponding to point a has a larger part falling into the projection area 21 than the candidate position point 24 corresponding to point b; therefore, the candidate position point corresponding to point a is reserved in the first position point 25 24, while the candidate position point 24 corresponding to point b is screened out. The above two implementation processes may be determined according to actual conditions, and are not limited here. When it is determined that the area of the projected area is less than or equal to the area threshold, the center point located in the projected area can also be determined; based on the center point and the preset radius length, determine the center point as the center, the preset Let the length of the radius be the second area of the radius; based on the second area and a preset second interval step, determine the first position point in the second area.

FIG. 3 a to FIG. 3 b are schematic diagrams of determining a first location point provided by an embodiment of the present disclosure. As shown in FIG. 3 a , a projection area 31 (indicated by a dotted line framed area in the figure) can be determined for the object to be recognized, and the projection area 31 is, for example, an irregular polygon. Using this projection area 31 , a center point 32 of the projection area can be determined. Using the center point 32 and the preset radius length, for example, the second area 33 (indicated by the area framed by a solid line in the figure) can be determined. For example, the circular second region 33 can be defined with the center point 32 as the center and a preset radius length as the radius. Wherein, the preset radius length may be, for example, 0.2 meters, 0.3 meters and so on.

In some implementations, for example, when the determined center point 32 radiates outward to the boundary of the projection area 31 , the determined maximum length may be used as a preset radius length. As shown in FIG. 3 a , the maximum length r from the center point 32 to the boundary of the projection area 31 can be determined, and then the maximum length r is used as a preset radius length to determine the second area 33 . In this way, for an object to be recognized with a small projection area, a larger number of first position points can be determined, so that a sufficient number of first position points can be used to further determine whether the object to be recognized is a target object to be deleted.

Using the preset second interval step, for example, a plurality of position points can be determined within the second area 33 . Wherein, in order to determine more position points for the object to be recognized to a greater extent, the position points falling on the edge of the second area 33 are retained when determining the position points. Here, the preset second interval step, for example, can be the same as the above-mentioned first preset interval step, for example, including 0.2 meters; it can also determine a value different from the above-mentioned preset first interval step according to the actual situation , such as 0.15 meters; it can be determined according to the actual situation during implementation. As shown in FIG. 3 b , multiple position points can be determined in the second area 33 by using the preset second interval step. Here, since the projection area 31 corresponding to the object to be identified is relatively small, in order to ensure a sufficient number of position points for the object to be identified, the position points located in the second area 33 can all be determined as the first For the position points 34 , compared with the position points falling into the projection area 31 , the number of the obtained first position points 34 is more.

For the above S102, after determining the first position point corresponding to the object to be identified in the target scene, it may also be determined whether the object to be identified is the object to be deleted based on the historical position information corresponding to the historical candidate deletion object in the historical frame radar scan data. target. In implementation, for example, the following manner may be adopted: based on the first position point corresponding to the object to be identified, it is judged whether the object to be identified is a current candidate for deletion; in response to the object to be identified being the current candidate for deletion For an object, determine whether the object to be identified is a target object to be deleted based on the historical location information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first location point.

In some implementation manners, when judging whether the object to be identified is a current candidate for deletion based on the first position point of the object to be identified, the following method may be specifically adopted: acquire the current frame image obtained by scanning the target scene; Projecting the first position point into the current frame image to obtain a second projection point; based on the position information of the second projection point in the current frame image and the obstacles included in the current frame image At a position in the current frame image, determine whether the object to be identified is the current candidate object for deletion.

In some implementations, when acquiring the current frame image, for example, it may be acquired by using an image acquisition device mounted on the self-driving vehicle. Specifically, the image acquisition device may include a color camera, for example. When acquiring the current frame image, the first position point can be projected into the current frame image to obtain the second projected point. Wherein, while obtaining the second projection point by projection, the position information of the obtained second projection point may also be determined.

Here, since the freespace of the self-driving vehicle can be determined by using the current frame image, it can be further determined whether the object to be recognized is in the Objects in the drivable area, and then judge whether to delete the object to be recognized.

In some implementations, the number of the first position point is at least one; the projecting the first position point into the current frame image to obtain the second projected point may include: adding at least one of the The first position point is projected into the current frame image to obtain at least one second projection point; based on the position information of the second projection point in the current frame image and the obstacles included in the current frame image At the position in the current frame image, when determining whether the object to be identified is the current candidate object for deletion, for example, the following method may be used:

For each of the second projection points, based on the position information of the second projection point in the current frame image and the position of obstacles included in the current frame image in the current frame image, predict the The obstacle prediction result corresponding to the second projection point; the obstacle prediction result includes: there is an obstacle at the position corresponding to the second projection point, or there is no obstacle; based on each of the second projection points corresponding to Obstacle prediction results, determine whether the object to be identified is the current candidate object for deletion.

In some implementations, when predicting the obstacle prediction result corresponding to the second projection point, for example, according to the position information of the second projection point in the current frame image, and the position information of the obstacle included in the current frame image in the current frame image The location is determined.

In some implementations, when the second projection point is located at the position of the obstacle included in the current frame image, since the second projection point can be considered with a higher degree of confidence to indicate that there is an obstacle at the corresponding position, the corresponding determined The obstacle prediction result is: there is an obstacle at the position corresponding to the second projection point; in another possible implementation manner, when the second projection point is not located at the position of the obstacle included in the current frame image , since the second projection point can be considered with a higher degree of confidence to indicate that there is no obstacle at the corresponding position, the correspondingly determined obstacle prediction result is: there is no obstacle at the position corresponding to the second projection point.

In this way, for multiple second projection points, the obstacle prediction result corresponding to each second projection point can be determined.

In some implementations, when determining whether the object to be identified is the current candidate object to be deleted using the obstacle prediction results corresponding to the second projection points, for example, it may be based on the corresponding Determine the confidence that the object to be identified is an obstacle based on the obstacle prediction result; determine whether the object to be identified is the current candidate for deletion based on the confidence and a preset confidence threshold.

In some implementation manners, the determined second projection points may include, for example, n; wherein, n is a positive integer. When determining the confidence that the object to be recognized is an obstacle based on the obstacle prediction results corresponding to the second projection points, for example, the following method can be used: traverse the 2nd to nth second projection points; The i second projection point, based on the obstacle prediction result of the i second projection point, determine the criterion function corresponding to the i second projection point; wherein, i is a positive integer greater than 1; based on the i second projection point The criterion function corresponding to the i second projection point and the fusion criterion result of the 1st to i-1th second projection point are determined to determine the fusion criterion result corresponding to the i second projection point; based on the The fusion criterion result corresponding to the i-th second projection point is used to obtain a confidence degree for determining that the object to be recognized is an obstacle. Exemplarily, for the first second projection point, the corresponding obstacle prediction result may include, for example, that there is an obstacle at the position corresponding to the second projection point, or there is no obstacle; when the obstacle prediction result is different In the case of , the determined criterion function is also different. For example, in the case where the obstacle prediction result includes an obstacle at the position corresponding to the second projection point, the corresponding criterion function M_1(·) may, for example, indicate that there is an obstacle at the second projection point, and determine the Assuming a probability function (mass) of reliability; when the obstacle prediction result includes that there is no obstacle at the position corresponding to the second projection point, the corresponding criterion function M_2(·) can represent the second projection, for example The point is the likelihood function to determine the confidence level of the hypothesis in the absence of obstacles.

Here, for the convenience of explanation, for the second projection point, its corresponding criterion function is represented by M ₂ (·), and corresponds to the second projection point with different obstacle prediction results, which can be specifically the above-mentioned M_1(·) , and M_2(·).

When traversing the 2nd to nth second projection points, the fusion criterion results of the 1st to i-1th second projection points can also be determined to determine the fusion criterion corresponding to the ith second projection point According to the results.

Taking the second second projection point as an example, when determining the fusion criterion result corresponding to the second second projection point, for example, the following formula (1) can be used:

Wherein, a1 represents the first second projection point, and a2 represents the second second projection point. Here, corresponding to a2, its corresponding criterion function can be represented by M ₂ (a ₂ ). Since when traversing to a2, the previous second projection point only includes a1, so the fusion criterion result M ₁ (·) can be expressed directly by M ₁ (a ₁ ); or it can also be written as M ₂ (a ₁ ) form.

In addition, K represents a normalization coefficient, which satisfies the following formula (2):

Using formula (1), the fusion criterion result M ² ₁₂ (a ₁ ; a ₁ ) of a1 and a2 determined when traversing to the second second projection point a2 can be determined. Here, the superscript i of M indicates traversing to the i-th second projection point.

Then, the third second projection point a3 is traversed. Similarly, the criterion function corresponding to a3 can be expressed as M ₂ (a ₃ ), for example. At this time, the fusion criterion result corresponding to the third second projection point determined when traversing to a3 can be determined, which satisfies the following formula (3):

Among them, the normalization coefficient K satisfies the following formula (4):

Continue the above process in this way to continue traversing other second projection points, and determine the criterion function corresponding to the second projection point, and determine the fusion criterion result from the first one to the second projection point currently traversed until traversing all The second projection point to obtain the fusion criterion result corresponding to the nth second projection point, so as to obtain the confidence of determining that the object to be recognized is an obstacle. Wherein, the confidence level that the object to be recognized is an obstacle may include, for example, a probability value, such as 0.61, 0.70, or 0.86.

After determining the confidence that the object to be recognized is an obstacle, it may be determined whether the object to be recognized is a current candidate for deletion by using a preset confidence threshold. Wherein, the preset confidence threshold may also include a probability value, such as 0.75, for example. The preset confidence threshold may be determined based on experience or multiple experiments during implementation. In a case where the determined confidence that the object to be identified is an obstacle is numerically greater than the preset confidence threshold, it is considered that the object to be identified can be determined as the current candidate for deletion. Exemplarily, when it is determined that the object to be identified is an obstacle with a confidence level of 0.6, and the preset confidence threshold is 0.55, it is determined that the object to be identified is not a current candidate for deletion; and when it is determined that the object to be identified is an obstacle When the confidence level of is 0.80, it is determined that the object to be identified is the current candidate for deletion. After determining the current candidate deletion object, it may also be determined whether the object to be identified is the target object to be deleted based on the historical position information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first position point.

In some implementations, it may be determined whether the object to be identified is the target object to be deleted in the following manner: based on the historical position information corresponding to the historical candidate object in the historical frame radar scan data and the first position point, from Determining a target candidate object from the historical candidate objects; determining the target candidate object to be identified based on a first projection area of the target candidate object on a preset plane and a second projection area of the object to be identified on the preset plane Whether the object is the target object to be deleted.

In some implementations, when the radar scans the target scene, for example, it further includes obtaining historical frame radar scan data determined before the current frame radar scan data. Similarly, the corresponding historical candidate deletion object can be determined by using the historical frame radar scanning data, and the historical position information of the historical candidate deletion object can be determined. After the historical location information corresponding to the historical candidate objects is determined, the distances between each historical candidate object and the object to be recognized can be determined by using the first location point. Wherein, the historical position information of the historical candidate object used when determining the distance may be, for example, the position coordinates representing the center point of the historical candidate object; The first position point represents the position of the object to be recognized. Then, using the determined historical position information and the determined first position point, the distances corresponding to each historical candidate object and the object to be recognized can be determined.

In this way, the target candidate object with the closest distance to the object to be recognized can be determined from the historical candidate objects according to the distances between each historical candidate object and the object to be recognized. In some implementations, the distance with the smallest value may be determined among the determined multiple distances, and the corresponding historical candidate object may be used as the target candidate object; or, the historical candidate object corresponding to multiple distances with smaller numerical values may be used as target candidates. During implementation, the manner of determining target candidate objects may be determined according to actual conditions.

After the target candidate object is determined, the first projection area of the target candidate object on the preset plane and the second projection area of the object to be recognized on the preset plane may also be determined. Here, the preset plane may include, for example, the plane on which the road on which the self-driving vehicle drives is located. For example, refer to FIG. 4a to FIG. 4b . FIG. 4a and FIG. 4b are respectively schematic diagrams of a projection area provided by an embodiment of the present disclosure. In the situation shown in Figure 4a, the first projection area 41 of the target candidate object on the preset plane, and the second projection area 42 of the object to be identified on the preset plane (for ease of distinction, the first projection area 41 of the first projection area 41 Boundaries are shown in dashed lines) with overlapping regions. In the case shown in FIG. 4 b , there is no overlapping area between the first projected area 41 and the second projected area 42 . Here, for the case where there is no overlapping area between the first projection area and the second projection area, it can be considered that there is no target candidate corresponding to the object to be identified in the latest frame of historical frame scan data, indicating that the object to be identified It is a new object that is different from the target candidate object, and has not appeared in the recent period; therefore, when detecting the object to be recognized in the current frame scan data, the object to be recognized can be judged in the current frame is the target object to be deleted.

In addition, since the object to be recognized is a new object, the object to be recognized can also be used as a new historical candidate object, and the position information of the object to be recognized can be saved. The location information of the object to be recognized can be used to perform obstacle detection processing on the next frame of radar scan data. Using the location information of the object to be identified to perform obstacle detection processing on the next frame of radar scan data, the positions corresponding to the above-mentioned target candidate object in the historical frame scan data and the object to be identified in the current frame scan data The information is processed similarly for obstacle detection.

Since the object to be identified is judged as the target object to be deleted only for the current frame; if the object to be identified appears in the same area on the preset plane in the continuous multi-frame radar scan data, multiple After the frame radar scanning data, it is determined that the object to be identified is not the target object to be deleted.

In some implementations, in response to the overlap between the first projection area and the second projection area in the current frame radar scanning data, the object to be identified is used as a new historical candidate object until the current frame radar scanning In the consecutive N frames of radar scanning data after the data, if there is an overlapping area between the first projection area and the second projection area, it is determined that the candidate deletion object is the target object to be deleted, and the new historical candidate object Delete; N is a positive integer. Wherein, the value of N may include, for example, 5, 6, 8, 10 and so on. N can be determined according to the shooting interval when the radar scanning data is acquired, or determined according to experiments and other methods. The following takes N as 10 as an example for illustration.

In some implementations, when it is determined that there are N consecutive frames of historical frame radar scanning data, the first projection area and the second projection area both have overlapping areas, then it can be judged accordingly that the object to be identified is actually present in the target scene object. Exemplarily, when a warning cone is placed on the road in the target scene, after obtaining the first frame of radar scan data containing the warning cone, since the warning cone really exists, the following continuous multi-frame radar scanning The warning cone is included in the data, for example, after 15 frames of radar scanning data, the warning cone cannot be captured in the radar scanning data. That is, for the object to be identified, if the object to be identified is an object that actually exists in the target scene, there will be multiple frames of continuous radar scan data that contain the object to be identified. Since objects that actually exist in the target scene may interfere with the form of the autonomous vehicle, they need to be considered as the target objects that are not to be deleted. In addition, since the object to be recognized can be determined not to be a target object to be deleted, correspondingly there is no need to continue detecting it, so the object to be recognized as a new historical candidate object can be directly deleted.

In some implementations, if for the object to be identified, there is no overlapping area between the first projection area and the second projection area in the consecutive N frames of historical frame radar scanning data. For example, after the object to be identified appears, the object to be identified is only included in the consecutive frames of radar scan data not exceeding N, and the object to be identified can be used as the target object to be deleted. Exemplarily, in the case where the object to be identified includes stagnant water, due to the influence of specular reflection and reflection angle, when multiple frames of scanning data are continuously acquired during the driving process of the autonomous vehicle, there may be unevenness in the multiple frames of continuous radar scanning data. This is included, but after multiple frames of radar scanning data, for example, after 3 frames of radar scanning data, due to changes in angle and distance when scanning the object to be identified, the radar will no longer detect stagnant water through scanning, so the radar In the scanning data, the radar scanning data of the object to be identified will not be collected at positions close to the object to be identified on the preset plane. In this case, it may be determined that the object to be identified is the target object to be deleted.

In some embodiments, as the self-driving vehicle is constantly driving, the obtained radar scanning data is also constantly changing. Therefore, for the historical candidate deletion object, if there is a historical candidate deletion object that has not appeared for a period of time, it can be considered that the historical candidate deletion object will not appear again in the process of continuing to drive.

In implementation, for each historical candidate deletion object, the time difference between the storage time of the historical candidate deletion object and the current time may be detected; if the time difference is greater than or equal to a preset time difference threshold, the historical candidate deletion object is deleted. Wherein, the preset time difference threshold may include, for example, 3 seconds, 4 seconds, etc.; specifically, it may be determined according to actual conditions or experiments. In this way, it can also ensure that the objects to be identified to be deleted are gradually screened out by using the historical candidate deletion objects, and the data to be stored is reduced to a certain extent, and the matching calculation of the historical candidate deletion objects and the current object to be identified is also reduced. Improve the detection efficiency of obstacles.

Regarding the above S103, if it is determined according to the above S102 that the object to be recognized is not the target object to be deleted, the object to be recognized may be determined as an obstacle in the target scene. At this time, for example, the self-driving vehicle can be controlled to take an evasive action. Since the obstacle detection method provided by the embodiment of the present disclosure has a high detection accuracy when detecting and judging whether the object to be recognized is the target object to be deleted, the automatic driving is performed by applying the obstacle detection method provided by the embodiment of the present disclosure When the vehicle is driving automatically, it can more accurately determine the obstacles that actually need to be avoided in the driving area, and complete effective obstacle avoidance. In this way, the person driving the self-driving vehicle can effectively reduce the sharp steering or sudden braking caused by the recognition of non-obstacles during the process of riding the self-driving vehicle, and the driving experience is better.

In some embodiments, a specific embodiment for detecting an object to be recognized in a target scene is also provided. FIG. 5 is a schematic diagram of an implementation process for detecting an object to be recognized provided by an embodiment of the present disclosure; wherein,

S501: Based on the current frame of radar scanning data obtained by scanning the target scene, determine a first position point corresponding to the object to be identified.

S502: Based on the first location point, determine whether the object to be identified is a current candidate for deletion; if yes, go to step S503; if not, go to step S507.

S503: Based on the historical location information corresponding to the historical candidate objects in the historical frame radar scanning data and the first location point, determine the target candidate object from the historical candidate objects.

S504: Determine whether there is an overlapping area between the first projection area of the candidate target object on the preset plane and the second projection area of the object to be recognized on the preset plane; if yes, go to S505; if not, go to S508.

S505: Use the object to be recognized as a new historical candidate object.

S506: Determine whether there is an overlapping area between the first projection area and the second projection area in the consecutive N frames of radar scan data after the current frame of radar scan data; if so, go to S507; if not, go to S508 .

S507: Determine that the object to be recognized is an obstacle in the target scene.

S508: Determine that the object to be identified is the target object to be deleted.

Those skilled in the art can understand that in the above method of specific implementation, the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possible The inner logic is OK.

Based on the same inventive concept, the embodiment of the present disclosure also provides an obstacle detection device corresponding to the obstacle detection method. Since the device in the embodiment of the present disclosure corresponds to the above-mentioned obstacle detection method in the embodiment of the present disclosure, the device The implementation can be found in the implementation of the method.

Fig. 6 is a schematic diagram of an obstacle detection device provided by an embodiment of the present disclosure. As shown in Fig. 6 , the device includes: a first determination part 61, a second determination part 62, and a third determination part 63; wherein,

The first determining part 61 is configured to determine a first position point corresponding to the object to be identified in the target scene based on the current frame radar scanning data obtained by scanning the target scene;

The second determination part 62 is configured to determine whether the object to be identified is a target object to be deleted based on the historical position information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first position point;

The third determination part 63 is configured to determine the object to be identified as an obstacle in the target scene in response to the object to be identified not being the target object to be deleted.

In some implementations, the second determination part 62 determines whether the object to be identified is to be deleted based on the historical position information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first position point. In the case of the target object, it is configured to: determine whether the object to be identified is a current candidate for deletion based on the first position point corresponding to the object to be identified; For an object, determine whether the object to be identified is a target object to be deleted based on the historical location information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first location point.

In some implementations, the second determination part 62 is further configured to: determine the object to be identified as an obstacle in the target scene in response to the object to be identified not being the current candidate object for deletion .

In some implementations, the first determination part 61 is configured in the case of determining the first position point corresponding to the object to be identified in the target scene based on the current frame radar scan data obtained by scanning the target scene is: for each of the objects to be identified, determine the point cloud points corresponding to the objects to be identified from the current frame radar scan data; based on the point cloud points corresponding to the objects to be identified in the target scene The three-dimensional position information is used to determine contour information corresponding to the object to be recognized; and to determine a first position point corresponding to the object to be recognized based on the contour information.

In some implementations, when the first determining part 61 determines the outline information corresponding to the object to be identified based on the three-dimensional position information of the point cloud point corresponding to the object to be identified in the target scene, It is configured to: project the point cloud point corresponding to the object to be identified onto a preset plane to obtain a first projection point; determine the first projection point based on two-dimensional position information of the first projection point in the preset plane Describe the contour information of the object to be recognized.

In some implementations, when determining the first position point corresponding to the object to be identified based on the outline information, the first determining part 61 is configured to: use the outline information of the object to be identified, Determine a projection area of the object to be identified on a preset plane; determine a first position point corresponding to the object to be identified based on the area of the projection area.

In some implementations, when determining the first position point corresponding to the object to be identified based on the area of the projection area, the first determining part 61 is configured to: combine the area of the projection area with Comparing with a preset area threshold; in response to the area being greater than the area threshold, based on the projection area, determining the minimum bounding box of the projection area; based on the first area corresponding to the minimum bounding box, and the preset Determine a plurality of candidate location points in the first area, and determine the candidate location points located in the projection area as the first location point.

In some implementations, when determining the first position point corresponding to the object to be identified based on the area of the projection area, the first determining part 61 is configured to: combine the area of the projection area with comparing with a preset area threshold; in response to the area being less than or equal to the area threshold, determine the center point located in the projection area; based on the center point and the preset radius length, determine the center point The point is the center of the circle, and the preset radius length is the second area of the radius; based on the second area and the preset second interval step, the first position point is determined in the second area.

In some implementations, the second determining part 62 is configured to: obtain the collected The current frame image obtained by the target scene; projecting the first position point into the current frame image to obtain a second projection point; based on the position information of the second projection point in the current frame image, and Determine whether the object to be identified is the current candidate object for deletion based on the position of the obstacle included in the current frame image.

In some implementations, the number of the first position point is at least one; when the second determining part 62 projects the first position point into the current frame image to obtain a second projected point , is configured to: project at least one of the first position points into the current frame image to obtain at least one second projection point; the second determining part 62 is based on the second projection point in the current The position information in the frame image and the position of the obstacle included in the current frame image, in the case of determining whether the object to be identified is the current candidate deletion object, are configured to: For each second projection point, predict the The obstacle prediction result corresponding to the second projection point; the obstacle prediction result includes: there is an obstacle at the position corresponding to the second projection point, or there is no obstacle; Obstacle prediction results determine whether the object to be identified is the current candidate object for deletion.

In some implementations, the second determination part 62 is configured to determine whether the object to be identified is the current candidate object for deletion based on the obstacle prediction results corresponding to the second projection points respectively. It is: based on the obstacle prediction results corresponding to the second projection points, determine the confidence that the object to be recognized is an obstacle; based on the confidence and a preset confidence threshold, determine the to-be-recognized Whether the object is the current candidate for deletion.

In some implementations, there are n second projection points, and n is an integer greater than 1; the second determining part 62 determines the In the case where the object is the confidence level of an obstacle, it is configured to: traverse the 2nd to nth second projection points; for the traversed ith second projection point, based on the For the obstacle prediction result, determine the criterion function corresponding to the i-th second projection point; wherein, i is a positive integer greater than 1; based on the criterion function corresponding to the i-th second projection point, and the first to i-th -1 fusion criterion result of the second projection point, determining the fusion criterion result corresponding to the ith second projection point; based on the fusion criterion result corresponding to the i second projection point, determining the fusion criterion result Confidence that the object to be recognized is an obstacle.

In some implementations, the second determination part 62 determines whether the object to be identified is to be deleted based on the historical position information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first position point. In the case of the target object, it is configured to: determine the target candidate object from the historical candidate object based on the historical position information corresponding to the historical candidate object in the historical frame radar scan data and the first position point; The first projection area of the target candidate object on the preset plane and the second projection area of the object to be identified on the preset plane, determine whether the object to be identified is the target object to be deleted.

In some implementations, the historical frame radar scan data includes at least one historical candidate object; the second determining part 62 is based on the historical position information corresponding to the historical candidate object in the historical frame radar scan data, and the The first position point, in the case of determining the target candidate object from the historical candidate objects, is configured to: based on the historical position information corresponding to each historical candidate object in the historical frame radar scan data, and the first position point , determine the distances between each historical candidate object and the object to be identified; based on the distances between each historical candidate object and the object to be identified, determine the history closest to the object to be identified from each of the historical candidate objects The candidate object is the target candidate object.

In some implementations, the second determining part 62 determines the target candidate object based on the first projection area of the target candidate object on the preset plane and the second projection area of the object to be identified on the preset plane. In the case of whether the object to be identified is a target object to be deleted, it is configured to: determine whether there is an overlapping area between the first projection area and the second projection area; respond to the first projection area and the second projection area There is no overlapping area between the two projection areas, and it is determined that the object to be recognized is the target object to be deleted.

In some implementations, the second determining part 62 determines the target candidate object based on the first projection area of the target candidate object on the preset plane and the second projection area of the object to be identified on the preset plane. In the case of whether the object to be identified is a target object to be deleted, it is further configured to: in response to an overlapping area between the first projection area and the second projection area in the current frame radar scan data, the object to be identified As a new historical candidate object, until in the continuous N frames of radar scanning data after the current frame of radar scanning data, there is an overlapping area between the first projection area and the second projection area, then it is determined that the candidate deletion object is not the The target object to be deleted, and the new historical candidate object is deleted; N is a positive integer.

In some implementations, the detection device further includes: a processing part 64 configured to, for each of the historical candidate deletion objects, detect the time difference between the storage time of the historical candidate deletion object and the current time; when the time difference is greater than Or if it is equal to the preset time difference threshold, delete the historical candidate deletion object.

For the description of the processing flow of each part in the device and the interaction flow between each part, reference may be made to the relevant descriptions in the foregoing method embodiments.

In the embodiments of the present disclosure and other embodiments, a "part" may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course it may also be a unit, a module or a non-modular one.

The embodiment of the present disclosure also provides a computer device. FIG. 7 is a schematic structural diagram of the computer device provided by the embodiment of the present disclosure. As shown in FIG. 7 , the device includes:

Processor 10 and memory 20; the memory 20 stores machine-readable instructions executable by the processor 10, the processor 10 is used to execute the machine-readable instructions stored in the memory 20, and the machine-readable instructions are executed by the processor 10 During execution, the processor 10 performs the following steps: based on the current frame radar scan data obtained by scanning the target scene, determine the first position point corresponding to the object to be identified in the target scene; The historical location information corresponding to the historical candidate deletion object and the first location point, determine whether the object to be identified is the target object to be deleted; in response to the object to be identified is not the target object to be deleted, set The object to be recognized is determined as an obstacle in the target scene. Above-mentioned memory 20 comprises memory 210 and external memory 220; Memory 210 here is also called internal memory, is used for temporarily storing the operation data in processor 10, and the data exchanged with external memory 220 such as hard disk, processor 10 communicates with memory 210 through memory 210. The external memory 220 performs data exchange. For the execution process of the above instructions, reference may be made to the steps of the obstacle detection method described in the embodiments of the present disclosure.

An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the obstacle detection method described in the above-mentioned method embodiments are executed. . Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

An embodiment of the present disclosure also provides a computer program product, the computer program product carries a program code, and the instructions included in the program code can be used to execute the steps of the obstacle detection method described in the above method embodiment, for details, please refer to Embodiment of the above-mentioned method. Wherein, the computer program product may be specifically realized by hardware, software or a combination thereof. In some embodiments, the computer program product is embodied as a computer storage medium, and in some embodiments, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.

An embodiment of the present disclosure also provides a computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, it realizes part of the above method or all steps.

Those skilled in the art can clearly understand that for the convenience and brevity of description, for the specific working process of the system and device described above, reference can be made to the corresponding process in the foregoing method embodiments. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disc and other media that can store program codes. .

Finally, it should be noted that: the above-mentioned embodiments are only specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure, rather than limit them, and the protection scope of the present disclosure is not limited thereto, although referring to the aforementioned The embodiments have described the present disclosure in detail, and those skilled in the art should understand that any person familiar with the technical field can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present disclosure Changes can be easily imagined, or equivalent replacements can be made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in this disclosure. within the scope of protection.

Industrial Applicability

Embodiments of the present disclosure provide an obstacle detection method, device, computer equipment, storage medium, computer program, and computer program product, wherein the obstacle detection method includes: based on the current frame radar obtained by scanning the target scene Scan the data to determine the first position point corresponding to the object to be identified in the target scene; determine whether the object to be identified is to be deleted based on the historical position information corresponding to the historical candidate deletion object in the historical frame radar scan data and the first position point is the target object; in response to the fact that the object to be identified is not the target object to be deleted, the object to be identified is determined as an obstacle in the target scene. The obstacle detection method provided by the embodiments of the present disclosure has high accuracy when detecting obstacles.

Claims

A method for detecting an obstacle, comprising:

Based on the current frame radar scan data obtained by scanning the target scene, determine a first position point corresponding to the object to be identified in the target scene;

Based on the historical position information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first position point, determine whether the object to be identified is a target object to be deleted;

In response to the object to be identified is not the target object to be deleted, the object to be identified is determined as an obstacle in the target scene.
The detection method according to claim 1, wherein, based on the historical position information corresponding to the historical candidate deletion object in the historical frame radar scan data and the first position point, it is determined whether the object to be identified is to be deleted target audience, including:

Based on the first position point corresponding to the object to be identified, determine whether the object to be identified is a current candidate for deletion;

Responding to the fact that the object to be identified is the current candidate for deletion, based on the historical position information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first position point, determine whether the object to be identified is Deleted target object.
The detection method according to claim 2, further comprising: determining the object to be identified as an obstacle in the target scene in response to the object to be identified not being the current candidate object for deletion.
The detection method according to any one of claims 1-3, wherein the first position point corresponding to the object to be identified in the target scene is determined based on the current frame radar scan data obtained by scanning the target scene, include:

For each of the objects to be identified, determining a point cloud point corresponding to the object to be identified from the current frame radar scan data;

Based on the three-dimensional position information of the point cloud point corresponding to the object to be identified in the target scene, determine the contour information corresponding to the object to be identified;

Based on the contour information, a first position point corresponding to the object to be recognized is determined.
The detection method according to claim 4, wherein the determining the contour information corresponding to the object to be identified based on the three-dimensional position information of the point cloud point corresponding to the object to be identified in the target scene comprises:

Projecting the point cloud points corresponding to the object to be identified onto a preset plane to obtain a first projection point;

Based on the two-dimensional position information of the first projection point in the preset plane, the contour information of the object to be recognized is determined.
The detection method according to claim 4 or 5, wherein said determining the first position point corresponding to the object to be identified based on the contour information comprises:

Using the outline information of the object to be identified, determine the projection area of the object to be identified in a preset plane;

Based on the area of the projection area, a first position point corresponding to the object to be identified is determined.
The detection method according to claim 6, wherein said determining the first position point corresponding to the object to be identified based on the area of the projection area comprises:

Comparing the area of the projected area with a preset area threshold;

determining a minimum bounding box of the projected area based on the projected area in response to the area being greater than the area threshold;

Based on the first area corresponding to the minimum bounding box and the preset first interval step, determine a plurality of candidate location points in the first area;

A candidate location point within the projection area is determined as the first location point.
The detection method according to claim 6, wherein said determining the first position point corresponding to the object to be identified based on the area of the projection area comprises:

Comparing the area of the projected area with a preset area threshold;

In response to the area being less than or equal to the area threshold, determine a center point located in the projection area;

Based on the center point and the preset radius length, determine a second area with the center point as the center and the preset radius length as the radius;

The first position point is determined in the second area based on the second area and a preset second interval step.
The detection method according to any one of claims 1-8, wherein the judging whether the object to be identified is a current candidate for deletion based on the first position point corresponding to the object to be identified comprises:

Acquiring the current frame image obtained by collecting the target scene;

projecting the first position point into the current frame image to obtain a second projected point;

Based on the position information of the second projection point in the current frame image and the position of obstacles included in the current frame image in the current frame image, determine whether the object to be identified is the current candidate delete object.
The detection method according to claim 9, wherein the number of the first position points is at least one;

The step of projecting the first position point into the current frame image to obtain a second projected point includes:

projecting at least one of the first position points into the current frame image to obtain at least one second projected point;

Determining whether the object to be recognized is the Current candidates for deletion include:

For each of the second projection points, based on the position information of the second projection point in the current frame image and the position of obstacles included in the current frame image in the current frame image, predict the The obstacle prediction result corresponding to the second projection point; the obstacle prediction result includes: there is an obstacle at the position corresponding to the second projection point, or there is no obstacle;

Based on the obstacle prediction results respectively corresponding to the second projection points, it is determined whether the object to be identified is the current candidate deletion object.
The detection method according to claim 10, wherein the determining whether the object to be identified is the current candidate deletion object based on the obstacle prediction results corresponding to each of the second projection points includes:

determining the confidence that the object to be identified is an obstacle based on the obstacle prediction results corresponding to each of the second projection points;

Based on the confidence level and a preset confidence level threshold, it is determined whether the object to be identified is the current candidate deletion object.
The detection method according to claim 11, wherein there are n second projection points, and n is an integer greater than 1;

The determining the confidence that the object to be identified is an obstacle based on the obstacle prediction results corresponding to the second projection points includes:

Traversing the 2nd to nth second projection points;

For the i-th second projection point traversed, based on the obstacle prediction result of the i-th second projection point, determine the criterion function corresponding to the i-th second projection point; wherein, i is greater than 1 positive integer;

Based on the criterion function corresponding to the i-th second projection point and the fusion criterion results of the 1st to i-1th second projection points, determine the fusion criterion result corresponding to the i-th second projection point;

Based on the fusion criterion result corresponding to the ith second projection point, a confidence degree for determining that the object to be recognized is an obstacle is obtained.
The detection method according to any one of claims 1-12, wherein the historical position information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first position point are used to determine the to-be-identified Whether the object is the target object to be deleted, including:

Determining target candidates from the historical candidate objects based on the historical position information corresponding to the historical candidate objects in the historical frame radar scan data and the first position point;

Based on a first projection area of the candidate target object on a preset plane and a second projection area of the object to be identified on the preset plane, it is determined whether the object to be identified is a target object to be deleted.
The detection method according to claim 13, wherein the historical frame radar scan data includes at least one historical candidate object; the historical position information corresponding to the historical candidate object in the historical frame radar scan data, and the The first position point, determining the target candidate object from the historical candidate objects, includes:

Based on the historical position information corresponding to each historical candidate object in the historical frame radar scanning data and the first position point, determine the distance between each historical candidate object and the object to be identified;

Based on the respective distances between each historical candidate object and the object to be identified, the historical candidate object with the closest distance to the object to be identified is determined from each of the historical candidate objects as the target candidate object.
The detection method according to claim 13 or 14, wherein, based on the first projection area of the target candidate object on a preset plane and the second projection area of the object to be identified on the preset plane, Determining whether the object to be identified is a target object to be deleted includes:

determining whether there is an overlapping area between the first projected area and the second projected area;

In response to the fact that there is no overlapping area between the first projection area and the second projection area, it is determined that the object to be recognized is the target object to be deleted.
The detection method according to claim 15, wherein the determination is based on the first projection area of the target candidate object on the preset plane and the second projection area of the object to be identified on the preset plane. Describe whether the object to be identified is the target object to be deleted, including:

In response to an overlapping area between the first projection area and the second projection area in the current frame of radar scan data, the object to be identified is used as a new historical candidate object until consecutive N frames after the current frame of radar scan data In the radar scan data, if there is an overlapping area between the first projection area and the second projection area, it is determined that the candidate deletion object is not the target object to be deleted, and the new historical candidate object is deleted; N is positive integer.
The detection method according to any one of claims 1-16, further comprising: for each of the historical candidate deletion objects, detecting the storage time of the historical candidate deletion object and the time difference with the current time;

If the time difference is greater than or equal to a preset time difference threshold, the historical candidate deletion object is deleted.
An obstacle detection device, comprising:

The first determining part is configured to determine a first position point corresponding to the object to be identified in the target scene based on the current frame radar scanning data obtained by scanning the target scene;

The second determination part is configured to determine whether the object to be identified is a target object to be deleted based on the historical position information corresponding to the historical candidate deletion object in the historical frame radar scanning data and the first position point;

The third determination part is configured to determine the object to be identified as an obstacle in the target scene in response to the object to be identified not being the target object to be deleted.
A computer device, comprising: a processor and a memory, the memory stores machine-readable instructions executable by the processor, the processor is configured to execute the machine-readable instructions stored in the memory, and the machine can When the read instruction is executed by the processor, the processor executes the steps of the obstacle detection method according to any one of claims 1 to 17.
A computer-readable storage medium, a computer program is stored on the computer-readable storage medium, and when the computer program is run by a computer device, the computer device executes the method according to any one of claims 1 to 17 The steps of the obstacle detection method.
A computer program comprising computer readable code for a processor in said computer device to perform the obstacle according to any one of claims 1 to 17 when said computer readable code is run in a computer device The steps of the detection method of the substance.
A computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, any one of claims 1 to 17 can be realized. The steps of the detection method of the obstacle described in item.