WO2023207845A1

WO2023207845A1 - Parking space detection method and apparatus, and electronic device and machine-readable storage medium

Info

Publication number: WO2023207845A1
Application number: PCT/CN2023/090064
Authority: WO
Inventors: 张经纬; 方梓成; 赵显�
Original assignee: 上海高德威智能交通系统有限公司
Priority date: 2022-04-28
Filing date: 2023-04-23
Publication date: 2023-11-02
Also published as: CN114882701A; CN114882701B

Abstract

A parking space detection method and apparatus, and an electronic device and a machine-readable storage medium. The parking space detection method comprises: acquiring point cloud data around a vehicle body by using a vehicle-mounted millimeter-wave radar, and performing rasterization processing on the point cloud data, so as to obtain a raster density map (S100); according to the raster density map, performing target detection by using a deep learning algorithm, so as to obtain a parking space detection result and a vehicle detection result (S110); analyzing an area size between adjacent vehicles according to the vehicle detection result, so as to obtain a parking space analysis result (S120); and fusing the parking space detection result with the parking space analysis result, so as to obtain a final parking space detection result (S130).

Description

Parking space detection method, device, electronic equipment and machine-readable storage medium

Technical field

The present application relates to the field of target detection technology, and in particular to a parking space detection method, device, electronic equipment and machine-readable storage medium.

Background technique

The detection performance of parking space targets is very critical to the realization of automatic parking. Currently, commonly used sensors for parking space detection include cameras, ultrasonic radar and millimeter wave radar.

Cameras are one of the most widely used sensors for autonomous driving. Vision-based parking space detection is currently relatively mature, and many are based on deep learning methods. The deep learning network is used to detect the parking space lines on the images obtained by the camera, and then perform post-processing of the parking spaces. However, the images acquired by the camera are susceptible to weather interference, and different lighting conditions have a greater impact on the detection results.

Ultrasonic radar is also a commonly used sensor for parking. However, the detection range of ultrasonic radar is short and the detection point cloud is relatively sparse, which restricts product application.

Millimeter wave radar overcomes the disadvantage of cameras being susceptible to environmental interference and has strong robustness. In addition, millimeter wave radar can generate denser point clouds than ultrasonic radar, has a longer detection range, and has greater potential in parking space detection.

Contents of the invention

Embodiments of the present application provide a parking space detection method, device, electronic device, and machine-readable storage medium.

According to a first aspect of the embodiment of the present application, a parking space detection method is provided, including:

Use vehicle-mounted millimeter wave radar to obtain point cloud data around the vehicle body, and rasterize the point cloud data to obtain a raster density map;

Based on the grid density map, a deep learning algorithm is used for target detection to obtain parking space detection results and vehicle detection results;

Based on the vehicle detection results, analyze the size of the area between adjacent vehicles to obtain parking space analysis results;

The parking space detection results and the parking space analysis results are fused to obtain the final parking space detection result.

According to a second aspect of the embodiment of the present application, a parking space detection device is provided, including:

A data preprocessing unit is used to obtain point cloud data around the vehicle body using a vehicle-mounted millimeter wave radar, and rasterize the point cloud data to obtain a raster density map;

A target detection unit, used to perform target detection using a deep learning algorithm based on the grid density map, and obtain parking space detection results and vehicle detection results;

An analysis unit is used to analyze the size of the area between adjacent vehicles based on the vehicle detection results to obtain parking space analysis results;

A fusion unit is used to fuse the parking space detection results and the parking space analysis results to obtain the final parking space detection result.

According to a third aspect of the embodiment of the present application, an electronic device is provided, including a processor and a memory. The memory stores machine-executable instructions that can be executed by the processor. The processor is configured to execute the machine-executable instructions. Execute instructions to implement the method provided by the first aspect.

According to a fourth aspect of the embodiment of the present application, a machine-readable storage medium is provided. Machine-executable instructions are stored in the machine-readable storage medium. When the machine-executable instructions are executed by a processor, the processor Implement the method provided in the first aspect.

By rasterizing the point cloud data around the car body obtained by the vehicle-mounted millimeter wave radar, a raster density map is obtained, and a deep learning algorithm is used to perform target detection on the raster density map to obtain parking space detection results and vehicle detection. As a result, the accuracy of the target detection results based on the point cloud data of the millimeter wave radar is improved; in addition, the area size between adjacent vehicles can be analyzed based on the vehicle detection results obtained by the target detection, and the parking space analysis results can be obtained , and fuse the parking space analysis results with the parking space detection results obtained from target detection to obtain the final parking space detection result, which improves the reliability and accuracy of parking space detection.

Description of the drawings

Figure 1 is a schematic flowchart of a parking space detection method according to an exemplary embodiment of the present application.

Figure 2 is a schematic diagram of installing millimeter wave radars at the four corners of a vehicle according to an exemplary embodiment of the present application.

Figure 3 is a schematic diagram of a parking space detection system framework according to an exemplary embodiment of the present application.

Figure 4 is a schematic diagram showing the effect of a detection model according to an exemplary embodiment of the present application.

Figure 5 is a schematic structural diagram of a parking space analysis module according to an exemplary embodiment of the present application.

FIG. 6 is a schematic diagram of determining a matching pair of adjacent targets according to an exemplary embodiment of the present application.

FIG. 7 is a schematic diagram of a candidate area according to an exemplary embodiment of the present application.

Figure 8 is a schematic structural diagram of a parking space fusion module according to an exemplary embodiment of the present application.

Figure 9 is a schematic diagram of removing obstacles according to an exemplary embodiment of the present application.

Figure 10 is a schematic structural diagram of a parking space detection device according to an exemplary embodiment of the present application.

Figure 11 is a schematic diagram of the hardware structure of an electronic device according to an exemplary embodiment of the present application.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the appended claims.

The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a," "the" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise.

In order to enable those skilled in the art to better understand the embodiments of the present application, and to make the above purposes, features and advantages of the embodiments of the present application more obvious and understandable, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

It should be noted that the sequence number of each step in the embodiment of the present application does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any influence on the implementation process of the embodiment of the present application. limited.

Please refer to Figure 1, which is a schematic flow chart of a parking space detection method provided by an embodiment of the present application. As shown in Figure 1, the parking space detection method may include the following steps S100-S130.

Step S100: Use the vehicle-mounted millimeter wave radar to obtain point cloud data around the vehicle body, and rasterize the point cloud data to obtain a raster density map.

In the embodiment of the present application, in order to realize parking space detection, multiple millimeter wave radars (which can be called vehicle-mounted millimeter wave radars) can be deployed on the vehicle, and the scanning range of the multiple millimeter wave radars can cover the surroundings of the vehicle body.

For example, millimeter wave radars can be installed at the four corners of the vehicle (which can be called vehicle-mounted angular millimeter wave radars), or millimeter wave radars can be installed at the front, rear, left and right of the vehicle.

For example, the point cloud data around the car body can be obtained through millimeter wave radar, and based on the millimeter wave radar and the vehicle Calibration parameters of the body coordinate system, convert point cloud data to the body coordinate system, and obtain object information around the body.

In the embodiment of the present application, for the point cloud data around the vehicle body obtained by using the vehicle-mounted millimeter wave radar, the point cloud data can be rasterized to obtain a raster density map.

Step S110: Use a deep learning algorithm to perform target detection based on the grid density map, and obtain parking space detection results and vehicle detection results.

In the embodiment of the present application, for the grid density map obtained in step S100, a deep learning algorithm can be used for target detection to obtain parking space detection results and vehicle detection results.

For example, the raster density map can be input into a pre-trained deep learning network model to obtain parking space detection results and vehicle detection results.

For example, the parking space detection results may include parking space location information and parking space size information.

For example, the parking space location information may include the center point position of the parking space detection frame, or the corner point position of the parking space detection frame. The parking space size information may include the length and width of the parking space detection frame.

For example, the vehicle detection results may include vehicle location information and vehicle size information.

For example, the vehicle position information may include the center point position of the vehicle detection frame, or the corner point position of the vehicle detection frame. The vehicle size information may include the length and width of the vehicle detection frame.

It should be noted that in the embodiments of this application, unless otherwise specified, the parking space detection/analysis mentioned refers to the detection/analysis of free parking spaces.

Step S120: Based on the vehicle detection results, analyze the size of the area between adjacent vehicles to obtain the parking space analysis result.

Step S130: Fusion of the parking space detection results and the parking space analysis results to obtain the final parking space detection result.

In the embodiment of this application, considering that in actual scenarios, when there is a large free area between adjacent vehicles parked in a parking space, there may be free parking spaces between the adjacent vehicles, that is, the vehicle detection results can be used , analyze the size of the area between adjacent vehicles to analyze the free parking space information.

For example, the area size may include, but is not limited to, one or more of area length, area width, and area area.

In addition, since the reliability of vehicles scanned by millimeter wave radar is usually higher than the reliability of free parking spaces scanned by millimeter wave radar, the free parking space information can be more accurately analyzed based on the vehicle detection results. Information, and then, by fusing the detected parking space information (i.e., parking space detection results) and the analyzed parking space information (i.e., parking space analysis results), the reliability and accuracy of parking space detection can be effectively improved.

Correspondingly, when the parking space detection results and the vehicle detection results are obtained in the manner described in step S110, the parking space analysis can be performed based on the vehicle detection results, the parking space analysis results can be obtained, and the parking space detection results and the parking space analysis results can be fused, Get the final result of parking space detection.

It can be seen that in the method process shown in Figure 1, the point cloud data around the car body obtained by using the vehicle-mounted millimeter wave radar is rasterized to obtain a raster density map, and a deep learning algorithm is used to obtain the raster density map. Carry out target detection to obtain parking space detection results and vehicle detection results, which improves the accuracy of target detection results based on point cloud data of millimeter wave radar; in addition, based on the vehicle detection results obtained from target detection, it is also possible to detect adjacent vehicles based on the vehicle detection results. The area size between the two is analyzed to obtain the parking space analysis results, and the parking space analysis results are fused with the parking space detection results obtained from target detection to obtain the final parking space detection result, which improves the reliability and accuracy of parking space detection.

In some embodiments, the vehicle detection results may include position information, size information, and rotation angle information of the vehicle target frame. The parking space detection results may include position information, size information, and rotation angle information of the parking space detection frame.

For example, considering that in actual scenarios, parking spaces are not all parallel or perpendicular to the road, and there are also inclined parking spaces. When the vehicle is parked in a parking space, the vehicle is not parallel or perpendicular to the road, but has a certain rotation angle (relative to the situation of being parallel or perpendicular to the road).

Correspondingly, in order to improve the accuracy of parking space detection and the applicable scenarios of the extended solution, when performing vehicle target detection and parking space target detection, in addition to detecting the position and size of the vehicle/parking space detection frame, the vehicle/parking space detection frame can also be detected. Rotation angle information of the parking space.

For example, a classification method can be used to obtain the rotation angle of the vehicle/parking space.

For example, the detection of rotation angle can be completed through two steps: direction classification and angle classification. Direction classification can include dividing the vehicle rotation angle into 0 to 180 degrees (positive direction) and -180 degrees to 0 degrees (negative direction), using a binary branch to predict the direction, and using a binary cross-entropy loss for supervision. Angle classification refers to k degrees as the resolution, which can be divided into 180/k categories.

Use classification loss such as cross entropy for supervision, and calculate the rotation angle of the vehicle/parking space based on the category with the highest probability. If the direction prediction is positive and the angle prediction probability of type i is the highest, then the rotation angle is 180/k*i.

For example, the positive or negative rotation angle may be predefined.

Taking the road as a north-south direction and parking spaces on both sides of the road as an example, it can be defined that when the parking space is on the left side of the road (west direction), the rotation angle is negative; when the parking space is on the right side of the road (east direction), the rotation angle is positive. .

In step S120, based on the vehicle detection results, the area size between adjacent vehicles is analyzed to obtain parking space analysis results, which may include:

For the two vehicle target frames with matching rotation angles and the closest center distance, the parking space analysis result is obtained based on the size of the area between adjacent sides of the two vehicle target frames.

For example, for any vehicle detected in step S110, other vehicle target frames that match the rotation angle of the vehicle target frame can be searched based on the rotation angle of the vehicle target frame, and the rotation angle of the vehicle target frame can be compared with the rotation angle of the vehicle target frame. Among the other matching vehicle target frames, the vehicle target frame closest to the center of the vehicle target frame is determined as the adjacent vehicle target frame of the vehicle target frame, based on the proximity between the vehicle target frame and the adjacent vehicle target frame. The size of the area between the edges is used to analyze the parking space analysis results between the two vehicle target frames.

It should be noted that in the embodiment of the present application, the rotation angle matching of the two vehicle target frames may include the rotation angles of the two vehicle target frames being the same, or the rotation angles of the two vehicle target frames being different, but the rotation angles of the two vehicle target frames are different. The difference in the rotation angles of the boxes is within the preset angle range.

In one example, the parking space analysis result obtained based on the size of the area between adjacent sides of the two vehicle target frames may include:

For any one of the two vehicle target frames, generate the largest rectangular frame with the side of the vehicle target frame adjacent to the other vehicle target frame as an edge and not overlapping with the other vehicle target frame, And use the largest rectangular frame as the candidate parking space analysis frame;

When the larger area of the candidate parking space analysis frames corresponding to the two vehicle target frames meets the preset size requirements, the larger area of the candidate parking space analysis frames corresponding to the two vehicle target frames is determined as the parking space. Analysis box.

For example, for any one of the two vehicle target frames with matching rotation angles and the closest center distance, the side of the vehicle target frame adjacent to the other vehicle target frame can be used as the side to generate a vehicle target frame corresponding to the vehicle target frame. The largest rectangular frame that does not overlap with another vehicle target frame is used as the candidate parking space analysis frame.

For example, assuming that the two vehicle target frames are frame 1 and frame 2 respectively, the side of frame 1 adjacent to frame 2 is side A1B1, and the side of frame 2 adjacent to frame 1 is side A2B2, then for the frame 1. Two rays perpendicular to side A1B1 can be drawn from points A1 and B1 respectively (assumed to be rays S1 and S2), and the intersection point of S1 and straight line A2B2 (assumed to be C1), and the intersection point of S2 and straight line A2B2 (assumed to be C1) can be obtained. Assume it is C2), determine the shorter of the line segments A1C1 and B1C2 (assume it is A1C1), and determine the rectangular box with the line segment A1C1 and the line segment A1B1 as the candidate. Select the parking space analysis box.

For example, according to the above method, for the two vehicle target frames, two candidate parking space analysis frames can be determined (the two candidate parking space analysis frames can completely overlap), and the areas of the two candidate parking space analysis frames can be compared to obtain A candidate parking space analysis frame with a larger area is determined, and it is determined whether the candidate parking space analysis frame meets the preset size requirements. If the candidate parking space analysis frame meets the preset size requirements, the candidate parking space analysis frame is determined to be a parking space analysis frame.

For example, if the candidate parking space analysis frame meets the preset size requirements, it may include that one or more of the length, width, and area of the candidate parking space analysis frame meets the preset size requirements (that is, it may be based on one or more of the length, width, and area). setting requirements, such as thresholds).

For example, taking the area as an example, the area of the candidate parking space analysis box meeting the preset area requirements may include the area of the candidate parking space analysis box being greater than the first area threshold and less than the second area threshold, and the first area threshold being less than the second area threshold. .

In some embodiments, the parking space detection result may include at least one parking space detection frame, and the parking space analysis result may include at least one parking space analysis frame. In step S130, the parking space detection result and the parking space analysis result are fused to obtain the parking space detection result. Final results include:

For any parking space analysis frame, determine the intersection ratio of each parking space detection frame and the parking space analysis frame;

When there is a target parking space detection frame whose intersection-to-union ratio with the parking space analysis frame is greater than a preset threshold, the larger area of the parking space analysis frame and the target parking space detection frame is determined as the candidate parking space area.

For example, when the parking space detection frame and the parking space analysis frame are obtained in the above manner, for any parking space analysis frame, each parking space detection frame can be traversed in turn, and the intersection of each parking space detection frame and the parking space analysis frame can be determined respectively. (Intersection over Union, referred to as IOU), and determine whether there is a target parking space detection frame whose intersection and union ratio with the parking space analysis frame is greater than the preset threshold.

For example, for any parking space analysis frame, if it is determined that there is a target parking space detection frame whose intersection-to-union ratio with the parking space analysis frame is greater than a preset threshold, for other parking space analysis frames, there may be no need to interact with the target parking space. The detection frame performs intersection and union ratio calculation.

It should be noted that in the embodiments of the present application, the implementation method of fusing the parking space detection results and the parking space analysis results to obtain the final parking space detection result is not limited to the method described in the above embodiments. For example, due to actual scenarios The area of the parking space is usually known. Therefore, for any parking space analysis frame, the area of the overlapping area of each parking space detection frame and the parking space analysis frame can be determined respectively, and when there is an overlapping area with the parking space analysis frame, the area is larger than the predetermined area. When a threshold target parking space detection frame is set, the larger area of the parking space analysis frame and the target parking space detection frame is determined as the candidate parking space area. Or, because for a real parking space, when the parking space detection result and parking space analysis result of the parking space are obtained in the above manner, the distance between the center points of the parking space detection result and the parking space analysis result will not be different. is too large. Therefore, for any parking space analysis frame, the distance between each parking space detection frame and the center point of the parking space analysis frame can be determined separately. If the distance between the parking space detection frame and the center point of the parking space analysis frame is less than the preset threshold In the case of a target parking space detection frame, the larger area of the parking space analysis frame and the target parking space detection frame is determined as the candidate parking space area.

In one example, when there is a target parking space detection frame whose intersection-to-union ratio with the parking space analysis frame is greater than a preset threshold, the parking space detection method provided by the embodiment of the present application may also include:

Set the first confidence label for the one with the larger area between the parking space analysis frame and the target parking space detection frame;

The parking space detection method provided by the embodiment of this application may also include:

When there is no target parking space detection frame whose intersection-to-union ratio with the parking space analysis frame is greater than the preset threshold, determine the parking space analysis frame as a candidate parking space area, and set a second confidence label for the candidate parking space area;

For any parking space detection frame, if there is no target parking space analysis frame whose intersection-to-union ratio with the parking space detection frame is greater than the preset threshold, the parking space detection frame is determined as a candidate parking space area, and the setting for the candidate parking space area is third confidence label;

Among them, the confidence corresponding to the first confidence label, the second confidence label and the third confidence label decreases in sequence.

For example, considering that millimeter wave radar has a higher scanning accuracy for actually existing objects, the reliability of the vehicle detection results obtained based on the point cloud data of the millimeter wave radar will be higher than the reliability of the parking space detection results. The reliability of the parking space analysis results obtained by analyzing the vehicle detection results in the above manner is usually higher than the reliability of the parking space detection results.

Correspondingly, for any parking space analysis frame, if there is a target parking space detection frame whose intersection-to-union ratio with the parking space analysis frame is greater than the preset threshold, the larger area of the parking space analysis frame and the target parking space detection frame can be is determined as a candidate parking space area, and a first confidence label is set for the candidate parking space area.

For any parking space analysis frame, if there is no target parking space detection frame whose intersection and union ratio with the parking space analysis frame is greater than the preset threshold, the parking space analysis frame can be determined as a candidate parking space area, and the candidate parking space area can be set up Second confidence label.

For any parking space detection frame, if there is no target parking space analysis frame whose intersection-to-union ratio with the parking space detection frame is greater than the preset threshold, the parking space detection frame is determined as a candidate parking space area, and the setting for the candidate parking space area is Third confidence label.

Among them, the confidence corresponding to the first confidence label, the second confidence label and the third confidence label decreases in sequence, that is, the candidate parking space area obtained by the fusion process, the candidate parking space area obtained by analysis, and the candidate parking space area obtained by detection. Confidence levels gradually decrease.

In one example, the parking space detection method provided by the embodiment of the present application may also include:

For any candidate parking space area, if there is an obstacle detection result in the candidate parking space area, determine the key edge to be adjusted for the candidate parking space area based on the detection position information of the obstacle;

Translate the key edges to be adjusted in the candidate parking space area to obtain a rectangular frame that does not cover the obstacle;

When the rectangular frame meets the preset size requirements, update the candidate parking space area to the rectangular frame area;

If the rectangular frame does not meet the preset size requirements, the candidate parking space area is deleted.

For example, considering the actual scenario, there may be some obstacles in the free parking space, such as triangular pyramids, pillars, etc. In order to prevent the vehicle from colliding with obstacles during automatic parking, the determined parking space area can also be adjusted based on the detected obstacle information.

For example, for any candidate parking space area, if there is an obstacle detection result in the candidate parking space area, the key edge to be adjusted for the candidate parking space area can be determined based on the detection position information of the obstacle.

For example, considering that in actual scenarios, the width of the parking space is usually significantly larger than the width of the vehicle, but the length of the parking space usually matches the length of the vehicle, therefore, in the case of obstacles, the candidate parking space area close to the obstacle can be The long side is used as the key side to be adjusted.

For example, when the key edges to be adjusted are determined, the key edges to be adjusted in the candidate parking area can be translated to obtain a rectangular frame that does not cover the obstacle. For example, the largest rectangular frame that does not cover the obstacle is obtained. Rectangle.

After completing the adjustment of the candidate parking space area, it can be determined whether the adjusted rectangular frame meets the preset size requirements.

In order to enable those skilled in the art to better understand the embodiments of the present application, the embodiments of the present application are described below with reference to specific examples.

In this embodiment, in order to realize parking space detection, millimeter wave radars can be installed at the four corners of the vehicle respectively, and the schematic diagram thereof can be shown in Figure 2 .

In this embodiment, a schematic diagram of the parking space detection system framework can be seen in Figure 3 . As shown in Figure 3, the parking space detection system framework can include a radar preprocessing module, a target detection module, a parking space analysis module, a parking space fusion module and a parking space tracking module.

The radar preprocessing module can use the vehicle-mounted millimeter wave radar to obtain point cloud data and unify the point cloud data into the body coordinate system (the body coordinate system can take the center of the vehicle's rear axle as the origin, the forward direction as the y-axis, and the right direction as the x-axis. The top is the z-axis), and the point cloud data is rasterized to generate a raster density map.

The target detection module can obtain the category and location information of vehicle targets, parking space targets and general obstacle targets based on the grid density map.

The parking space analysis module analyzes the location of the vehicle target and obtains free parking space information. The parking space position obtained through analysis and the parking space position obtained by the detection model are sent to the parking space fusion module for processing to obtain the fusion detection parking space (i.e., the final result of the above-mentioned parking space detection). Finally, the fusion detected parking space is input into the parking space tracking module for multi-frame stabilization to obtain the final parking space detection result.

Each module is described in detail below.

1. Radar preprocessing module

The four angular millimeter-wave radars are calibrated with the body coordinate system (the body coordinate system has the center of the rear axle of the vehicle as the origin, the forward direction is the y-axis, the right direction is the x-axis, and the upper direction is the z-axis), and is converted to the body coordinates through the calibration parameters system to obtain surrounding obstacle information. For the point cloud data of a single frame or multiple frames superimposed, through rasterization processing, the number of data points falling into each raster is counted, and a raster density map is generated.

2. Target detection module

Based on the raster density map, deep learning network is used for target detection.

For example, the yolov3 network model can be used for target detection. The yolov3 network is an anchor-based target detection network.

For example, the traditional yolov3 network model is usually used in the image coordinate system, and the detected targets are all positive Cross rectangular frame. However, in the vehicle body coordinate system, the vehicle target frame and the parking space target frame may not be orthogonal frames. Therefore, the yolov3 network needs to be improved to adapt to the detection of rotating target frames.

For example, when the traditional yolov3 network model returns the position of the target frame, it generally returns the center point (x, y) of the target frame, as well as the width and height (w, h) of the target frame (which can also be called length and width). ). In this embodiment, the improved yolov3 network model also needs to obtain rotation angle information in addition to the parameters of the orthogonal frame.

For example, the detection of rotation angle can be completed through two steps: direction classification and angle classification. Direction classification can include dividing the vehicle rotation angle into 0 to 180 degrees and -180 degrees to 0 degrees, using a binary branch to predict the direction, and using a binary cross-entropy loss for supervision. Angle classification refers to k degrees as the resolution, which can be divided into 180/k categories.

Among them, the detection model effect can be shown in Figure 4. The types of detection targets include vehicles, parking spaces and general obstacles. Limited by the quality of radar imaging, sometimes point cloud features are not obvious, which may affect the detection of parking spaces. Therefore, vehicle targets can be used to perform analysis on parking space targets to supplement the analysis.

3. Parking space analysis module

The parking space analysis module can estimate available parking spaces based on vehicle detection results. The parking space analysis module may include a neighboring target matching submodule, a neighboring corner point acquisition submodule, a candidate area generation submodule, and a candidate area selection submodule, and its schematic diagram can be shown in Figure 5.

Neighboring target matching submodule: Matches neighboring vehicle targets. For each vehicle target, look for candidate vehicle targets that are similar to its status (the heading angle deviation is less than a certain threshold, that is, the difference in rotation angle is less than a certain threshold), and then select the vehicle target with the closest center distance to form a matching pair of adjacent targets, such as A and B, B and C, etc. in Figure 6.

Adjacent corner point acquisition sub-module: Match pairs of adjacent targets, find their two adjacent edges, and select the corresponding corner points, such as the solid corner points of A and B and the hollow corner points of B and C in Figure 6.

Candidate area generation sub-module: Match pairs of adjacent targets, and generate rectangular boxes with adjacent edges and corners as boundaries. In this way, each vehicle target will generate a candidate parking area, as shown in Figure 7.

Candidate region selection submodule: Each neighboring target matching pair will generate two candidate regions. Candidate areas can be evaluated, such as selecting the candidate area with the largest area. Subsequently, certain restrictions are placed on the length, width and/or area of the candidate areas (ie, the above-mentioned preset size requirements), and candidate areas that are too large or too small are removed.

For example, as shown in Figure 7, the candidate regions generated by A and B have smaller width and area and should be deleted.

4. Parking space integration module

The parking space fusion module merges the parking spaces obtained by the detection model (i.e., the above-mentioned parking space detection results) and the parking spaces obtained by analysis (i.e., the above-mentioned parking space analysis results) to eliminate abnormal parking spaces. The parking space fusion module may include a parking space target merging sub-module and an obstacle elimination sub-module, the schematic diagram of which can be shown in Figure 8.

Parking space target merging sub-module: Merges the parking space target obtained by the detection model (which can be called "detected parking space") and the parking space target obtained by analysis (which can be called "analyzed parking space").

For example, the IOU (intersection over union ratio, A and B are the parking space target positions). If the IOU is greater than a certain threshold, it means that the detected parking space and the analyzed parking space match. Select the parking space with the largest area among the detected parking space and the analyzed parking space, and add the confidence label 1 (that is, the first confidence level mentioned above Label).

For the analyzed parking spaces that do not match the detected parking spaces, add confidence label 2 (that is, the above-mentioned second confidence label). For the detected parking spaces that do not match the analyzed parking spaces, add confidence label 3 (that is, the third confidence label mentioned above).

Obstacle elimination sub-module: Considering that there may be obstacles such as triangular cones and pillars in free parking spaces, the candidate parking space area can be fine-tuned based on the detected obstacle information.

For example, based on the location information of the candidate parking area and the location information of the obstacle, the key edge to be adjusted can be selected, the key edge can be translated, and the obstacle area is excluded. As shown in Figure 9, the key boundary of the parking space is adjusted to below the obstacle. . Finally, the size requirements of the adjusted candidate parking space areas are judged, unreasonable candidate parking space areas are deleted, and the final result of the parking space detection is obtained.

5. Parking space tracking module

The parking space tracking module can stabilize the parking space detection results in multiple frames and reduce the chance of single-frame detection results.

The parking space detection method provided by the embodiment of the present application has been described above. The parking space detection device provided by the embodiment of the present application is described below.

Please refer to Figure 10, which is a schematic structural diagram of a parking space detection device provided by an embodiment of the present application. As shown in Figure 10, the parking space detection device may include a data preprocessing unit 1010, a target detection unit 1020, an analysis unit 1030 and a fusion unit. 1040.

The data preprocessing unit 1010 is used to obtain point cloud data around the vehicle body using a vehicle-mounted millimeter wave radar, and rasterize the point cloud data to obtain a raster density map.

The target detection unit 1020 is configured to use a deep learning algorithm to perform target detection based on the grid density map, and obtain parking space detection results and vehicle detection results.

The analysis unit 1030 is configured to analyze the size of the area between adjacent vehicles based on the vehicle detection results to obtain parking space analysis results.

The fusion unit 1040 is used to fuse the parking space detection results and the parking space analysis results to obtain the final parking space detection result.

In some embodiments, the vehicle detection results include position information, size information and rotation angle information of the vehicle target frame;

The analysis unit 1030 analyzes the size of the area between adjacent vehicles based on the vehicle detection results. When obtaining the parking space analysis results, it is further used to:

In some embodiments, when obtaining the parking space analysis result based on the size of the area between adjacent sides of the two vehicle target frames, the analysis unit 1030 is further used to:

In some embodiments, the parking space detection result may include at least one parking space detection frame, and the parking space analysis result may include at least one parking space analysis frame, and the fusion unit 1040 performs on the parking space detection result and the parking space analysis result. After fusion, when the final result of parking space detection is obtained, it is further used for:

In some embodiments, the fusion unit 1040 is also configured to combine the parking space analysis frame and the target parking space detection frame when there is a target parking space detection frame whose intersection-to-union ratio with the parking space analysis frame is greater than a preset threshold. The one with the larger area is set as the first confidence label;

The fusion unit 1040 is also used to:

Wherein, the confidences corresponding to the first confidence label, the second confidence label and the third confidence label decrease in sequence.

In some embodiments, the fusion unit 1040 is also configured to: for any candidate parking space area, if there is an obstacle detection result in the candidate parking space area, determine the candidate parking space based on the detection position information of the obstacle. The key edges to be adjusted in the area; the key edges to be adjusted in the candidate parking space area are translated to obtain a rectangular frame that does not cover the obstacle; when the rectangular frame meets the preset size requirements, the candidate parking space area is updated to The rectangular frame area; if the rectangular frame does not meet the preset size requirements, delete the candidate parking space area.

An embodiment of the present application provides an electronic device, including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor is used to execute the machine-executable instructions to implement the parking space described above. Detection method.

Please refer to Figure 11, which is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application. The electronic device may include a processor 1101 and a memory 1102 storing machine-executable instructions. Processor 1101 and memory 1102 may communicate via system bus 1103 . Furthermore, by reading and executing machine-executable instructions corresponding to the parking space detection logic in the memory 1102, the processor 1101 can execute the parking space detection method described above.

Memory 1102 as referred to herein may be any electronic, magnetic, optical, or other physical storage device that may contain or store information, such as executable instructions, data, and the like. For example, machine-readable storage media can be: RAM (Radom Access Memory, random access memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard drive), solid state drive, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or a combination thereof.

Embodiments of the present application also provide a machine-readable storage medium, such as the memory 1102 in Figure 11. The machine-readable storage medium stores machine-executable instructions. When the machine-executable instructions are executed by the processor, the The processor implements the parking space detection method described above. For example, the machine-readable storage medium may be ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations are mutually exclusive. any such actual relationship or sequence exists between them. Furthermore, the terms "comprises," "comprises," or any other variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also those not expressly listed other elements, or elements inherent to the process, method, article or equipment. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or apparatus that includes the stated element.

The above are only some embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application shall be included in the scope of protection of this application.

Claims

A parking space detection method includes:

Use vehicle-mounted millimeter wave radar to obtain point cloud data around the vehicle body, and rasterize the point cloud data to obtain a raster density map;

Based on the grid density map, a deep learning algorithm is used for target detection to obtain parking space detection results and vehicle detection results;

Based on the vehicle detection results, analyze the size of the area between adjacent vehicles to obtain parking space analysis results;

The parking space detection results and the parking space analysis results are fused to obtain the final parking space detection result.
The method according to claim 1, wherein the vehicle detection result includes position information, size information and rotation angle information of the vehicle target frame;

Based on the vehicle detection results, the area size between adjacent vehicles is analyzed to obtain parking space analysis results, including:

For the two vehicle target frames with matching rotation angles and the closest center distance, the parking space analysis result is obtained based on the size of the area between adjacent sides of the two vehicle target frames.
The method according to claim 2, wherein obtaining the parking space analysis result based on the size of the area between adjacent sides of the two vehicle target frames includes:

For any one of the two vehicle target frames, generate the largest rectangular frame with the side of the vehicle target frame adjacent to the other vehicle target frame as an edge and not overlapping with the other vehicle target frame, And use the largest rectangular frame as the candidate parking space analysis frame;

When the larger area of the candidate parking space analysis frames corresponding to the two vehicle target frames meets the preset size requirements, the larger area of the candidate parking space analysis frames corresponding to the two vehicle target frames is determined as the parking space. Analysis box.
The method according to claim 1, wherein the parking space detection result includes at least one parking space detection frame, and the parking space analysis result includes at least one parking space analysis frame,

The parking space detection results and the parking space analysis results are fused to obtain the final parking space detection results, including:

For any parking space analysis frame, determine the intersection ratio of each parking space detection frame and the parking space analysis frame;

When there is a target parking space detection frame whose intersection-to-union ratio with the parking space analysis frame is greater than a preset threshold, the larger area of the parking space analysis frame and the target parking space detection frame is determined as the candidate parking space area.
The method of claim 4, further comprising:

In the case where there is a target parking space detection frame whose intersection-to-union ratio with the parking space analysis frame is greater than the preset threshold, set a first confidence label for the larger area of the parking space analysis frame and the target parking space detection frame;

When there is no target parking space detection frame whose intersection-to-union ratio with the parking space analysis frame is greater than the preset threshold, determine the parking space analysis frame as a candidate parking space area, and set a second confidence label for the candidate parking space area;

For any parking space detection frame, if there is no target parking space analysis frame whose intersection-to-union ratio with the parking space detection frame is greater than the preset threshold, the parking space detection frame is determined as a candidate parking space area, and the setting for the candidate parking space area is third confidence label;

Wherein, the confidences corresponding to the first confidence label, the second confidence label and the third confidence label decrease in sequence.
The method according to claim 4 or 5, further comprising:

For any candidate parking space area, if there is an obstacle detection result in the candidate parking space area, determine the key edge to be adjusted for the candidate parking space area based on the detection position information of the obstacle;

Translate the key edges to be adjusted in the candidate parking space area to obtain a rectangular frame that does not cover the obstacle;

When the rectangular frame meets the preset size requirements, update the candidate parking space area to the rectangular frame area;

If the rectangular frame does not meet the preset size requirements, the candidate parking space area is deleted.
A parking space detection device includes:

A data preprocessing unit is used to obtain point cloud data around the vehicle body using a vehicle-mounted millimeter wave radar, and rasterize the point cloud data to obtain a raster density map;

A target detection unit, used to perform target detection using a deep learning algorithm based on the grid density map, and obtain parking space detection results and vehicle detection results;

An analysis unit is used to analyze the size of the area between adjacent vehicles based on the vehicle detection results to obtain parking space analysis results;

A fusion unit is used to fuse the parking space detection results and the parking space analysis results to obtain the final parking space detection result.
The device according to claim 7, wherein the vehicle detection result includes position information, size information and rotation angle information of the vehicle target frame;

The analysis unit is further used for:

For the two vehicle target frames with matching rotation angles and the closest center distance, the parking space analysis result is obtained based on the size of the area between adjacent sides of the two vehicle target frames.
The device according to claim 8, wherein when the analysis unit obtains the parking space analysis result based on the size of the area between adjacent sides of the two vehicle target frames, it is further used to:

For any one of the two vehicle target frames, generate the largest rectangular frame with the side of the vehicle target frame adjacent to the other vehicle target frame as an edge and not overlapping with the other vehicle target frame, And use the maximum rectangular frame as Analysis box for candidate parking spaces;

When the larger area of the candidate parking space analysis frames corresponding to the two vehicle target frames meets the preset size requirements, the larger area of the candidate parking space analysis frames corresponding to the two vehicle target frames is determined as the parking space. Analysis box.
The device according to claim 7, wherein the parking space detection result includes at least one parking space detection frame, and the parking space analysis result includes at least one parking space analysis frame,

The fusion unit is further used for:

For any parking space analysis frame, determine the intersection ratio of each parking space detection frame and the parking space analysis frame;

When there is a target parking space detection frame whose intersection-to-union ratio with the parking space analysis frame is greater than a preset threshold, the larger area of the parking space analysis frame and the target parking space detection frame is determined as the candidate parking space area.
The device according to claim 10, wherein the fusion unit is also used for:

In the case where there is a target parking space detection frame whose intersection-to-union ratio with the parking space analysis frame is greater than the preset threshold, set a first confidence label for the larger area of the parking space analysis frame and the target parking space detection frame;

When there is no target parking space detection frame whose intersection-to-union ratio with the parking space analysis frame is greater than the preset threshold, determine the parking space analysis frame as a candidate parking space area, and set a second confidence label for the candidate parking space area;

For any parking space detection frame, if there is no target parking space analysis frame whose intersection-to-union ratio with the parking space detection frame is greater than the preset threshold, the parking space detection frame is determined as a candidate parking space area, and the setting for the candidate parking space area is third confidence label;

Wherein, the confidences corresponding to the first confidence label, the second confidence label and the third confidence label decrease in sequence.
The device according to claim 10 or 11, wherein the fusion unit is also used for:

For any candidate parking space area, if there is an obstacle detection result in the candidate parking space area, determine the key edge to be adjusted for the candidate parking space area based on the detection position information of the obstacle;

Translate the key edges to be adjusted in the candidate parking space area to obtain a rectangular frame that does not cover the obstacle;

When the rectangular frame meets the preset size requirements, update the candidate parking space area to the rectangular frame area;

If the rectangular frame does not meet the preset size requirements, the candidate parking space area is deleted.
An electronic device includes a processor and a memory, the memory stores machine-executable instructions that can be executed by the processor, and the processor is used to execute the machine-executable instructions to implement claims 1-6 any of the methods described.
A machine-readable storage medium in which machine-executable instructions are stored, wherein when executed by a processor, the machine-executable instructions cause the processor to implement any one of claims 1-6 method described in the item.