WO2021000800A1

WO2021000800A1 - Reasoning method for road drivable region and device

Info

Publication number: WO2021000800A1
Application number: PCT/CN2020/098642
Authority: WO
Inventors: 陈名扬; 要志良
Original assignee: 华为技术有限公司
Priority date: 2019-06-29
Filing date: 2020-06-28
Publication date: 2021-01-07
Also published as: CN112230642A; CN115291596A; CN112230642B

Abstract

Provided is an automatic driving technology in the field of artificial intelligence, specifically relating to a reasoning method for road drivable region, including: acquiring a perceptual drivable region; checking the perceptual drivable region to obtain a first region and a second region; if the first region does not cover a region of interest ROI, reasoning on the second region according to perceptual memory information of the drivable region, to obtain a third region and a fourth region; if the first region and the third region do not cover the ROI, reasoning on the fourth region according to a drivable position point to obtain a fifth region; and determining the first region, the third region and the fifth region as the road drivable region. A reasoning device for road drivable region is also disclosed. It is beneficial to obtain an accurate road drivable region when the road drivable region perception result is abnormal, thereby improving the safety, system availability and user experience of the automatic driving system.

Description

Reasoning method and device for road driving area

This application claims the priority of a Chinese patent application filed with the China Intellectual Property Office on June 29, 2019, the application number is 201910584332.1, and the invention title is "Road Driveable Area Reasoning Method and Device", the entire content of which is incorporated herein by reference Applying.

Technical field

The present invention relates to the field of equipment artificial intelligence, and in particular to a method and device for reasoning on road traversable areas in intelligent auxiliary driving or automatic driving technology.

Background technique

Artificial Intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making. Research in the field of artificial intelligence includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and basic AI theories.

Autonomous driving is a mainstream application in the field of artificial intelligence. Autonomous driving technology relies on the collaboration of computer vision, radar, monitoring devices, and global positioning systems to allow motor vehicles to achieve autonomous driving without the need for human active operations. Self-driving vehicles use various computing systems to help transport passengers from one location to another. Some autonomous vehicles may require some initial input or continuous input from an operator (such as a navigator, driver, or passenger). The self-driving vehicle allows the operator to switch from the manual mode to the self-driving mode or a mode in between. Since autonomous driving technology does not require humans to drive motor vehicles, it can theoretically effectively avoid human driving errors, reduce traffic accidents, and improve highway transportation efficiency. Therefore, autonomous driving technology has received more and more attention.

For autonomous driving systems, road-driving area perception, driving route decision planning, and control are indispensable core key technologies. The decision-making and planning of the driving route is realized based on the perception result of the drivable area of the road. Therefore, the perception result of the drivable area of the road directly affects the performance of the automatic driving system. At present, the mainstream solution in the industry is to perceive the drivable area of the road based on the visual detection image of the camera and the point cloud measurement of the lidar. Camera vision detection images combined with machine learning algorithms are suitable for the extraction of road features such as lane lines and turning arrows. The lidar three-dimensional measurement point cloud is suitable for the extraction of positive and negative obstacles on the edge of the road and on the road surface. However, sensors such as cameras and lidars are easily affected by the environment, and lighting conditions and extreme weather will affect the accuracy of the road-driving area perception results.

In traffic jams, the occlusion caused by surrounding vehicles will also affect the accuracy of the perception of the road's drivable area, and the current lidar is expensive and is not a standard configuration for all mass-produced models. Therefore, the perception result of the drivable area of the road is uncertain and cannot reach 100% reliability, which will lead to short-term abnormalities or even long-term failure when the vehicle is driving in the automatic driving mode.

The autonomous driving system, which is based entirely on the road-driving area perception results for driving route decision planning and control, cannot cope with scenarios where the drivable area perception results are abnormal. The abnormality of the perception result of the drivable area of the road will directly lead to abnormal driving route planning decisions and control, and then affect the safety of autonomous driving. When the perception result of the drivable area of the road is abnormal, some automatic driving systems will automatically stop or remind the driver to take over driving. In this case, the safety of the autopilot system is improved, but the usability of the autopilot system is strongly dependent on the perception ability, and the autopilot function is unavailable when the perception result of the drivable area of the road is abnormal, and the user experience is affected. For this reason, some automatic driving systems have developed a target-following driving function for abnormal perceptions of the road's drivable area, and directly generate a planned trajectory according to the tracking target trajectory. This function increases the scene coverage of the automatic driving system, but due to the lack of road information input, the performance of the automatic driving system is strongly dependent on the behavior of following the car, and the system safety has hidden dangers.

In summary, the uncertainty of the perception results of the drivable area of the road directly affects the safety, system availability and user experience of the autonomous driving system.

Summary of the invention

The embodiment of the present invention provides a road drivable area reasoning method and device. Using the embodiment of the present invention is beneficial to obtain an accurate road drivable area when the road drivable area perception result is abnormal, thereby improving the safety and security of the automatic driving system. System availability and user experience.

In the first aspect, an embodiment of the present invention provides a road drivable area reasoning method, including:

Obtain the perceived drivable area; perform verification on the perceived drivable area to obtain the first area and the second area, where the first area is the drivable area with reliable verification and the second area is the drivable area with unreliable verification Area; if the first area does not cover the area of interest ROI, the second area is inferred based on the perceptual memory information of the drivable area to obtain the third area and the fourth area, the third area is the perceptual memory area overlapping the second area The fourth area is the area not covered by the sensory memory area in the second area; if the first area and the third area do not cover the ROI, the fourth area is inferred based on the driving position point to obtain the fifth area; The fifth area is a drivable area in the fourth area; the first area, the third area and the fifth area are determined as road drivable areas.

By verifying and inferring the perceived drivable area, an accurate road drivable area can be obtained. When route planning is based on the accurate road drivable area, collisions with obstacles and surrounding vehicles can be avoided, thereby improving the performance of the autonomous driving system. Security, system availability and user experience.

In a feasible embodiment, verifying the perceived drivable area to obtain the first area and the second area includes:

Determine whether there is a road boundary on both sides of the perceived drivable area; if it is determined that the road boundary on both sides of the perceived drivable area exists, divide the perceptible drivable area to obtain multiple sub-areas; determine that sub-area I satisfies condition 1- Each item in condition 4; if sub-area I meets each of conditions 1-condition 4, sub-area I is determined to be a sub-area with reliable verification; if sub-area I does not meet any of conditions 1-condition 4 Item, it is determined that sub-region I is an area with unreliable verification; among them, sub-region I is any one of a plurality of sub-regions, the first region is an area composed of sub-regions with reliable verification among multiple sub-regions, and the second region It is an area composed of sub-areas with unreliable verification among multiple sub-areas.

In a feasible embodiment, condition 1 to condition 4 are:

Condition 1: The width of sub-region I satisfies the following conditions:

k _min W≤w _i ≤k _max W

Among them, w _i is the width of the sub-area I, and W is determined according to the experience width of the drivable area and the memory width of the drivable area;

Condition 2: The angle between the boundary of the sub-region I and the boundary of the adjacent sub-region is not greater than the first preset angle;

Condition 3: The distance between the boundary of the sub-region I and the boundary of the perceptual memory area verified before the current moment is not greater than the preset width;

Condition 4: The ratio of the drivable position points in the subregion I is greater than the preset ratio.

When verifying the perceived drivable area, not only the perceptual memory area is taken into account, but also road common sense information, such as the detection of boundary integrity, width, and boundary angles of adjacent areas, to improve the ultimate road drivability. The accuracy of the area can avoid collisions with obstacles and surrounding vehicles when planning routes based on accurate road drivable areas, thereby improving the safety, system availability and user experience of the automatic driving system.

In a feasible embodiment, the perceptual memory information includes perceptual memory grid maps at multiple historical moments and the drivability value of each grid in each perceptual memory grid map, and the second area is performed according to the perceptual memory information. Reasoning to get the third area and the fourth area, including:

Convert the sensory memory grid maps of multiple historical moments from the vehicle coordinate system of the vehicle at the historical moment to the world coordinate system to obtain multiple world grid maps; obtain the reasoning area, which is multiple world grids The area where the grid map overlaps the second area; convert the inference area from the world coordinate system to the vehicle coordinate system of the vehicle at the current moment to obtain the first inference grid map; according to the perceptual memory of the grid in the grid map The driving ability value calculates the driving ability value of each grid in the first inference grid map; the third area and the fourth area are determined according to the driving ability value of each grid in the first inference grid map; the third area It is an area composed of grids in the first inference grid map whose drivability value is greater than the first threshold; the fourth area is an area composed of grids in the first inference grid map whose drivability value is not greater than the first threshold.

The driving area in the second area is obtained by reasoning on the second area through the perceptual memory grid map, which realizes that the reasoning of the driving area can be continued when the driving area is abnormal, and the perception of the automatic driving system is reduced. The dependence of the driving area increases the fault tolerance of the automatic driving system to the real-time senseable driving area, and improves the reliability and safety of the automatic driving system.

In a feasible embodiment, the perceptual memory grid maps of multiple historical moments are respectively transformed from the vehicle coordinate system of the vehicle at the historical moment to the world coordinate system to obtain multiple world grid maps, including:

According to the first conversion formula, the perceptual memory grid maps of multiple historical moments are respectively converted from the vehicle coordinate system of the vehicle at the historical moment to the world coordinate system to obtain multiple world grid maps;

Among them, the first conversion formula is:

Among them, (x _vt0 ,y _vt0 ) are the coordinates of any _drivable location point P in the perception memory grid map at historical time t0 in the vehicle coordinate system of the own vehicle, and (x _wt0 ,y _wt0 ) is the _drivable location point The coordinates of P in the world coordinate system,

Is the first conversion matrix,

First conversion matrix

(x _t0 , y _t0 ) are the coordinates of the vehicle at historical time t0 in the world standard system, and θ _t0 is the heading angle of the vehicle at historical time t0.

In a feasible embodiment, converting the inference area from the world coordinate system to the vehicle coordinate system of the vehicle at the current moment to obtain the first inference grid map includes:

Convert the inference area from the world coordinate system to the vehicle coordinate system of the vehicle at the current moment according to the second conversion formula to obtain the first inference grid map;

Among them, the second conversion formula is:

(x _wp ,y _wp ) is the coordinates of any travelable position point _P'in the inference area in the world coordinate system, (x _vp ,y _vp ) is the vehicle coordinate system of the self-vehicle at the current moment The coordinates below,

Is the second conversion matrix;

Second conversion matrix

(x ₀ , y ₀ ) are the coordinates of the vehicle at the current moment in the world coordinate system, and θ ₀ is the heading angle of the vehicle at the current moment.

In a feasible embodiment, calculating the drivability value of each grid in the first inference grid map according to the drivability value of each grid in the perceptual memory grid map includes:

Perform a weighted summation on the drivability value of multiple historical moments corresponding to the grid in the p-th column and the q-th row in the first inference grid map to obtain the drivability value of each grid in the first inference grid map The drivability value of multiple historical moments is the drivability value of the corresponding grid in the perceptual memory grid map of the grid of the p-th column and the q-th row at multiple historical moments;

Among them, the drivability value of the grid in the p-th column and the q-th row in the first inference grid map is:

As historic time t 'corresponding to the sensing grid map raster memory may driving ability value, _{k' t} 'is

the weight of.

By calculating the drivability value of each grid, the drivable area can be determined using the grid as the basic unit, which improves the accuracy of the drivable area, and further improves the reliability and safety of the automatic driving system.

In a feasible embodiment, the fourth area is inferred based on the drivable location point to obtain the fifth area, including:

Obtain the location point to be inferred from the driveable location point, which is the driveable location point located in the area where the fourth area overlaps the ROI; convert the coordinates of the location to be inferred from the world coordinate system to that of the vehicle Under the vehicle coordinate system, in order to obtain the driving area to be inferred, the driving area to be inferred is the area formed by the inferred position points in the vehicle coordinate system of the own vehicle; grid division is performed on the driving area to be inferred to obtain the second Inference grid map; calculate the drivability value of each grid according to the drivable position point information in each grid in the second inference grid map; determine the fifth area according to the drivability value of each grid, The fifth area is an area composed of grids with a drivability value greater than the second threshold in the second inference grid map.

In the case of short-term or long-term abnormal or long-term abnormality in the perception of information or results in the drivable area, the fourth area can be reasoned through the drivable location points, and the driving route decision planning can be made according to the drivable area obtained by reasoning, which avoids the actual effect of the automatic driving system. The scope of use of the automatic driving system is increased, and the dependence of the automatic driving system on real-time sensing of the drivable area information is reduced, thereby increasing the fault tolerance of the automatic driving system to the real-time sensing of the drivable area information.

In a feasible embodiment, transforming the coordinates of the location point to be inferred from the world coordinate system to the vehicle coordinate system of the vehicle to obtain the travelable area to be inferred includes:

Convert the coordinates of the location point to be inferred according to the third conversion formula to obtain the driveable area to be inferred;

Among them, the third conversion formula is:

Among them, (x _dw , y _dw ) is the coordinate of any inferred position point D in the world coordinate system of the inferred position points, (x _dv , y _dv ) is the coordinate system of the inferred position D in the own vehicle The coordinates below,

Is the second conversion matrix,

Second conversion matrix

In a feasible embodiment, calculating the drivability value of each grid according to the drivable position point information in each grid in the second inference grid map includes:

The drivability values at different moments are calculated according to the drivable position point information in the i-th column and j-th row grid in the second inference grid map; the drivability values at different times are weighted and summed to obtain the first The drivability value of the grid in column i and row j;

Among them, the drivability value of the grid in the i-th column and the j-th row is

Is the drivability value at time t, k _t is

the weight of,

Is the number of travelable location points of surrounding vehicles traveling in the same direction obtained at time t in the grid of column i and row j,

Is the number of locations where the surrounding vehicles can travel in the reverse direction obtained at time t in the j-th row of the i-th column,

Is the number of safe driving position points obtained at time t in the grid of the i-th column and the j-th row,

Is the number of dangerous location points of the self-driving vehicle obtained at time t in the grid of the i-th column and the j-th row,

Is the number of travelable location points of surrounding vehicles in the same direction obtained at time t in the j-th grid,

Is the number of locations where the surrounding vehicles can travel in the reverse direction obtained at time t in the j-th grid,

Is the number of safe driving position points obtained at time t in the j-th grid,

It is the number of dangerous location points of the self-driving vehicle obtained at time t in the j-th grid.

In a feasible embodiment, after determining the drivability of the safe driving position and the driving dangerous position, the drivability value of the safe driving position is increased, and the drivability value of the dangerous driving position is reduced, which can increase The accuracy of the determined drivable area, and then the automatic driving system has the characteristics of "the more open the better".

In a feasible embodiment, the drivable position point includes the drivable position point of the self-vehicle. Before inferring the fourth area based on the drivable position point, the method further includes:

Obtain the driving position points of the own vehicle. The driving position points of the own vehicle include safe driving position points and driving risk position points; among them, obtaining the driving position points of the own vehicle includes: judging whether the driving mode of the own vehicle at its current position is manual Driving mode; if the driving mode of the own car at its current position is manual driving mode, the current position of the own car is determined to be a safe driving position; if the driving mode of the own car at its current position is automatic driving mode, it is determined Whether the vehicle has a collision risk or abnormal driving behavior at its current location; if it is determined that the vehicle has no risk of collision and no abnormal driving behavior at its current location, the current position of the vehicle is determined to be a safe driving position; if the vehicle is determined If there is a risk of collision or abnormal driving behavior at its current position, the current position of the vehicle is determined to be a dangerous driving position.

Through the self-vehicle's drivable location point, when perceiving short-term or long-term abnormality in the drivable area, the driveable area can be inferred based on the self-vehicle's drivable location point, and then travel route planning based on the drivable area is avoided. The failure of the automatic driving system has improved the application range of the automatic driving system and the reliability of the system.

In a feasible embodiment, judging whether the vehicle has a collision risk at its current position includes:

Obtain the angle θ between the driving direction of the own vehicle and the vehicle E; the vehicle E is the surrounding vehicle of the own vehicle; if the included angle θ is greater than the second preset angle, the intersection mode risk judgment method is used to determine whether the own vehicle has a collision at its current position Risk: If the included angle θ is not greater than the second preset angle, the rear-end collision mode risk judgment method is used to determine whether the vehicle has a collision risk at its current position.

In a feasible embodiment, the intersection mode risk discrimination method is used to determine whether the vehicle has a collision risk at the current position, including:

Obtain the relative speed and relative position coordinates of the vehicle E in the vehicle coordinate system of the vehicle E, and the absolute speed of the vehicle in the traveling direction; obtain the first time and the second time according to the relative speed, relative position coordinates and absolute speed. The first time is the time required for the vehicle to travel from its current position to the potential collision point, and the second time is the time required for the vehicle E to travel from its current position to the potential collision point; if the first time and the second time satisfy formula 1 and Formula 2, it is determined that the vehicle has a risk of collision at its current position; if the first time and the second time do not meet formula 1 or formula 2, it is determined that the vehicle has no risk of collision at its current position; where formula 1 is: | TTX ₁ -TTX ₂ |<α, formula 2 is:

TTX ₁ is the first time, TTX ₂ is the second time, α is the preset threshold, and R ₀ is the risk threshold.

In a feasible embodiment, adopting a rear-end collision mode risk discrimination method to determine whether the vehicle has a collision risk at the current position includes:

Acquire the lateral relative speed ΔV _Ex and relative position coordinates (x _Ev , y _Ev ) of the vehicle E in the vehicle coordinate system of the vehicle E and the absolute speed V _{s of the} vehicle in the traveling direction;

Obtain the third time TTC and the fourth time TTW according to the lateral relative velocity ΔV _Ex , the relative position coordinates (x _Ev , y _Ev ) and the absolute velocity V _s , where,

If the relative position ordinate satisfies formula 3, and the third time and fourth time satisfy formula 4, it is determined that the vehicle has a risk of collision at its current position; if the relative position ordinate does not satisfy formula 3, or the third time and fourth time When the time meets formula 4, it is determined that there is no risk of collision of the vehicle at its current position;

Among them, the formula 3 is: |y _Ev |<ψ, and the formula 4 is:

a and b are constants, R ₀ is the risk threshold, ψ is the horizontal distance threshold, and |y _Ev | is the horizontal distance between the own vehicle and the vehicle E.

In a feasible embodiment, determining whether the own vehicle has abnormal driving behavior at its current position includes:

Determine whether the own vehicle has an emergency braking behavior or an emergency steering behavior at its current position; if the own vehicle has an emergency braking behavior or an emergency steering behavior at its current position, the current position of the own vehicle is determined to be a dangerous driving position; If the vehicle has no emergency braking behavior and no emergency steering behavior at its current position, the current position of the vehicle is determined to be a safe driving position.

In a feasible embodiment, determining whether the own vehicle has an emergency braking behavior at its current position includes:

Obtain the longitudinal acceleration of the vehicle at its current position; if the longitudinal acceleration is less than the preset acceleration, determine that the vehicle has emergency braking at its current position; if the longitudinal acceleration is not less than the preset acceleration, determine that the vehicle is at its current position There is no emergency braking behavior.

In a feasible embodiment, determining whether the own vehicle has an emergency steering behavior at its current position includes:

Obtain the steering wheel rate of the own vehicle at its current position; if the steering wheel rate of rotation is greater than the preset rate, determine that the own vehicle has an emergency steering behavior at its current position; if the steering wheel rate of rotation is not greater than the preset acceleration, determine the own vehicle There is no emergency steering behavior at its current position.

In a feasible embodiment, after obtaining the driving position point of the self-vehicle, the method further includes:

If the vehicle is driving along road direction 1 at its current position, determine the travelable position point of the vehicle as the drivable position point on road direction 1, and save the drivable position point on road direction 1 to the side of road direction 1. In the roadside unit, where the drivable position point in the road direction 1 includes a safe driving position in the road direction 1 and a driving risk position in the road direction 1;

If the vehicle is driving along road direction 2 at its current position, determine the travelable position point of the vehicle as the drivable position point on road direction 2, and save the drivable position point on road direction 2 to the side of road direction 2. In the roadside unit in the roadside unit, where the drivable position point on the road direction 2 includes the driving safety position on the road direction 1 and the driving risk position on the road direction 2; wherein, the road direction 1 and the road direction 2 are opposite on the same road Direction.

By saving the driveable location point of the self-vehicle in the roadside unit, other vehicles can directly obtain the driveable location point from the roadside unit when driving on the road section, making it possible to sense short-term or long-term abnormalities in the drivable area When the autopilot system of other vehicles can infer the drivable area based on the drivable location points obtained from the roadside unit, and then plan the driving route based on the drivable area, avoid the failure of the automatic driving system and improve the performance of the automatic driving system. Application scope and system reliability.

In a feasible embodiment, the method further includes:

Get the driving position point information of surrounding vehicles.

Through the drivable location points of surrounding vehicles, when perceiving short-term or long-term abnormalities in the drivable area, the drivable area can be inferred based on the drivable location points of the surrounding vehicles, and then the travel route planning based on the drivable area is avoided. The failure of the automatic driving system has improved the application range of the automatic driving system and the reliability of the system.

In a feasible embodiment, the drivable position point information of surrounding vehicles includes the coordinates of the same direction drivable position point and the reverse direction drivable position point coordinates, and obtaining the drivable position point information of the surrounding vehicles includes:

Obtain the driving information of any vehicle A in the surrounding vehicles and the driving information of its own vehicle. Among them, the driving information of vehicle A includes relative position coordinates and longitudinal relative speed, and the driving information of own vehicle includes absolute position coordinates and absolute speed in the direction of travel. And the heading angle;

Obtain the position coordinates of vehicle A according to the absolute position coordinates of the vehicle, the heading angle of the vehicle, and the relative position coordinates of the vehicle A;

Determine the type of the drivable position point of vehicle A according to the longitudinal relative speed of vehicle A and the absolute speed of the vehicle; the type of drivable position point coordinate of vehicle A includes the coordinates of the reverse drivable position point or the same direction drivable position point coordinate;

Among them, the relative position coordinates are the coordinates in the vehicle coordinate system, and the vehicle A's travelable position point coordinates are the coordinates in the world coordinate system.

In a feasible embodiment, obtaining the drivable position point coordinates of vehicle A according to the absolute position coordinates of the own vehicle, the heading angle of the vehicle, and the relative position coordinates of vehicle A includes:

Calculate the absolute position coordinates of the own vehicle, the heading angle of the vehicle, and the relative position coordinates of the vehicle A through the fourth conversion formula to obtain the absolute position point coordinates of the vehicle A;

Among them, the fourth conversion formula is:

(x _Av , y _Av ) are the relative position coordinates of vehicle A, (x _Aw , y _Aw ) are the coordinates of the position where vehicle A can travel;

Third conversion matrix

(x ₀ ,y ₀ ) is the absolute position coordinate of the own vehicle at the current moment, and θ ₀ is the heading angle of the own vehicle at the current moment.

In a feasible embodiment, determining the type of the vehicle A's travelable position point coordinates according to the longitudinal relative speed and absolute speed of the vehicle A includes:

Obtain the longitudinal absolute speed of vehicle A according to the longitudinal relative speed and absolute speed of vehicle A;

If the longitudinal absolute speed of vehicle A is greater than the preset speed threshold, the coordinates of the vehicle A can be driven position point are determined to be the same direction; if the longitudinal absolute speed of vehicle A is less than the preset speed threshold, the vehicle A's The coordinates of the driving position point are the coordinates of the driving position point in the reverse direction.

In a feasible embodiment, the method further includes:

If the vehicle A is traveling along the road direction 1, the coordinates of the vehicle A's travelable location point are determined as the coordinates on the road direction 1, and the vehicle A's travelable location point coordinates are saved to the roadside unit on the road direction 1 side;

If vehicle A is traveling along road direction 2, determine that the driveable location point of lane A is the coordinate on road direction 2, and save the driveable location point coordinate of vehicle A to the roadside unit on the road direction 2 side; , Road direction 1 and road direction 2 are two opposite directions on the same road. By saving the drivable position points of surrounding vehicles in the roadside unit, when other vehicles are driving to the road section, there is no vehicle around the vehicle or they can directly obtain the drivable position points of the surrounding vehicles from the roadside unit. When sensing short-term or long-term abnormalities in the drivable area, the automatic driving system of other vehicles can infer the drivable area based on the drivable location points of the surrounding vehicles, and then plan the driving route based on the drivable area, avoiding the automatic driving system Failure to improve the application scope of the automatic driving system and the reliability of the system.

In a feasible embodiment, obtaining information about the drivable location points of surrounding vehicles includes:

Obtain the driving position point information of surrounding vehicles from the roadside unit on the side of the current road driving direction of the vehicle or from the cloud information platform.

In a feasible embodiment, the method further includes:

If the first area covers the ROI, the first area is determined to be a drivable area on the road.

In a feasible embodiment, the method further includes:

If the first area and the third area cover the ROI, then the first area and the third area are determined as road-driving areas.

In the second aspect, an embodiment of the present invention provides a road drivable area reasoning device, including:

The acquisition module is used to acquire the perceived drivable area;

The verification module is used to verify the perceivable travelable area to obtain the first area and the second area, where the first area is the travelable area with reliable verification and the second area is the travelable area with unreliable verification area;

The inference module is used to infer the second area based on the perceptual memory information of the drivable area if the first area does not cover the area of interest ROI to obtain the third area and the fourth area. The third area is the perceptual memory area and the first area. The area where the two areas overlap, the fourth area is the area that is not covered by the perceptual memory area in the second area; if the first area and the third area do not cover the ROI, the fourth area is inferred based on the driving position point , To get the fifth area; the fifth area is the drivable area in the fourth area;

The determining module is used to determine the first area, the third area, and the fifth area as road-drivable areas.

In a feasible embodiment, the verification module is specifically used for:

In a feasible embodiment, condition 1 to condition 4 are:

Condition 1: The width of sub-region I satisfies the following conditions:

k _min W≤w _i ≤k _max W

In a feasible embodiment, the perceptual memory information includes perceptual memory grid maps at multiple historical moments and the drivability value of each grid in each perceptual memory grid map. The perceptual memory information is used to compare the second area Perform reasoning to get the aspects of the third area and the fourth area. The reasoning module is specifically used to:

In a feasible embodiment, in the aspect of converting the perceptual memory grid maps of multiple historical moments from the vehicle coordinate system of the vehicle at the historical moment to the world coordinate system to obtain multiple world grid maps, The reasoning module is specifically used for:

Among them, the first conversion formula is:

Is the first conversion matrix,

First conversion matrix

In a feasible embodiment, in terms of converting the inference area from the world coordinate system to the vehicle coordinate system of the vehicle at the current moment to obtain the first inference grid map, the inference module is specifically used for:

Among them, the second conversion formula is:

Is the second conversion matrix;

Second conversion matrix

In a feasible embodiment, in terms of calculating the drivability value of each grid in the first inference grid map according to the drivability value of each grid in the perceptual memory grid map, the inference module is specifically used for :

the weight of.

In a feasible embodiment, in terms of inferring the fourth area based on the drivable location point to obtain the fifth area, the inference module is specifically configured to:

In a feasible embodiment, in terms of transforming the coordinates of the location point to be inferred from the world coordinate system to the vehicle coordinate system of the vehicle to obtain the driving area to be inferred, the inference module is specifically used for:

Among them, the third conversion formula is:

Is the second conversion matrix,

Second conversion matrix

In a feasible embodiment, in terms of calculating the drivability value of each grid according to the drivable position point information in each grid in the second inference grid map, the inference module is specifically used to:

Is the drivability value at time t, k _t is

the weight of,

In a feasible embodiment, the drivable position point includes the drivable position point of the self-vehicle, and the acquisition module is further used for:

Before inferring the fourth area based on the driving position points, obtain the driving position points of the own vehicle. The driving position points of the own vehicle include the driving safety position points and the driving risk position points; wherein, obtaining the driving position points of the own vehicle includes : Determine whether the driving mode of the own vehicle at its current position is manual driving mode; if the driving mode of the own vehicle at its current position is manual driving mode, determine the current position of the own vehicle as a safe driving position; If the driving mode at its current location is automatic driving mode, it is determined whether the vehicle has a risk of collision or abnormal driving behavior at its current location; if it is determined that the vehicle has no risk of collision and no abnormal driving behavior at its current location, the vehicle is determined The current position point of is a safe driving position; if it is determined that the vehicle has a risk of collision or abnormal driving behavior at its current position, the current position of the own vehicle is determined to be a dangerous driving position.

In a feasible embodiment, in terms of judging whether the vehicle has a collision risk at its current position, the acquiring module is specifically used to:

In a feasible embodiment, in the aspect of using the intersection mode risk discrimination method to determine whether the vehicle has a collision risk at the current position, the acquisition module is specifically used to:

In a feasible embodiment, in terms of using the rear-end collision mode risk discrimination method to determine whether the vehicle has a collision risk at the current position, the acquisition module is specifically used to:

Among them, the formula 3 is: |y _Ev |<ψ, and the formula 4 is:

In a feasible embodiment, in terms of judging whether the own vehicle has abnormal driving behavior at its current position, the acquiring module is specifically used for:

In a feasible embodiment, in terms of judging whether the own vehicle has an emergency braking behavior at its current position, the acquiring module is specifically used to:

In a feasible embodiment, in determining whether the own vehicle has an emergency steering behavior at its current position, the acquiring module is specifically used to:

In a feasible embodiment, the road drivable area reasoning device further includes a storage module;

The determining module is also used to determine that the self-vehicle can travel along the road direction 1 after the acquisition module obtains the driveable location point of the vehicle at its current location as the driveable location point on the road direction 1. , Save module, used to save the drivable position point on road direction 1 to the roadside unit on the side of road direction 1, where the drivable position point on road direction 1 includes the safe driving position and road in road direction 1. The driving risk position in direction 1;

The determination module is also used for determining that the vehicle can travel along the road direction 2 at its current position as the travelable location point on the road direction 2. The drivable position points are saved to the roadside unit on the road direction 2 side, where the drivable position points on the road direction 2 include the safe driving position on the road direction 1 and the driving risk position on the road direction 2; where, the road direction 1 and road direction 2 are opposite directions on the same road.

In a feasible embodiment, the acquisition module is also used to:

Get the driving position point information of surrounding vehicles.

In a feasible embodiment, the driving position point information of surrounding vehicles includes the coordinates of the driving position point in the same direction and the coordinates of the driving position point in the reverse direction. In terms of obtaining the driving position point information of the surrounding vehicles, the acquisition module is also used for :

Obtain the driving information of any vehicle A in the surrounding vehicles and the driving information of its own vehicle. Among them, the driving information of vehicle A includes relative position coordinates and longitudinal relative speed, and the driving information of own vehicle includes absolute position coordinates and absolute speed in the direction of travel. And the heading angle of the vehicle; according to the absolute position coordinates of the vehicle, the heading angle of the vehicle, and the relative position coordinates of the vehicle A, the coordinates of the vehicle A's driving position are obtained; according to the longitudinal relative speed of the vehicle A and the absolute speed of the vehicle, the vehicle A can be determined The type of the driving position point; the type of the driving position point coordinate of the vehicle A includes the reverse driving position point coordinate or the same direction driving position point coordinate; wherein, the relative position coordinate is the coordinate in the vehicle coordinate system, and the vehicle A The coordinates of the driving position point are the coordinates in the world coordinate system.

In a feasible embodiment, in terms of acquiring the drivable position point coordinates of vehicle A according to the absolute position coordinates of the vehicle, the heading angle of the vehicle, and the relative position coordinates of vehicle A, the acquiring module is also used for:

Among them, the fourth conversion formula is:

Third conversion matrix

In a feasible embodiment, in terms of determining the type of the vehicle A's drivable position point coordinates according to the longitudinal relative speed and absolute speed of the vehicle A, the acquiring module is further used to:

In a feasible embodiment, the determining module is also used to determine if the vehicle A is driving along the road direction 1, then determining that the vehicle A’s travelable position point coordinates are the coordinates on the road direction 1, the saving module is also used to save the vehicle The coordinates of the driving position point of A are saved to the roadside unit on the side of the road direction 1;

The determination module is also used to determine if the vehicle A is traveling along the road direction 2, the driveable position point of the lane A is determined as the coordinate on the road direction 2, and the storage module is used to save the vehicle A's driveable position point coordinates to the road In the roadside unit on the direction 2 side; wherein the road direction 1 and the road direction 2 are two opposite directions on the same road.

In a feasible embodiment, in terms of obtaining information about the drivable location points of surrounding vehicles, the obtaining module is specifically used to:

In a feasible embodiment, the determining module is also used to:

If the first area and the third area cover the ROI, the first area and the third area are determined as the road drivable area.

In a third aspect, an embodiment of the present invention provides a road drivable area reasoning device, including:

A memory for storing executable program codes;

A processor coupled to the memory; when the processor invokes the executable program code stored in the memory, it executes part or all of the method described in the first aspect.

In a fourth aspect, an embodiment of the present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed by a computing platform or a processor with processing capability, the method described in the first aspect Part or all of the steps of the method.

These and other aspects of the present invention will be more concise and understandable in the description of the following embodiments.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

Figure 1a is a schematic diagram of a vehicle coordinate system provided by an embodiment of the present invention;

Figure 1b is a schematic structural diagram of an autonomous vehicle provided by an embodiment of the present invention;

2 is a schematic structural diagram of a computer system provided by an embodiment of the present invention;

3 is a schematic structural diagram of a neural network processor provided by an embodiment of the present invention;

4 is a schematic diagram of the application of a cloud-side commanded autonomous vehicle according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of an application of a cloud-side commanded autonomous vehicle according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of an application scenario of a method for reasoning on a road drivable area provided by an embodiment of the present invention;

FIG. 7 is a schematic flowchart of a method for reasoning on a road drivable area according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of the positional relationship between area I and area II according to an embodiment of the present invention;

9 is a schematic diagram of the relationship between the historical moment perception memory grid map and the inference grid map provided by an embodiment of the present invention;

FIG. 10 is a schematic diagram of a method for absolute position and direction of a vehicle according to an embodiment of the present invention;

FIG. 11 is a schematic diagram of risk identification of intersection modes provided by an embodiment of the present invention;

FIG. 12 is a schematic diagram of a drivable area in an area of interest provided by an embodiment of the present invention;

FIG. 13 is a schematic structural diagram of a road drivable area reasoning device provided by an embodiment of the present invention;

FIG. 14 is a schematic structural diagram of a road drivable area reasoning device provided by an embodiment of the present invention;

Fig. 15 is a schematic structural diagram of a computer program product provided by an embodiment of the present invention.

Detailed ways

Detailed descriptions are given below with reference to the drawings.

The terminology involved in the present invention will be described first.

Structured roads: roads with a single pavement structure, clear edge lines and obvious road geometric features, such as highways and urban arterial roads.

Unstructured roads: roads with complex pavement structures, no lane lines and clear road boundaries, and road geometric features that are not obvious, such as residential district roads, rural roads, and urban non-main roads.

Drivenability grid map: refers to dividing the environment map into a series of grids, where each grid is given a driveability value to indicate whether the grid can be driven.

Vehicle coordinate system: When the vehicle is at a standstill on a horizontal road, the x-axis is parallel to the ground and points forward, the z-axis passes vertically upward through the center of the rear axle, the y-axis points to the left side of the driver's seat, and the center of the rear axle is the origin of the coordinate system O, as shown in Figure 1a.

World coordinate system: refers to a coordinate system fixed relative to the ground. There are many ways to define the world coordinate system. For example, you can define the origin at the initial position of the vehicle, and the x-axis along the positive direction of the target. When the vehicle moves, the origin position and the x-axis direction are fixed on the ground and do not move with the vehicle, or The origin is defined at a certain position on the earth, and the x axis is north.

Fig. 1b is a functional block diagram of a vehicle 100 provided by an embodiment of the present invention. In one embodiment, the vehicle 100 is configured in a fully or partially autonomous driving mode. For example, the vehicle 100 can control itself while in the automatic driving mode, and can determine the current state of the vehicle and its surrounding environment through human operations, determine the possible behavior of at least one other vehicle in the surrounding environment, and determine the other vehicle The confidence level corresponding to the possibility of performing possible actions is controlled based on the determined information. When the vehicle 100 is in the automatic driving mode, the vehicle 100 can be placed to operate without human interaction.

The vehicle 100 may include various subsystems, such as a travel system 102, a sensor system 104, a control system 106, one or more peripheral devices 108 and a power supply 110, a computer system 112, and a user interface 116. Alternatively, the vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple elements. In addition, each of the subsystems and elements of the vehicle 100 may be wired or wirelessly interconnected.

The travel system 102 may include components that provide power movement for the vehicle 100. In one embodiment, the propulsion system 102 may include an engine 118, an energy source 119, a transmission 120, and wheels/tires 121. The engine 118 may be an internal combustion engine, an electric motor, an air compression engine, or other types of engine combinations, such as a hybrid engine composed of a gasoline engine and an electric motor, or a hybrid engine composed of an internal combustion engine and an air compression engine. The engine 118 converts the energy source 119 into mechanical energy.

Examples of energy sources 119 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. The energy source 119 may also provide energy for other systems of the vehicle 100.

The transmission device 120 can transmit mechanical power from the engine 118 to the wheels 121. The transmission device 120 may include a gearbox, a differential, and a drive shaft. In an embodiment, the transmission device 120 may also include other devices, such as a clutch. Among them, the drive shaft may include one or more shafts that can be coupled to one or more wheels 121.

The sensor system 104 may include several sensors that sense information about the environment around the vehicle 100. For example, the sensor system 104 may include a positioning system 122 (the positioning system may be a GPS system, a Beidou system or other positioning systems), an inertial measurement unit (IMU) 124, a radar 126, a laser rangefinder 128, and Camera 130. The sensor system 104 may also include sensors of the internal system of the monitored vehicle 100 (for example, an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, direction, speed, etc.). Such detection and identification are key functions for the safe operation of the autonomous vehicle 100.

The positioning system 122 can be used to estimate the geographic location of the vehicle 100. The IMU 124 is used to sense changes in the position and orientation of the vehicle 100 based on inertial acceleration. In an embodiment, the IMU 124 may be a combination of an accelerometer and a gyroscope.

The radar 126 may use radio signals to sense objects in the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing the object, the radar 126 may also be used to sense the speed and/or direction of the object.

The laser rangefinder 128 can use laser light to sense objects in the environment where the vehicle 100 is located. In some embodiments, the laser rangefinder 128 may include one or more laser sources, laser scanners, and one or more detectors, as well as other system components.

The camera 130 may be used to capture multiple images of the surrounding environment of the vehicle 100. The camera 130 may be a still camera or a video camera.

The control system 106 controls the operation of the vehicle 100 and its components. The control system 106 may include various components, including a steering system 132, a throttle 134, a braking unit 136, a sensor fusion algorithm 138, a computer vision system 140, a route control system 142, and an obstacle avoidance system 144.

The steering system 132 is operable to adjust the forward direction of the vehicle 100. For example, it may be a steering wheel system in one embodiment.

The throttle 134 is used to control the operating speed of the engine 118 and thereby control the speed of the vehicle 100.

The braking unit 136 is used to control the vehicle 100 to decelerate. The braking unit 136 may use friction to slow down the wheels 121. In other embodiments, the braking unit 136 may convert the kinetic energy of the wheels 121 into electric current. The braking unit 136 may also take other forms to slow down the rotation speed of the wheels 121 to control the speed of the vehicle 100.

The computer vision system 140 may be operable to process and analyze the images captured by the camera 130 in order to identify objects and/or features in the surrounding environment of the vehicle 100. The objects and/or features may include traffic signals, road boundaries and obstacles. The computer vision system 140 may use object recognition algorithms, Structure from Motion (SFM) algorithms, video tracking, and other computer vision technologies. In some embodiments, the computer vision system 140 may be used to map the environment, track objects, estimate the speed of objects, and so on.

The route control system 142 is used to determine the travel route of the vehicle 100. In some embodiments, the route control system 142 may combine data from the sensor 138, the GPS 122, and one or more predetermined maps to determine the driving route for the vehicle 100.

The obstacle avoidance system 144 is used to identify, evaluate, and avoid or otherwise cross over potential obstacles in the environment of the vehicle 100.

Of course, in one example, the control system 106 may add or alternatively include components other than those shown and described. Alternatively, a part of the components shown above may be reduced.

The vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users through peripheral devices 108. The peripheral device 108 may include a wireless communication system 146, an onboard computer 148, a microphone 150 and/or a speaker 152.

In some embodiments, the peripheral device 108 provides a means for the user of the vehicle 100 to interact with the user interface 116. For example, the onboard computer 148 may provide information to the user of the vehicle 100. The user interface 116 can also operate the onboard computer 148 to receive user input. The on-board computer 148 can be operated through a touch screen. In other cases, the peripheral device 108 may provide a means for the vehicle 100 to communicate with other devices located in the vehicle. For example, the microphone 150 may receive audio (eg, voice commands or other audio input) from a user of the vehicle 100. Similarly, the speaker 152 may output audio to the user of the vehicle 100.

The wireless communication system 146 may wirelessly communicate with one or more devices directly or via a communication network. For example, the wireless communication system 146 may use 3G cellular communication, such as CDMA, EVDO, GSM/GPRS, or 4G cellular communication, such as LTE. Or 5G cellular communication. The wireless communication system 146 may use WiFi to communicate with a wireless local area network (WLAN). In some embodiments, the wireless communication system 146 may directly communicate with the device using an infrared link, Bluetooth, or ZigBee. Other wireless protocols, such as various vehicle communication systems. For example, the wireless communication system 146 may include one or more dedicated short-range communications (DSRC) devices, which may include vehicles and/or roadside stations. Public and/or private data communications.

The power supply 110 may provide power to various components of the vehicle 100. In one embodiment, the power source 110 may be a rechargeable lithium ion or lead-acid battery. One or more battery packs of such batteries may be configured as a power source to provide power to various components of the vehicle 100. In some embodiments, the power source 110 and the energy source 119 may be implemented together, such as in some all-electric vehicles.

Part or all of the functions of the vehicle 100 are controlled by the computer system 112. The computer system 112 may include at least one processor 113 that executes instructions 115 stored in a non-transitory computer readable medium such as a data storage device 114. The computer system 112 may also be multiple computing devices that control individual components or subsystems of the vehicle 100 in a distributed manner.

The processor 113 may be any conventional processor, such as a commercially available CPU. Alternatively, the processor may be a dedicated device such as an ASIC or other hardware-based processor. Although FIG. 1b functionally illustrates the processor, memory, and other elements of the computer 110 in the same block, those of ordinary skill in the art should understand that the processor, computer, or memory may actually include Multiple processors, computers, or memories stored in the same physical enclosure. For example, the memory may be a hard disk drive or other storage medium located in a housing other than the computer 110. Therefore, a reference to a processor or computer will be understood to include a reference to a collection of processors or computers or memories that may or may not operate in parallel. Rather than using a single processor to perform the steps described here, some components such as steering components and deceleration components may each have its own processor that only performs calculations related to component-specific functions .

In the various aspects described herein, the processor may be located away from the vehicle and wirelessly communicate with the vehicle. In other aspects, some of the processes described herein are executed on a processor disposed in the vehicle and others are executed by a remote processor, including taking the necessary steps to perform a single manipulation.

In some embodiments, the data storage device 114 may include instructions 115 (eg, program logic), which may be executed by the processor 113 to perform various functions of the vehicle 100, including those functions described above. The data storage device 114 may also contain additional instructions, including sending data to, receiving data from, interacting with, and/or performing data on one or more of the propulsion system 102, the sensor system 104, the control system 106, and the peripheral device 108. Control instructions.

In addition to the instructions 115, the data storage device 114 may also store data, such as road maps, route information, the location, direction, and speed of the vehicle, and other such vehicle data, as well as other information. Such information may be used by the vehicle 100 and the computer system 112 during the operation of the vehicle 100 in autonomous, semi-autonomous, and/or manual modes.

The processor 113 obtains the road drivable area perception information through the sensor system 104, and obtains the perceptible drivable area according to the road drivable area perception information; checks the perceptual drivable area to obtain the first area and the second area, Wherein, the first area is a drivable area with reliable verification, and the second area is a drivable area with unreliable verification; if the first area covers the ROI, the processor 113 determines the first area as a road drivable area; If the first area does not cover the ROI, the processor 113 obtains the perceptual memory information from the data storage device 114, and performs inference operations on the second area according to the perceptual memory information to obtain the third area and the fourth area. Is the area where the sensing memory area overlaps with the second area, and the fourth area is the area not covered by the sensing memory area in the second area; if the first area and the third area cover the ROI, the processor 113 compares the first area with the third area The area is determined to be a drivable area on the road; if the first area and the third area do not cover the ROI, the processor 113 obtains the drivable position point information from the data storage device 114 or other units or servers, and compares the drivable position point information according to the drivable position point information. The fourth area performs inference to obtain the fifth area, which is the drivable area in the fourth area; the processor 113 determines the first area, the third area, and the fifth area as road drivable areas; 113 makes driving route planning decisions based on the road's drivable area to obtain a planned driving route; the processor 113 sends the planned driving route to the control system 106, and each functional module of the control system 106 controls the vehicle 100 to travel according to the planned driving route.

The user interface 116 is used to provide information to or receive information from a user of the vehicle 100. Optionally, the user interface 116 may include one or more input/output devices in the set of peripheral devices 108, such as a wireless communication system 146, a car computer 148, a microphone 150, and a speaker 152.

The computer system 112 may control the functions of the vehicle 100 based on inputs received from various subsystems (eg, travel system 102, sensor system 104, and control system 106) and from the user interface 116. For example, the computer system 112 may utilize input from the control system 106 in order to control the steering unit 132 to avoid obstacles detected by the sensor system 104 and the obstacle avoidance system 144. In some embodiments, the computer system 112 is operable to provide control of many aspects of the vehicle 100 and its subsystems.

Optionally, one or more of these components described above may be installed or associated with the vehicle 100 separately. For example, the data storage device 114 may exist partially or completely separately from the vehicle 100. The aforementioned components may be communicatively coupled together in a wired and/or wireless manner.

Optionally, the above-mentioned components are only an example. In actual applications, components in the above-mentioned modules may be added or deleted according to actual needs. FIG. 1b should not be understood as a limitation to the embodiment of the present invention.

An autonomous vehicle traveling on a road, such as the vehicle 100 above, can recognize objects in its surrounding environment to determine the adjustment to the current speed. The object may be other vehicles, traffic control equipment, or other types of objects. In some examples, each recognized object can be considered independently, and based on the respective characteristics of the object, such as its current speed, acceleration, distance from the vehicle, etc., can be used to determine the speed to be adjusted by the autonomous vehicle.

Optionally, the self-driving car vehicle 100 or the computing device associated with the self-driving vehicle 100 (such as the computer system 112, the computer vision system 140, and the data storage device 114 in FIG. 1b) may be based on the characteristics of the identified object and the surrounding environment The state of the object (e.g., traffic, rain, ice on the road, etc.) to predict the behavior of the identified object. Optionally, each recognized object depends on each other's behavior, so all recognized objects can also be considered together to predict the behavior of a single recognized object. The vehicle 100 can adjust its speed based on the predicted behavior of the identified object. In other words, an autonomous vehicle can determine what stable state the vehicle will need to adjust to (for example, accelerate, decelerate, or stop) based on the predicted behavior of the object. In this process, other factors may also be considered to determine the speed of the vehicle 100, such as the lateral position of the vehicle 100 on the road on which it is traveling, the curvature of the road, the proximity of static and dynamic objects, and so on.

In addition to providing instructions to adjust the speed of the self-driving car, the computing device can also provide instructions to modify the steering angle of the vehicle 100, so that the self-driving car follows a given trajectory and/or maintains an object near the self-driving car (for example, , The safe horizontal and vertical distances of cars in adjacent lanes on the road.

The above-mentioned vehicle 100 can be a car, truck, motorcycle, bus, boat, airplane, helicopter, lawn mower, recreational vehicle, playground vehicle, construction equipment, tram, golf cart, train, and trolley, etc. The embodiments of the invention are not particularly limited.

Scenario example 2: Autonomous driving system

According to FIG. 2, the computer system 101 includes a processor 103, and the processor 103 is coupled to a system bus 105. The processor 103 may be one or more processors, where each processor may include one or more processor cores. A display adapter (video adapter) 107 can drive the display 109, and the display 109 is coupled to the system bus 105. The system bus 105 is coupled with an input output (I/O) bus 113 through a bus bridge 111. The I/O interface 115 is coupled to the I/O bus. The I/O interface 115 communicates with a variety of I/O devices, such as an input device 117 (such as a keyboard, a mouse, a touch screen, etc.), a media tray 121 (such as a CD-ROM, a multimedia interface, etc.). Transceiver 123 (can send and/or receive radio communication signals), camera 155 (can capture scene and dynamic digital video images) and external USB interface 125. Wherein, optionally, the interface connected to the I/O interface 115 may be a USB interface.

The processor 103 may be any conventional processor, including a reduced instruction set computing ("RISC") processor, a complex instruction set computing ("CISC") processor, or a combination of the foregoing. Alternatively, the processor may be a dedicated device such as an application specific integrated circuit ("ASIC"). Optionally, the processor 103 may be a neural network processor or a combination of a neural network processor and the foregoing traditional processors.

Optionally, in various embodiments described herein, the computer system 101 may be located far away from the autonomous driving vehicle, and may communicate with the autonomous driving vehicle O wirelessly. In other respects, some of the processes described herein are executed on a processor provided in an autonomous vehicle, and others are executed by a remote processor, including taking actions required to perform a single manipulation.

The computer 101 can communicate with the software deployment server 149 through the network interface 129. The network interface 129 is a hardware network interface, such as a network card. The network 127 may be an external network, such as the Internet, or an internal network, such as an Ethernet or a virtual private network (VPN). Optionally, the network 127 may also be a wireless network, such as a WiFi network, a cellular network, and so on.

The hard disk drive interface is coupled to the system bus 105. The hardware drive interface is connected with the hard drive. The system memory 135 is coupled to the system bus 105. The data running in the system memory 135 may include the operating system 137 and application programs 143 of the computer 101.

The operating system includes Shell 139 and kernel 141. Shell 139 is an interface between the user and the kernel of the operating system. The shell is the outermost layer of the operating system. The shell manages the interaction between the user and the operating system: waiting for the user's input, explaining the user's input to the operating system, and processing the output of various operating systems.

The kernel 141 is composed of those parts of the operating system for managing memory, files, peripherals, and system resources. Directly interact with hardware, the operating system kernel usually runs processes and provides inter-process communication, providing CPU time slice management, interrupts, memory management, IO management, and so on.

Application programs 143 include programs related to controlling auto-driving cars, such as programs that manage the interaction between autonomous vehicles and road obstacles, programs that control the route or speed of autonomous vehicles, and programs that control interaction between autonomous vehicles and other autonomous vehicles on the road. . The application program 143 also exists on the system of the software deployment server 149. In one embodiment, when the application program 147 needs to be executed, the computer system 101 may download the application program 143 from the software deployment server 149.

The sensor 153 and the camera 155 obtain the road travelable area perception information, and save the road travelable area perception information to the hard drive 131 through the I/O interface 115, the bus bridge 111, the system bus 105, and the hard drive interface 121. The processor 103 obtains the road drivable area perception information from the hard disk drive 131 through the system bus 105 and the hard disk drive interface 121, and executes the automatic driving related program 147 in the application 143 for the road drivable area perception information, and executes the automatic driving related program 147 When the time, the processor 103 specifically executes the following steps; obtain the perceived drivable area according to the road drivable area perception information, and verify the perceived drivable area to obtain the first area and the second area, where the first area is If the first area covers the ROI, the processor 103 will determine the first area as the road-drivable area; if the first area is not covered by the ROI In ROI, the processor 103 obtains the perceptual memory information from the hard disk drive 133, and performs inference operations on the second area according to the perceptual memory information to obtain the third area and the fourth area. The third area is the perceptual memory area and the second area. The area where the area overlaps, the fourth area is the area not covered by the perceptual memory area in the second area; if the first area and the third area cover the ROI, the processor 103 determines the first area and the third area as road-driving areas If the first area and the third area do not cover the ROI, the processor 103 obtains the drivable position point information from the hard disk drive 133 or other unit or server, and infers the fourth area based on the drivable position point information to obtain The fifth area, the fifth area is a drivable area in the fourth area; the processor 113 determines the first area, the third area, and the fifth area as road drivable areas; the processor 113 drives according to the road drivable area Route planning decisions are made to obtain a planned driving route; the processor 113 controls the vehicle to travel according to the planned driving route.

The sensor 153 is associated with the computer system 101. The sensor 153 is used to detect the environment around the computer 101. For example, the sensor 153 can detect animals, cars, obstacles, and crosswalks. Further, the sensor can also detect the environment around objects such as animals, cars, obstacles, and crosswalks, such as the environment around the animals, for example, when the animals appear around them. Other animals, weather conditions, the brightness of the surrounding environment, etc. Optionally, if the computer 101 is located on a self-driving car, the sensor may be a camera, an infrared sensor, a chemical detector, a microphone, etc.

For example, the absolute speed of the own vehicle and the relative speed of surrounding vehicles are obtained through the speed sensor, the relative position coordinates of the own vehicle are obtained through the position sensor, etc., and the angle of the head of the own vehicle in the driving direction is obtained through the angle sensor.

The concrete realization of the hardware:

Figure 3 is a chip hardware structure diagram provided by an embodiment of the present invention.

The neural network processor NPU 50 is mounted on the main CPU (Host CPU) as a coprocessor, and the Host CPU allocates tasks. The core part of the NPU is the arithmetic circuit 50. The controller 504 controls the arithmetic circuit 503 to extract data from the memory (weight memory or input memory) and perform calculations.

In some implementations, the arithmetic circuit 503 includes multiple processing units (Process Engine, PE). In some implementations, the arithmetic circuit 503 is a two-dimensional systolic array. The arithmetic circuit 503 may also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 503 is a general-purpose matrix processor.

For example, suppose there is an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit fetches the corresponding data of matrix B from the weight memory 502 and buffers it on each PE in the arithmetic circuit. The arithmetic circuit takes the data of matrix A and matrix B from the input memory 501 to perform matrix operations, and the partial results or final results of the obtained matrix are stored in the accumulator 508.

The vector calculation unit 507 can perform further processing on the output of the arithmetic circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison and so on. For example, the vector calculation unit 507 can be used for network calculations in the non-convolutional/non-FC layer of the neural network, such as pooling, batch normalization, local response normalization, etc. .

In some implementations, the vector calculation unit 507 can store the processed output vector in the unified buffer 506. For example, the vector calculation unit 507 may apply a nonlinear function to the output of the arithmetic circuit 503, such as a vector of accumulated values, to generate the activation value. In some implementations, the vector calculation unit 507 generates a normalized value, a combined value, or both. In some implementations, the processed output vector can be used as an activation input to the arithmetic circuit 503, for example for use in subsequent layers in a neural network.

The unified memory 506 is used to store input data and output data.

The memory unit access controller 505 (Direct Memory Access Controller, DMAC) transfers the input data in the external memory to the input memory 501 and/or the unified memory 506, stores the weight data in the external memory into the weight memory 502, and stores the unified memory The data in 506 is stored in the external memory.

A bus interface unit (BIU) 510 is used to implement interaction between the main CPU, the DMAC, and the fetch memory 509 through the bus.

An instruction fetch buffer 509 connected to the controller 504 is used to store instructions used by the controller 504;

The controller 504 is used to call the instruction cached in the instruction fetch memory 509 to control the working process of the operation accelerator.

Generally, the unified memory 506, the input memory 501, the weight memory 502, and the instruction fetch memory 509 are all on-chip (On-Chip) memories, and the external memory is a memory external to the NPU. The external memory can be a double data rate synchronous dynamic random access memory. Memory (Double Data Rate Synchronous Dynamic Random Access Memory, referred to as DDR SDRAM), High Bandwidth Memory (HBM) or other readable and writable memory.

The program algorithm in Figure 1b and Figure 2 can be completed by the main CPU and NPU together.

A description example of the embodiment on the cloud side:

Example 1:

The computer system 112 may also receive information from other computer systems or transfer information to other computer systems. Alternatively, the sensor data collected from the sensor system 104 of the vehicle 100 may be transferred to another computer to process the data. As shown in FIG. 4, data from the computer system 112 may be transmitted to the computer 720 on the cloud side via the network for further processing. The network and intermediate nodes can include various configurations and protocols, including the Internet, World Wide Web, Intranet, virtual private network, wide area network, local area network, private network using one or more company's proprietary communication protocols, Ethernet, WiFi and HTTP, And various combinations of the foregoing. This communication can be by any device capable of transferring data to and from other computers, such as modems and wireless interfaces.

In one example, the computer 720 may include a server with multiple computers, such as a load balancing server group, which exchanges information with different nodes of the network for the purpose of receiving, processing, and transmitting data from the computer system 112. The server can be configured similarly to the computer system 110 and has a processor 730, a memory 740, instructions 750, and data 760.

In an example, the data in the server 720 may include information such as the vehicle's drivable position point coordinates, the angle of the front of the vehicle, and the like sent by the communication device on the vehicle. The data in the server 720 may also include data such as historical grid maps.

Example 2:

FIG. 5 shows an example of an autonomous driving vehicle and a cloud service center according to an example embodiment. The cloud service center 520 may receive information (such as data collected by vehicle sensors or other information) from the

autonomous vehicles

510, 512, and 514 in its operating environment 500 via a network 502 such as a wireless communication network.

For example, the coordinates of the driving position of the own vehicle, the coordinates of the driving position of the surrounding vehicles, and the area information of the driving area perception.

According to the received data, the cloud service center 520 runs its stored programs related to controlling automatic driving of automobiles to control the

autonomous vehicles

510, 512, and 514. Programs related to controlling auto-driving can be programs that manage the interaction between autonomous vehicles and obstacles on the road, programs that control the route or speed of autonomous vehicles, and programs that control interaction between autonomous vehicles and other autonomous vehicles on the road.

The cloud service center 520 obtains the road-drivable area perception information, and obtains the perceived drivable area according to the road-drivable area perception information; checks the perceived drivable area to obtain the first area and the second area. The area is a drivable area with reliable verification, and the second area is a drivable area with unreliable verification; if the first area covers the ROI, the first area is determined as the road drivable area; if the first area does not cover the ROI When the time, the cloud service center 520 obtains the perceptual memory information, and performs inference operations on the second area according to the perceptual memory information to obtain the third area and the fourth area. The third area is the area where the perceptual memory area and the second area overlap. The fourth area is the area not covered by the perceptual memory area in the second area; if the first area and the third area cover the ROI, the cloud service center 520 determines the first area and the third area as road-driving areas; If the area and the third area do not cover the ROI, the cloud service center 520 obtains the drivable position point information, and infers the fourth area based on the drivable position point information to obtain the fifth area, which is the fourth area The cloud service center 520 determines the first area, the third area, and the fifth area as road drivable areas. The cloud service center 520 makes driving route planning decisions based on the driveable area of the road to obtain the planned driving route; the cloud service center 520 sends the planned driving route to the vehicle control system so that the functional modules of the control system control the vehicle according to the planned driving route Driving.

The network 502 externally provides part of the map to the

autonomous vehicle

510, 512, or 514. In other examples, operations can be divided between different locations or centers. For example, multiple cloud service centers 520 may receive, confirm, combine, and/or send information reports. In some examples, information reports and/or sensor data can also be sent between autonomous vehicles. Other configurations are also possible.

In some examples, the cloud service center 520 sends to the autonomous vehicle a suggested solution for possible driving situations in the environment (eg, inform the obstacle ahead and tell how to avoid it). For example, the cloud service center 520 may assist the vehicle in determining how to proceed when facing a specific obstacle in the environment. The cloud service center 520 sends a response to the autonomous vehicle indicating how the vehicle should travel in a given scene. For example, the cloud service center 520 can confirm the existence of a temporary stop sign in front of the road based on the collected sensor data, and also determine that the lane has been impaired due to the “lane closed” sign and the sensor data of construction vehicles on the lane. Closed. Correspondingly, the cloud service center 520 sends a suggested operation mode for the automatic driving vehicle to pass the obstacle (for example, instructing the vehicle to change lanes on another road). When the cloud service center 520 observes the video stream in its operating environment and has confirmed that the autonomous driving vehicle can safely and successfully pass through the obstacle, the operation steps used for the autonomous driving vehicle can be added to the driving information map. Correspondingly, this information can be sent to other vehicles in the area that may encounter the same obstacle, so as to assist other vehicles not only to recognize the closed lane but also how to pass.

Referring to FIG. 6, FIG. 6 is a schematic diagram of an application scenario of a method for reasoning on a road drivable area provided by an embodiment of the present invention. As shown in Figure 6, the application scenario includes: a vehicle-mounted device 601, a roadside unit 602, and a cloud information platform.

Among them, the data interaction between the cloud information platform 603 and the roadside unit (RSU) 602 is realized through remote communication, such as 4G, 5G, optical fiber communication, etc.; the vehicle device 601 and the cloud information platform 603 Data interaction is completed through remote communication, such as 4G, 5G, etc.; information interaction between the vehicle-mounted device 601 and the roadside unit 602 is achieved through short-range communication, such as DSRC technology, long term evolution for vehicles, LTE-V ) Technology etc.

The vehicle-mounted device 601 obtains road-drivable area perception information, and obtains the perceived drivable area according to the road-drivable area perception information; checks the perceived drivable area to obtain the first area and the second area, where the first area is Check the reliable drivable area, the second area is the unreliable drivable area; if the first area covers the ROI, the vehicle-mounted device 601 determines the first area as the road drivable area; if the first area is not covered In the case of ROI, the vehicle-mounted device 601 obtains the perceptual memory information of the drivable area from its memory, and uses the perceptual memory area to perform inference operations on the unreliable drivable area to obtain the third area and the fourth area. The third area Is the area where the sensing memory area overlaps with the second area, and the fourth area is the area not covered by the sensing memory area in the second area; if the first area and the third area cover the ROI, the vehicle-mounted device 601 combines the first area and the third area. The area is determined to be a drivable area on the road; if the first area and the third area do not cover the ROI, the vehicle-mounted device 601 obtains the drivable position point information from the RSU 602 or the cloud information platform 603, and compares the fourth area according to the drivable position point information. The area is inferred to obtain the fifth area, which is the drivable area in the fourth area, and determines the first area, the third area, and the fifth area as the road drivable area. Then, the driving route planning decision is made according to the driving area of the road to obtain the planned driving route.

RSU602 collects the drivable position point information generated by the vehicle on the road section it is on, uploads the generated drivable position point information to the cloud information platform 603, and sends the drivable position point information to the vehicle traveling on the current road section.

The cloud information platform 603 collects the drivable location point information generated by each vehicle and collected by RSU602, and integrates the information; combined with the current location of the vehicle, the user's current driving destination and navigation route, the user's common driving route, etc., update the drivable location of nearby roads Point information to the on-board device of the vehicle; update the driving position point information to the RSU602 of the corresponding road according to different road directions.

Refer to FIG. 7, which is a schematic flowchart of a method for reasoning on a road drivable area according to an embodiment of the present invention. As shown in Figure 7, the method includes:

S701. The vehicle-mounted device obtains the driving area perception information of the road, and determines the perceived driving area according to the driving area perception information.

Among them, the driving area perception information includes relevant information obtained by the environmental sensor of the vehicle, such as the information of the area in front of the vehicle obtained by the camera or lidar, including road information, obstacle information, and related information of surrounding vehicles; The wave radar obtains the speed information and relative position of the obstacle and the surrounding vehicles, and determines the acceleration of the obstacle and the surrounding vehicles according to the speed of the obstacle and the surrounding vehicles.

The vehicle-mounted device determines the perceived drivable area according to the relevant information obtained by the environmental sensor. The perceived drivable area is essentially a grid map, and the drivability value of each grid in the grid map is used to characterize the probability of the grid being drivable.

S702. The vehicle-mounted device verifies the perceived travelable area to obtain a verification result. Wherein, the verification result includes the first area and the second area.

Among them, the first area is a drivable area with reliable verification, and the second area is a drivable area with unreliable verification.

Specifically, the vehicle-mounted device verifies the perceived travelable area to obtain the verification result, which specifically includes:

Determine whether there is a road boundary on both sides of the perceived drivable area. If it is determined that the road boundary on one side or the road boundary on both sides of the perceived drivable area does not exist, determine that the perceived drivable area is a drivable area with unreliable verification; if It is determined that both sides of the road boundary of the perceivable drivable area are present, then the perceptible drivable area is divided to obtain multiple sub-areas; determine whether each sub-areas of the multiple sub-areas satisfies the following conditions 1-condition 4; if the sub-areas If I satisfies each of the conditions 1-condition 4, the sub-area I is determined to be a reliable sub-area; if the sub-area I does not meet any of the conditions 1-condition 4, the sub-area I is determined to be a check Unreliable sub-region; where, sub-region I is any one of multiple sub-regions.

The first area is an area composed of sub-areas with reliable verification among multiple sub-areas, and the second area is an area composed of sub-areas with reliable verification among multiple sub-areas.

Among them, condition 1: the width of sub-region I meets the following conditions:

k _min W≤w _i ≤k _max W

W is determined jointly by the experience width We of the drivable area and the memory width Wm of the drivable area.

W=ξW _e +(1-ξ)W _m

Among them, the value range of the driving area experience width We can be determined according to road construction specifications and research literature, and the driving area memory width Wm can be calculated based on the weighted average of the driving area width values for a period of time before the current moment.

Among them, the memory width of the drivable area

W _i at time i is travelable area memory width, [mu] _i of the weighting value W _i; the present time distances travelable area, the greater the weighted value width of the memory.

Condition 2: The angle between the boundary of the sub-region I and the boundary of the adjacent sub-region is not greater than the first preset angle.

Specifically, if the angle between the left boundary of the subregion I and the left boundary of its adjacent subregion is not greater than the first preset angle, and the angle between the right boundary of the subregion I and the right boundary of its adjacent subregion is not greater than the first preset angle, If the angle is set, it is determined whether the included angle between the boundary of the sub-region I and the boundary of the adjacent sub-region is not greater than the first preset angle; if the included angle between the left boundary of the sub-region I and the left boundary of the adjacent sub-region is greater than the first preset Assuming that the angle or the included angle between the right boundary of the subregion I and the right boundary of the adjacent subregion is greater than the first preset angle, it is determined that the included angle between the boundary of the subregion I and the boundary of the adjacent subregion is greater than the first preset angle .

It should be noted here that the size of the first preset angle depends on the degree of structure of the drivable area; the higher the degree of structure of the drivable area, the smaller the first predetermined angle.

For example, as shown in Figure 8, area I and area II are adjacent, and the angle between the left boundary of area I and the left boundary of area II is a vector

With vector

The angle between the right boundary of area I and the right boundary of area II is a vector

With vector

The included angle.

Condition 3: The distance between the boundary of the sub-region I and the boundary of the perceptual memory area verified before the current moment is not greater than the preset width.

Specifically, if the distance between the left boundary of the subregion I and the left boundary of the perceptual memory area verified before the current moment is not greater than the preset width, and the right boundary of the subregion I is equal to that verified before the current moment. The distance between the right boundary of the perceptual memory area is not greater than the preset width, then it is determined whether the distance between the boundary of the sub-area I and the boundary of the perceptual memory area verified before the current moment is not greater than the preset width; if the left boundary of the sub-area I The distance from the left boundary of the sensory memory area verified before the current moment is greater than the preset width or the distance between the right boundary of the sub-region I and the right boundary of the sensory memory area verified before the current moment is greater than the preset width, It is determined that the distance between the boundary of the sub-region I and the boundary of the perceptual memory area verified before the current moment is greater than the preset width.

Among them, the ratio of the drivable position points in sub-area I is

Is the number of driving position points in sub-area I,

Is the number of non-driving position points in subarea I.

S703. The vehicle-mounted device determines whether the first area covers the ROI.

Among them, the region of interest (region of interest, ROI) is the region of interest of the subsequent driving route decision planning module.

Wherein, if the first area covers the ROI, step S707 is executed; if the first area does not cover the ROI, step S704 is executed.

S704. The vehicle-mounted device infers the second area according to the perceptual memory information to obtain the third area and the fourth area.

Specifically, the perceptual memory information includes perceptual memory grid maps at multiple historical moments and the drivability value of each grid in each perceptual memory grid map, and the second region is inferred based on the perceptual memory information to obtain the first The third area and the fourth area include:

According to the first conversion formula, the perceptual memory grid maps of the multiple historical moments are respectively converted from the vehicle coordinate system of the vehicle at the historical moment to the world coordinate system to obtain the multiple world grid maps;

Among them, the first conversion formula is:

Is the first conversion matrix,

First conversion matrix

What needs to be explained here is that at different moments, the own vehicle is in different positions, and the vehicle coordinate system of the own vehicle is also changing. Therefore, in the vehicle coordinate system of the own vehicle at different times, the The coordinates of the same location point are different, and the perceptual memory grid maps at different historical moments need to be converted from the vehicle coordinate system of the own vehicle at the historical moment to the world coordinate system.

Among them, the second conversion formula is:

Is the second conversion matrix;

Second conversion matrix

Further, calculating the drivability value of each grid in the first inference grid map according to the drivability value of each grid in the perceptual memory grid map includes:

Perform a weighted summation on the drivability value of multiple historical moments corresponding to the grid in the p-th column and the q-th row in the first inference grid map to obtain the drivability value of each grid in the first inference grid map The drivability value of the multiple historical moments is the drivability value of the corresponding grid in the perceptual memory grid map of the multiple historical moments in the grid of the p-th column and the q-th row;

Sensing a corresponding memory in raster map raster historic time t travelable ability value, k _'t' is

the weight of.

It should be noted that, as shown in Figure 9, the upper part is the perceptual memory grid map of n historical moments, and the lower part is the area where the perceptual memory grid map of n historical moments overlaps with the second area, that is, the lower part Part is the first inference grid map. The gray grid in the perceptual memory grid map of n historical moments is the grid corresponding to the gray grid in the lower part.

S705. The vehicle-mounted device determines whether the first area and the third area cover the ROI.

Wherein, if the first area and the third area cover the ROI, step S707 is executed; if the first area and the third area do not cover the ROI, then step S706 is executed.

S706. The vehicle-mounted device infers the fourth area according to the information of the driving position point to obtain the fifth area.

In a feasible embodiment, before inferring the fourth area based on the drivable position point information, the vehicle-mounted device obtains the drivable area position point information, where the drivable position point information includes the self-vehicle drivable position point information and/ Or the location information of the surrounding vehicles.

What needs to be pointed out here is that the point information of the drivable position is expressed in coordinates in the world coordinate system.

In an example, the location information of the self-vehicle can be obtained from the driving information of the self-vehicle, and it can also be obtained from the roadside unit and/or the cloud information platform. The location information of the surrounding vehicles can be obtained according to the information of the surrounding vehicles. , It can also be obtained from the cloud information platform and/or from the roadside unit. After obtaining the driving position information of the own car according to the driving information of the own car, the driving position point information can be transmitted to the roadside unit and/or the cloud information platform; the vehicle-mounted device can obtain the surrounding vehicles according to the driving information of the surrounding vehicles The drivable position point information of the surrounding vehicles is transmitted to the roadside unit and/or cloud information platform; the roadside unit will obtain the drivable position point information of the surrounding vehicles and the drivable position of its own vehicle Point information is transmitted to the cloud information platform.

In a feasible embodiment, the vehicle-mounted device obtains the driving position point information of surrounding vehicles according to the driving information of the surrounding vehicles, including:

The vehicle-mounted device acquires the driving information of surrounding vehicles and the driving information of the own vehicle. The driving information of the surrounding vehicles includes relative position coordinates and longitudinal relative speed. The driving information of the own vehicle includes the absolute position coordinates, absolute speed and heading angle of the vehicle. , The relative position coordinates are the coordinates of the surrounding vehicles in the vehicle coordinate system, and the absolute position coordinates are the coordinates in the world coordinate system; the vehicle-mounted device obtains the surrounding vehicles according to the absolute position coordinates of the own vehicle, the heading angle of the vehicle and the relative position coordinates of the surrounding vehicles The drivable position point coordinates of the surrounding vehicles are the coordinates of the surrounding vehicles in the world coordinate system; the type of the drivable position point coordinates of the surrounding vehicles is determined according to the longitudinal relative speed of the surrounding vehicles and the absolute speed of the own vehicle, where , The types of the drivable position point coordinates include the reverse direction drivable position point coordinates and the same direction drivable position point coordinates.

Further, the vehicle-mounted device obtains the coordinates of the vehicle A's traversable position point according to the absolute position coordinates of the vehicle, the heading angle of the vehicle, and the relative position coordinates of the vehicle A, including:

Among them, the fourth conversion formula is:

Third conversion matrix

(x ₀ , y ₀ ) is the absolute position coordinate of the own vehicle at the current moment, θ ₀ is the heading angle of the own vehicle at the current moment, and vehicle A is any of the surrounding vehicles.

What needs to be explained here is that when obtaining the relative position coordinates and longitudinal relative speed of surrounding vehicles, the stationary vehicles are excluded; specifically, the following methods can be used:

Assuming that the absolute speed of the vehicle in the direction of travel is V _s , the absolute speed of vehicle A is

If the absolute speed V _{0 of} vehicle A is less than the first speed threshold _VT1 , vehicle A is determined to be a stationary vehicle; otherwise, vehicle A is determined to be a moving vehicle, where (ΔV _x , ΔV _y ) means that vehicle A is in the vehicle coordinate system The horizontal relative speed and the vertical relative speed of the bottom.

Further, the vehicle-mounted device determines the type of absolute position coordinates of the surrounding vehicles at t0 according to the absolute speed of the vehicle and the longitudinal relative speed of the surrounding vehicles. Specifically, the vehicle-mounted device determines the surrounding vehicles according to the longitudinal relative speed of the surrounding vehicles and the absolute speed of the vehicle. For vehicle A, if V _s +ΔV _x >V _T2 , it is determined that the driving direction of vehicle A is the same as the driving direction of its own vehicle according to the driving direction of surrounding vehicles. And then determine that the vehicle A’s travelable position point coordinates are the same direction travelable position point coordinates. If V _s +ΔV _x <-V _T2 , determine that the driving direction of vehicle A is opposite to the driving direction of the vehicle, and then determine the vehicle The coordinates of the drivable position point of A are the coordinates of the drivable position point in the reverse direction, where V _T2 is the second speed threshold.

Further, the vehicle-mounted device divides the coordinates of the driving position points of the surrounding vehicles into road direction 1 and road direction 2, and road direction 1 and road direction 2 are two opposite directions on the same road.

Specifically, as shown in FIG. 10, for the reverse direction travelable position point coordinates in the surrounding vehicles, if the own vehicle travels along the road direction 1, the vehicle-mounted device determines that the reverse direction travelable position point coordinates are the coordinates in the road direction 2. And save the coordinates of the reverse drivable position point to the roadside unit on the road direction 2 side; if the own vehicle is not traveling along the road direction 1, the vehicle-mounted device determines that the reverse drivable position point coordinates are the coordinates on the road direction 1. , And save the coordinates of the reverse drivable position point to the roadside unit on the side of the road direction 1. For the same direction drivable position point coordinates in the surrounding vehicles, if the vehicle is traveling along the road direction 1, the on-board device should synchronize The coordinates of the driving position point are the coordinates on the road direction 1, and the coordinates of the driving position point in the same direction are saved in the roadside unit on the side of the road direction 1. If the vehicle is not traveling along the road direction 1, the vehicle-mounted device It is determined that the coordinates of the driving position point in the same direction are the coordinates on the road direction 2, and the coordinates of the driving position point in the same direction are saved in the roadside unit on the side of the road direction 2.

It should be noted here that the process of obtaining the coordinates of the vehicle A's driving position point by the above-mentioned on-board device can be regarded as obtaining the coordinates of the driving position point of the surrounding vehicles at a certain moment, and the vehicle device may obtain the surrounding vehicles at different times according to the above method. The coordinates of the driving position point can be the driving position point information.

In a feasible embodiment, the roadside unit on the side of the road direction 1 sends the received coordinates of the drivable position point to the cloud information platform, and the roadside unit on the side of the road direction 2 sends the received coordinates of the drivable position point To the cloud information platform.

In a feasible embodiment, the in-vehicle device sends the driving position point information of surrounding vehicles to the cloud information platform.

In a feasible embodiment, the driving position points of the own vehicle include safe driving position points and driving risk position points, and the vehicle-mounted device obtains the driving position point information of the own vehicle according to the driving information of the own vehicle, which specifically includes:

The vehicle-mounted device obtains the coordinates of the current position of the vehicle and determines whether the current position of the vehicle is a safe driving position. The vehicle-mounted device determining whether the current position of the vehicle is a safe driving position includes: judging whether the vehicle is at the current position Whether the driving mode is manual driving mode, if it is determined that the driving mode of the own car at the current position is manual driving mode, the current position of the own car is determined to be a safe driving position; if the driving mode of the own car at the current position is determined to be In the automatic driving mode, it is judged whether there is a driving safety risk at the current position of the vehicle; if the vehicle has a driving safety risk at the current position, the current position of the vehicle is determined as the driving risk position; if the vehicle is at the current position If there is no driving safety risk, the current position of the vehicle is determined to be a safe driving position.

Further, the driving safety position point includes the driving safety position point on the road direction 1 and the driving safety position point on the road direction 2, and the driving risk position point includes the driving risk position point on the road direction 1 and the driving risk on the road direction 2. Location point, if the own vehicle is currently driving along road direction 1, then the safe driving position of the own vehicle is determined to be the safe driving position in road direction 1, and the driving risk position of the own vehicle is driving in road direction 1. Risk location points, and save the driving safety location points and driving risk location points of the own vehicle to the roadside unit on the side of the road direction 1. If the own vehicle is currently driving along the road direction 2, determine the driving safety location point of the own vehicle It is a safe driving position point in road direction 2, the driving risk position point of the own vehicle is the driving risk position point in road direction 2, and the driving safety position point and driving risk position point of the own vehicle are saved to road direction 2. In the roadside unit on the side.

In a feasible embodiment, the vehicle-mounted device determines whether the vehicle has a driving safety risk at the current location, specifically determining whether the vehicle has a collision risk or abnormal driving behavior at the current location. If it is determined that the vehicle is at the current location If there is a collision risk or abnormal driving behavior at the location point, it is determined that the own vehicle has a driving safety risk at the current location point; if it is determined that the own vehicle has no collision risk and no abnormal driving behavior at the current location point, it is determined that the own vehicle does not Driving safety risks.

In an example, determining whether the vehicle has a collision risk at the current location by the vehicle-mounted device specifically includes:

The vehicle-mounted device obtains the included angle θ formed by the traveling direction of the own vehicle and the traveling direction of the vehicle S, and the vehicle S is any of the surrounding vehicles traveling in the same direction as the own vehicle; when the included angle θ is greater than the second preset angle, according to the intersection The mode risk discrimination method determines whether the own vehicle has a collision risk at the current position; when the included angle θ is less than the second preset angle, the rear-end collision risk discrimination method determines whether the own vehicle has a collision risk at the current position.

Based on the method for judging the risk of the intersection mode shown in Figure 11, the relative speed of the vehicle E in the vehicle coordinate system is (ΔV _Ex , ΔV _Ey ), the coordinates of the relative position point B are (x _Ev , y _Ev ), and the vehicle is in the driving direction The absolute speed on is V _s , and the absolute speed of vehicle E in the vehicle coordinate system is (V _ox , _Voy ), where V _ox =V _s +ΔV _Ex , _Voy =ΔV _Ey , the vehicle is in the vehicle coordinate system The lower relative position point is denoted as A', as shown in Figure 11, point O is the potential collision point between the own vehicle and the vehicle E.

The time from the vehicle to the potential collision point O is:

The time from the weekly car to the potential collision point O is:

If |TTX ₁ -TTX ₂ |<α and

Then the vehicle-mounted device determines that the vehicle has a risk of collision at the current location point, where α is the preset threshold, and R ₀ is the risk threshold.

Optionally, the preset threshold α may be 0.5s, and the risk threshold R ₀ may be 0.5.

In a feasible embodiment, the vehicle-mounted device determines whether the vehicle has a collision risk at the current position according to a rear-end collision risk discrimination method, including:

If the lateral distance ψ between the vehicle in front of the vehicle and the vehicle in the surrounding vehicles satisfies the condition 1 and the TTC and TTC meet the condition 2, it is determined that the vehicle is at risk of collision at the current position:

Among them, condition 1 is: |y _v |<ψ, condition 2 is:

TTC is the collision time between the vehicle and the surrounding vehicle in front of it.

When the vehicle E is in front of its own vehicle,

When vehicle E is behind its own vehicle,

b and c are constants, ψ is the horizontal distance threshold, |y _Ev | is the horizontal distance between the own vehicle and the vehicle E.

In an example, the vehicle-mounted device determines whether the own vehicle has abnormal driving behavior at the current location point, which specifically includes:

The vehicle-mounted device determines whether there is emergency braking or emergency steering at the current location, and if it is determined that the own vehicle has emergency braking or emergency steering at the current location, it determines that the own vehicle has abnormal driving behavior at the current location ; If it is determined that the own vehicle has no emergency braking and no emergency steering behavior at the current position, it is determined that the own vehicle has no abnormal driving behavior at the current position.

Further, the vehicle-mounted device determines whether the own vehicle has an emergency braking behavior at the current position, which specifically includes:

The vehicle-mounted device obtains the longitudinal acceleration a _{lon of the} vehicle at the current position. If the longitudinal acceleration a _{lon is} less than the preset acceleration a _T , it is determined that the vehicle has emergency braking behavior at the current position; if the longitudinal acceleration a _{lon is} not less than the preset acceleration a _lon With acceleration a _T , it is determined that the vehicle has no emergency braking behavior at the current position.

Optionally, the preset acceleration a _T may be -6 m/s ² or other values.

Further, the vehicle-mounted device determines whether the own vehicle has an emergency steering behavior at the current position, which specifically includes:

The vehicle-mounted device obtains the steering wheel angle rate of the vehicle at the current position

If the steering wheel angle rate

Greater than preset rate

It is determined that the vehicle has an emergency steering behavior at the current position; if the steering wheel angle rate

Not greater than the preset rate

It is determined that the vehicle has no emergency steering behavior at the current position.

Optionally, preset rate

It can be 100°/s ² or other values.

In a feasible embodiment, the road-drivable position point information may be generated by the own vehicle or another vehicle, or may be obtained by the own vehicle from the roadside unit or cloud information platform of the road section where it is located.

In a feasible embodiment, the vehicle-mounted device can obtain the position point coordinates of the self-vehicle at different times according to the above method, and determine the safe driving position and the driving risk position of the self-vehicle at different times.

In a feasible embodiment, the vehicle-mounted device infers the fourth area according to the drivable location point to obtain the fifth area, which specifically includes:

Among them, the drivable location point information can also be represented by the data structure [timestamp,(x _w ,y _w ,θ _w ),Label], and timestamp is the time at which the drivable location point is obtained, that is, the timestamp; (x _w ,y _w , θ _w ) are the coordinates of the drivable position point in the world coordinate system and the angle of the front of the vehicle, and Label is used to characterize the type of the drivable position point. Label includes four types, namely direct, inverse, safe and danger. Among them, direct means that the drivable position point is the drivable position point of the car traveling in the same direction, inverse means the drivable position point is the drivable position point of the car traveling in the reverse direction, and safe means the drivable position point is the safety of the vehicle. The driving position point, danger indicates that the driving position point is the driving dangerous position point of the vehicle.

Specifically, the vehicle-mounted device converts the coordinates of the location point to be inferred from the world coordinate system to the vehicle coordinate system of the vehicle to obtain the travelable area to be inferred, including:

Among them, the third conversion formula is:

Is the second conversion matrix,

Second conversion matrix

In a feasible embodiment, the vehicle-mounted device calculates the drivability value of each grid according to the drivable position point information in each grid in the second inference grid map, which specifically includes:

The drivability values at different moments are calculated according to the drivable position point information in the i-th column and j-th row grid in the second inference grid map; the drivability values at different times are weighted and summed to obtain the first The drivability value of the grid in column i and row j.

Among them, the drivability value of the raster in the i-th column and the j-th row in the raster map of the drivable area

Is the drivability value at time t in the grid of column i and row j, k _t is

the weight of.

It should be noted here that the larger the timestamp t, the newer the data, and the larger the corresponding weight, thereby ensuring the real-time nature of the inference results of the road drivable area.

The drivability value at time t in the grid of column i and row j

It can be calculated using the following formula:

among them,

Represents the number of locations where the surrounding vehicles can travel in the same direction obtained at time t in the grid in the i-th column and the j-th row,

Represents the number of travelable location points of surrounding vehicles in the reverse direction obtained at time t in the grid of the i-th column and the j-th row,

Represents the number of safe driving position points obtained at time t in the grid of column i and row j,

Represents the number of dangerous location points of the self-driving vehicle obtained at time t in the grid of column i and row j,

Represents the number of travelable location points of surrounding vehicles traveling in the same direction obtained at time t in the j-th grid,

Represents the number of locations where the surrounding vehicles can travel in the reverse direction obtained at time t in the j-th grid,

Represents the number of safe driving position points obtained at time t in the j-th grid,

Represents the number of dangerous locations of the vehicle at time t in the j-th grid.

In a feasible embodiment, after calculating the drivability value of the safe driving position point and the driving dangerous position point, increase the drivability value of the safe driving position point and decrease the drivability value of the dangerous driving position point, It is helpful to improve the accuracy of the drivable area on the road.

Specifically, the fifth area is determined according to the drivability value of each grid, including:

According to the second threshold μ and the drivability value of the grid in the drivable area grid map, the drivable area grid map is divided into drivable area, non-driving area or uncertain area, where the drivable area is the drivable area The area composed of grids with a value greater than μ, the non-driving area is the area composed of grids with a raster drivability value less than -μ, and the uncertain area is a grid composed of grids with a drivability value not less than -μ and not greater than μ Area.

As shown in the right image of Figure 12, the black area is the drivable area, the gray area is the undrivable area, and the white area is the uncertain area.

S707. The vehicle-mounted device performs route decision planning according to the drivable area of the road to obtain a planned driving route.

Specifically, if the first area covers the ROI, the road drivable area includes the first area; if the first area and the third area cover the ROI, the road drivable area includes the first area and the third area; if the vehicle-mounted device executes the steps S706: The road traversable area includes a first area, a third area, and a fifth area.

In another embodiment, the cloud information platform obtains the drivable area of the own vehicle according to the relevant content executed by the on-board device in the above steps S702-S706, and then transmits the drivable area to the on-board device of the own vehicle. Make route decision planning according to the drivable area of the road to obtain the planned driving route.

In a feasible embodiment, the vehicle-mounted device verifies the perceived drivable area, and determines that the perceptual drivable area is a reliable travelable area; if the verified travelable area covers the ROI, the vehicle-mounted device determines The perceptually reliable drivable area performs route decision planning to obtain a planned driving route; if the verified drivable area does not cover the ROI, the vehicle-mounted device compares the ROI with the perceptual memory area except for the verified drivable area The reasoning is performed outside the area. For the specific reasoning process, please refer to the relevant description of step S704, which will not be described here.

In a feasible embodiment, if the vehicle-mounted device verifies the perceived travelable area, and it is determined that the perceived travelable area is a travelable area with unreliable verification, the vehicle-mounted device is not reliable for the verification based on the perceived memory area For the specific reasoning process, refer to the relevant description of step S704, which will not be described here.

It can be seen that in the solution of the embodiment of the present application, the perceived drivable area is acquired; the perceptual drivable area is verified to obtain the first area and the second area, where the first area is a reliable drivable area. Area, the second area is a drivable area with unreliable verification; if the first area does not cover the area of interest ROI, the second area is inferred based on the perceptual memory information of the drivable area to obtain the third area and the fourth area , The third area is the area where the perceptual memory area overlaps with the second area, and the fourth area is the area not covered by the perceptual memory area in the second area; if the first area and the third area do not cover the ROI, then according to the driving position point The fourth area is inferred to obtain the fifth area; the fifth area is the drivable area in the fourth area; the first area, the third area and the fifth area are determined as road drivable areas. According to the position information of the traffic flow in the same direction, the position information of the traffic flow in the reverse direction, the safe road position of automatic driving, the position of the risky road of automatic driving, and the driving position of the human driving mode, the road can be driven position points are generated, and with the help of car end-cloud-roadside The end data sharing mode allows all vehicles to use the data to reason about the drivable area. It is suitable for structured roads and unstructured roads, does not depend on the vehicle's own motion state, and does not require other vehicles around the vehicle in real time. The automatic driving system can therefore make decision planning with the use of reasoning information in the drivable area when it perceives short-term and long-term abnormalities in the drivable area, avoiding the failure of the automatic driving system, increasing the coverage of the operating conditions of the system, and improving system availability and user experience. The invention can reduce the dependence of the automatic driving system on real-time perception, increase the fault-tolerant ability of the automatic driving system on real-time perception, and improve the reliability and safety of the automatic driving system when the perception of the road drivable area is uncertain.

When reasoning with the help of drivable location points, the data of the drivable location points with a newer timestamp has a higher weight, so as to ensure that the inference results have better real-time performance and avoid temporary changes in road structure (such as road construction). Negative Effects. The specific implementation takes into account the historical driving position of the human driver and the safe road position in automatic driving mode (increase the drivability value of this position) and the risk road position in the automatic driving mode (decrease the drivable value of this position). For the user's common driving route, the automatic driving system using this application has the characteristic of "the more open the better".

Referring to FIG. 13, FIG. 13 is a schematic structural diagram of a road drivable area reasoning device provided by an embodiment of the present invention. As shown in FIG. 13, the device 1300 for inference of road drivable area includes:

The obtaining module 1301 is used to obtain the perceived drivable area;

The verification module 1302 is used to verify the perceived drivable area to obtain the first area and the second area, where the first area is a drivable area with reliable verification, and the second area is a drivable area with unreliable verification. Driving area

The inference module 1303 is used to if the first area does not cover the area of interest ROI, infer the second area according to the perceptual memory information of the drivable area to obtain the third area and the fourth area. The third area is the perceptual memory area and The area where the second area overlaps, and the fourth area is the area that is not covered by the sensory memory area in the second area; if the first area and the third area do not cover the ROI, the fourth area is performed according to the driving position. Reason to get the fifth area; the fifth area is the drivable area in the fourth area;

The determining module 1304 is configured to determine the first area, the third area, and the fifth area as road drivable areas.

In a feasible embodiment, the verification module 1302 is specifically configured to:

In a feasible embodiment, condition 1 to condition 4 are:

Condition 1: The width of sub-region I satisfies the following conditions:

k _min W≤w _i ≤k _max W

In a feasible embodiment, the perceptual memory information includes perceptual memory grid maps at multiple historical moments and the drivability value of each grid in each perceptual memory grid map. The perceptual memory information is used to compare the second area Perform reasoning to obtain aspects of the third area and the fourth area, and the reasoning module 1303 is specifically used to:

In a feasible embodiment, in the aspect of converting the perceptual memory grid maps of multiple historical moments from the vehicle coordinate system of the vehicle at the historical moment to the world coordinate system to obtain multiple world grid maps, The reasoning module 1303 is specifically used for:

Among them, the first conversion formula is:

Is the first conversion matrix,

First conversion matrix

In a feasible embodiment, in terms of converting the inference area from the world coordinate system to the vehicle coordinate system of the vehicle at the current moment to obtain the first inference grid map, the inference module 1303 is specifically configured to:

Among them, the second conversion formula is:

Is the second conversion matrix;

Second conversion matrix

In a feasible embodiment, in terms of calculating the drivability value of each grid in the first inference grid map according to the drivability value of each grid in the perceptual memory grid map, the inference module 1303 specifically uses in:

the weight of.

In a feasible embodiment, in terms of inferring the fourth area according to the driving position point to obtain the fifth area, the inference module 1303 is specifically configured to:

In a feasible embodiment, in terms of transforming the coordinates of the location point to be inferred from the world coordinate system to the vehicle coordinate system of the vehicle to obtain the travelable area to be inferred, the inference module 1303 is specifically configured to:

Among them, the third conversion formula is:

Is the second conversion matrix,

Second conversion matrix

In a feasible embodiment, in terms of calculating the drivability value of each grid according to the drivable position point information in each grid in the second inference grid map, the inference module 1303 is specifically configured to:

Is the drivability value at time t, k _t is

the weight of,

In a feasible embodiment, the drivable position point includes the drivable position point of the self-vehicle, and the acquisition module 1301 is further used for:

In a feasible embodiment, in terms of judging whether the vehicle has a collision risk at its current position, the acquiring module 1301 is specifically configured to:

In a feasible embodiment, in terms of using the intersection mode risk discrimination method to determine whether the vehicle has a collision risk at the current position, the acquiring module 1301 is specifically configured to:

In a feasible embodiment, in terms of using the rear-end collision mode risk discrimination method to determine whether the vehicle has a collision risk at the current position, the acquiring module 1301 is specifically configured to:

Among them, the formula 3 is: |y _Ev |<ψ, and the formula 4 is:

In a feasible embodiment, in terms of judging whether the own vehicle has abnormal driving behavior at its current position, the acquiring module 1301 is specifically configured to:

In a feasible embodiment, in terms of determining whether the own vehicle has an emergency braking behavior at its current position, the acquiring module 1301 is specifically configured to:

In a feasible embodiment, in terms of determining whether the own vehicle has an emergency steering behavior at its current position, the acquiring module 1301 is specifically configured to:

In a feasible embodiment, the road drivable area reasoning device 1300 further includes a storage module 1305;

The determining module 1304 is also used for determining that the driving position point of the own vehicle is the driving position in the road direction 1 if the driving position point of the own vehicle is driving along the road direction 1 at its current position after the acquiring module 1301 acquires Location point, saving module 1305, used to save the drivable location point on the road direction 1 to the roadside unit on the side of the road direction 1, where the drivable location point on the road direction 1 includes the driving safety in the road direction 1. Location and driving risk location on road direction 1;

The determination module 1304 is also used for determining that the vehicle can travel along the road direction 2 at its current position, and then determine the travelable location point of the vehicle as the travelable location point on the road direction 2, and the storage module 1305 is used to change the road direction 2 The drivable position points on the road side are saved to the roadside unit on the road direction 2 side, where the drivable position points on the road direction 2 include the safe driving position on the road direction 1 and the driving risk position on the road direction 2; wherein, Road direction 1 and road direction 2 are opposite directions on the same road.

In a feasible embodiment, the obtaining module 1301 is also used to:

Get the driving position point information of surrounding vehicles.

In a feasible embodiment, the driving position point information of surrounding vehicles includes the coordinates of the driving position point in the same direction and the coordinates of the driving position point in the reverse direction. In terms of obtaining the driving position point information of the surrounding vehicles, the acquisition module 1301 also uses in:

In a feasible embodiment, in terms of acquiring the drivable position point coordinates of vehicle A according to the absolute position coordinates of the own vehicle, the heading angle of the vehicle, and the relative position coordinates of vehicle A, the acquiring module 1301 is further used to:

Among them, the fourth conversion formula is:

Third conversion matrix

In a feasible embodiment, in terms of determining the type of the vehicle A's drivable position point coordinates according to the longitudinal relative speed and absolute speed of the vehicle A, the acquiring module 1301 is also used to:

In a feasible embodiment, the determining module 1304 is also used to determine if the vehicle A is driving along the road direction 1, the coordinates of the vehicle A's driving position point are the coordinates in the road direction 1, and the saving module 1305 is also used to Save the point coordinates of the vehicle A's driveable position to the roadside unit on the side of the road direction 1;

The determining module 1304 is also used to determine if the vehicle A travels along the road direction 2, the driveable position point of the lane A is the coordinate on the road direction 2, and the storage module 1305 is used to save the vehicle A's driveable position point coordinates To the roadside unit on the side of road direction 2; wherein, road direction 1 and road direction 2 are two opposite directions on the same road.

In a feasible embodiment, in terms of acquiring information about the drivable location points of surrounding vehicles, the acquiring module 1301 is specifically configured to:

In a feasible embodiment, the determining module 1304 is further configured to:

It should be noted that the above-mentioned units (the acquisition module 1301, the verification module 1302, the inference module 1303, the determination module 1304, and the storage module 1305) are used to execute the relevant steps of the above method. For example, the acquiring module 1301 is used to execute the relevant content of steps S701 and S706, the verification module 1302 is used to execute the relevant content of step S702, the inference module 1303 is used to execute the relevant content of steps S704 and S706, the determining module 1304 and the saving module 1305 Used to perform S702-S706 related content.

In this embodiment, the road drivable area reasoning device 1300 is presented in the form of a module. The "module" here can refer to application-specific integrated circuits (ASICs), processors and memories that execute one or more software or firmware programs, integrated logic circuits, and/or other devices that can provide the above functions . In addition, the above acquisition module 1301, verification module 1302, inference module 1303, and determination module 1304 can be implemented by the processor 1401 of the road drivable area inference device shown in FIG. 14.

As shown in FIG. 14, the road drivable area inference device 1400 can be implemented with the structure in FIG. 14. The road drivable area inference device 1400 includes at least one processor 1401, at least one memory 1402 and at least one communication interface 1403. The processor 1401, the memory 1402, and the communication interface 1403 are connected through the communication bus and complete mutual communication.

The processor 1401 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the above program programs.

The communication interface 1403 is used to communicate with other devices or communication networks, such as Ethernet, wireless access network (RAN), wireless local area network (Wireless Local Area Networks, WLAN), etc.

The memory 1402 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM), or other types that can store information and instructions The dynamic storage device can also be Electrically Erasable Programmable Read-Only Memory (EEPROM), CD-ROM (Compact Disc Read-Only Memory, CD-ROM) or other optical disc storage, optical disc storage (Including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or can be used to carry or store desired program codes in the form of instructions or data structures and can be used by a computer Any other media accessed, but not limited to this. The memory can exist independently and is connected to the processor through a bus. The memory can also be integrated with the processor.

Wherein, the memory 1402 is used to store application program codes for executing the above solutions, and the processor 1401 controls the execution. The processor 1401 is configured to execute application program codes stored in the memory 1402.

The code stored in the memory 1402 can execute the above-provided road drivable area reasoning method, such as:

Examples of program products:

In some embodiments, the disclosed methods may be implemented as computer program instructions encoded on a computer-readable storage medium in a machine-readable format or encoded on other non-transitory media or articles. Figure 15 schematically illustrates a conceptual partial view of an example computer program product arranged in accordance with at least some of the embodiments shown herein, the example computer program product comprising a computer program for executing a computer process on a computing device. In one embodiment, the example computer program product 1500 is provided using a signal bearing medium 1501. The signal bearing medium 1501 may include one or more program instructions 1502, which, when executed by one or more processors, can provide the functions or part of the functions described above with respect to FIG. 7. Thus, for example, referring to the embodiment shown in FIG. 3, one or more of the features of blocks 302-306 may be undertaken by one or more instructions associated with the signal bearing medium 1501. In addition, program instructions 1502 in FIG. 15 also describe example instructions.

In some examples, the signal-bearing medium 1501 may include a computer-readable medium 1503, such as, but not limited to, a hard disk drive, compact disk (CD), digital video compact disk (DVD), digital tape, memory, read-only storage memory (Read -Only Memory, ROM) or Random Access Memory (RAM), etc. In some embodiments, the signal bearing medium 1501 may include a computer recordable medium 1504, such as, but not limited to, memory, read/write (R/W) CD, R/W DVD, and so on. In some embodiments, the signal-bearing medium 1501 may include a communication medium 1505, such as, but not limited to, digital and/or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.). Therefore, for example, the signal bearing medium 1501 may be communicated by a wireless communication medium 1505 (for example, a wireless communication medium that complies with the IEEE 802.11 standard or other transmission protocols). The one or more program instructions 1502 may be, for example, computer-executable instructions or logic-implemented instructions. In some examples, a computing device such as that described with respect to FIG. 7 may be configured to, in response to communicating to the computing device via one or more of the computer readable medium 1503, the computer recordable medium 1504, and/or the communication medium 1505, The program instructions 1502 provide various operations, functions, or actions. It should be understood that the arrangement described here is for illustrative purposes only. Thus, those skilled in the art will understand that other arrangements and other elements (for example, machines, interfaces, functions, sequences, and functional groups, etc.) can be used instead, and some elements can be omitted altogether depending on the desired result . In addition, many of the described elements are functional entities that can be implemented as discrete or distributed components, or combined with other components in any appropriate combination and position.

It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described sequence of actions. Because according to the present invention, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the involved actions and modules are not necessarily required by the present invention.

In the above-mentioned embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in an embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed device may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable memory. Based on this understanding, the technical solution of the present invention essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, A number of instructions are included to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned memory includes: U disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable memory, and the memory can include: flash disk , Read-only memory (English: Read-Only Memory, abbreviation: ROM), random access device (English: Random Access Memory, abbreviation: RAM), magnetic disk or optical disc, etc.

The embodiments of the present invention are described in detail above, and specific examples are used in this article to illustrate the principles and implementation of the present invention. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present invention; Those of ordinary skill in the art, based on the idea of the present invention, will have changes in the specific implementation and the scope of application. In summary, the content of this specification should not be construed as limiting the present invention.

Claims

A method for reasoning on a road drivable area, which is characterized in that it includes:

Obtain the perceived drivable area;

Perform verification on the perceived drivable area to obtain a first area and a second area, wherein the first area is a drivable area with reliable verification, and the second area is a drivable area with unreliable verification. area;

If the first area does not cover the area of interest ROI, the second area is inferred based on the perceptual memory information of the drivable area to obtain the third area and the fourth area, and the third area is the perceptual memory region and the second region overlap region, the fourth region to the second region in the sensing region of the memory region that is not covered;

If the first area and the third area do not cover the ROI, the fourth area is inferred based on the drivable position point to obtain the fifth area; the fifth area is the fourth area The drivable area in

The first area, the third area and the fifth area are determined as road-driving areas.
The method according to claim 1, wherein the verifying the perceived drivable area to obtain the first area and the second area comprises:

Determine whether there is a road boundary on both sides of the perceived drivable area;

If it is determined that the two-sided road boundary of the perceived drivable area exists, then the perceptible drivable area is divided into areas to obtain multiple sub-areas;

Determine that the subregion I satisfies each of the conditions 1 to 4; if the subregion I meets each of the conditions 1 to 4, determine that the subregion I is a subregion with reliable verification; if If the sub-area I does not meet any one of conditions 1 to 4, it is determined that the sub-area I is an area with unreliable verification;

Wherein, the sub-region I is any one of the multiple sub-regions, the first region is an area formed by sub-regions with reliable verification among the multiple sub-regions, and the second region is the multiple sub-regions. The region is composed of sub-regions whose verification is unreliable.
The method according to claim 2, wherein the condition 1 to condition 4 are:

Condition 1: The width of the sub-region I satisfies the following conditions:

k min W≤w i ≤k max W

Wherein, the w i is the width of the sub-region I, and the W is determined according to the empirical width of the drivable area and the memory width of the drivable area;

Condition 2: The angle between the boundary of the sub-region I and the boundary of the adjacent sub-region is not greater than the first preset angle;

Condition 3: The distance between the boundary of the sub-region I and the boundary of the perceptual memory area verified before the current moment is not greater than a preset width;

Condition 4: The ratio of the drivable position points in the sub-region I is greater than the preset ratio.
The method according to any one of claims 1 to 3, wherein the perceptual memory information includes perceptual memory grid maps at multiple historical moments and the perceptual memory grid maps of each perceptual memory grid map. The drivability value, inferring the second area according to the perceptual memory information to obtain the third area and the fourth area, includes:

Transforming the sensory memory grid maps of the multiple historical moments from the vehicle coordinate system of the vehicle at the historical moment to the world coordinate system to obtain multiple world grid maps;

Acquiring an inference area, where the inference area is an area where the multiple world grid maps overlap the second area;

Converting the inference area from the world coordinate system to the vehicle coordinate system of the vehicle at the current moment to obtain the first inference grid map;

Calculating the drivability value of each grid in the first reasoning grid map according to the drivability value of the grid in the perceptual memory grid map;

The third area and the fourth area are determined according to the drivability value of each grid in the first inference grid map; the third area is the drivability in the first inference grid map An area composed of grids with a value greater than a first threshold; the fourth area is an area composed of grids with a drivability value not greater than the first threshold in the first inference grid map.
The method according to claim 4, wherein the perceptual memory grid maps of the multiple historical moments are respectively converted from the vehicle coordinate system of the own vehicle at the historical moment to the world coordinate system to obtain multiple A grid map of the world, including:

Convert the sensory memory grid maps of the multiple historical moments from the vehicle coordinate system of the vehicle at the historical moment to the world coordinate system respectively according to the first conversion formula to obtain the multiple world grid maps;

Wherein, the first conversion formula is:
Where (x vt0 ,y vt0 ) is the coordinates of any drivable position P in the sensory memory grid map at historical time t0 in the vehicle coordinate system of the own vehicle, (x wt0 ,y wt0 ) is The coordinates of the drivable position point P in the world coordinate system, the
Is the first conversion matrix,

The first conversion matrix
(x t0 , y t0 ) are the coordinates of the vehicle at the historical time t0 in the world standard system, and the θ t0 is the heading angle of the vehicle at the historical time t0.
The method according to claim 5, wherein the converting the inference area from the world coordinate system to the vehicle coordinate system of the vehicle at the current moment to obtain the first inference grid map comprises:

Converting the inference area from the world coordinate system to the vehicle coordinate system of the vehicle at the current moment according to the second conversion formula to obtain the first inference grid map;

Wherein, the second conversion formula is:
(x wp ,y wp ) is the reasoning area

The coordinates of any drivable position point P'in the domain in the world coordinate system, (x vp , y vp ) are the coordinates of the drivable position point P’ in the vehicle coordinate system of the own vehicle at the current moment, the
Is the second conversion matrix;

The second conversion matrix
The (x 0 , y 0 ) is the coordinates of the own vehicle in the world coordinate system at the current moment, and the θ 0 is the heading angle of the own vehicle at the current moment.
The method according to any one of claims 4-6, wherein the calculation of each grid in the first inference grid map according to the drivability value of each grid in the perceptual memory grid map The drivability value of the grid, including:

Perform a weighted summation of the drivability values of multiple historical moments corresponding to the grid in the p-th column and the q-th row in the first inference grid map to obtain the value of each grid in the first inference grid map The drivability value; the drivability value of the multiple historical moments is the drivability value of the corresponding grid in the perceptual memory grid map of the p-th column and the q-th row grid in the multiple historical moments ；

Wherein, the drivability value of the grid in the p-th column and the q-th row in the first inference grid map is:

As historic time t 'corresponding to the sensing grid map raster memory may driving capability value, the k' t 'is
the weight of.
The method according to any one of claims 1-7, wherein the inferring the fourth area according to the drivable location point to obtain the fifth area comprises:

Acquiring a to-be-inferred position point from the drivable position point, where the to-be-inferred position point is a drivable position point located in an area where the fourth area overlaps the ROI;

The coordinates of the location point to be inferred are converted from the world coordinate system to the vehicle coordinate system of the own vehicle to obtain the driveable area to be inferred, and the driveable area to be inferred is in the vehicle coordinate system of the own vehicle The area constituted by the location points to be inferred;

Grid division of the driving area to be inferred to obtain a second inference grid map;

Calculating the drivability value of each grid according to the drivable position point information in each grid in the second inference grid map;

The fifth area is determined according to the drivability value of each grid, where the fifth area is an area composed of grids with drivability values greater than a second threshold in the second inference grid map .
The method according to claim 8, wherein the converting the coordinates of the location point to be inferred from the world coordinate system to the vehicle coordinate system of the own vehicle to obtain the driveable area to be inferred comprises:

Convert the coordinates of the location point to be inferred according to the third conversion formula to obtain the driveable area to be inferred;

Wherein, the third conversion formula is:
Wherein, (x dw , y dw ) is the coordinates of any position point D to be inferred in the world coordinate system, (x dv , y dv ) is the position point D to be inferred in the The coordinates in the vehicle coordinate system of the own vehicle, the
Is the second conversion matrix,

The second conversion matrix
The (x 0 , y 0 ) is the coordinates of the own vehicle in the world coordinate system at the current moment, and the θ 0 is the heading angle of the own vehicle at the current moment.
The method according to claim 8 or 9, wherein the calculation of the drivability value of each grid according to the drivable position point information in each grid in the second inference grid map ,include:

Calculate the drivability value at different moments according to the drivable position point information in the i-th column and j-th row grid in the second inference grid map;

Perform a weighted summation of the drivability values at different moments to obtain the drivability value of the grid in the i-th column and j-th row;

Wherein, the drivability value of the grid in the i-th column and the j-th row is
Is the drivability value at time t, k t is the
The weight of the

Said
Is the number of travelable location points of surrounding vehicles traveling in the same direction acquired at time t in the grid of the i-th column and the j-th row, the
Is the number of travelable location points of surrounding vehicles in the reverse direction obtained at time t in the grid of the i-th column and the j-th row, the
Is the number of self-driving safe position points acquired at time t in the grid of the i-th column and the j-th row, the
Is the number of dangerous location points of the self-driving vehicle obtained at time t in the grid of the i-th column and the j-th row, the
Is the number of travelable location points of surrounding vehicles traveling in the same direction acquired at time t in the j-th grid, the
Is the number of travelable location points of surrounding vehicles in the reverse direction acquired at time t in the j-th grid, the
Is the number of safe driving position points obtained at time t in the j-th grid, the
Is the number of dangerous location points of the self-driving vehicle acquired at time t in the j-th grid.
The method according to any one of claims 1-10, wherein the drivable position point comprises a self-vehicle drivable position point, and before inferring the fourth area based on the drivable position point information, The method also includes:

Acquiring the driving position point of the own vehicle, and the driving position point information of the own vehicle includes a driving safety position point and a driving risk position point; wherein, acquiring the driving position point of the own vehicle includes:

Determine whether the driving mode of the own vehicle at its current position is manual driving mode; if the driving mode of the own vehicle at its current position is manual driving mode, determine that the current position of the own vehicle is the safe driving position point ；

If the driving mode of the self-vehicle at its current position is the automatic driving mode, determine whether the self-vehicle has a risk of collision or abnormal driving behavior at its current position; if it is determined that the self-vehicle has no risk of collision at its current position If there is no abnormal driving behavior, the current position of the own vehicle is determined to be the safe driving position; if it is determined that the own vehicle has a collision risk or abnormal driving behavior at its current position, then the current position of the own vehicle is determined The location point is a dangerous location point for driving.
The method according to claim 11, wherein the judging whether the own vehicle has a collision risk at its current position comprises:

Obtain the angle θ between the driving direction of the own vehicle and the vehicle E; the vehicle E is the surrounding vehicles of the own vehicle;

If the included angle θ is greater than the second preset angle, adopt an intersection mode risk judgment method to determine whether the self-vehicle has a collision risk at its current position;

If the included angle θ is not greater than the second preset angle, a rear-end collision mode risk determination method is adopted to determine whether the own vehicle has a collision risk at its current position.
The method according to claim 11 or 12, wherein the drivable location point comprises a drivable location point of surrounding vehicles, and the method further comprises:

Acquire the drivable position point information of the surrounding vehicles, where the drivable position point information of the surrounding vehicles includes the coordinates of the driving position point in the same direction and the coordinates of the driving position point in the reverse direction,

The obtaining information of the driving position points of surrounding vehicles includes:

Acquire driving information of any vehicle A in the surrounding vehicles and driving information of its own vehicle, wherein the driving information of the vehicle A includes relative position coordinates and longitudinal relative speed, and the driving information of the own vehicle includes absolute position coordinates, The absolute speed of the direction and the heading angle of the car;

Obtaining the coordinates of the driving position point of the vehicle A according to the absolute position coordinates of the own vehicle, the heading angle of the vehicle, and the relative position coordinates of the vehicle A;

The type of the drivable position point of the vehicle A is determined according to the longitudinal relative speed of the vehicle A and the absolute speed of the own vehicle; the type of the drivable position point coordinate of the vehicle A includes the reverse drivable position point coordinate or the same Point coordinates to the driving position;

Wherein, the relative position coordinates are coordinates in the vehicle coordinate system, and the drivable position point coordinates of the vehicle A are coordinates in the world coordinate system.
The method according to claim 13, wherein the obtaining the coordinates of the drivable position of the vehicle A according to the absolute position coordinates of the own vehicle, the heading angle of the vehicle, and the relative position coordinates of the vehicle A comprises :

Calculate the absolute position coordinates of the own vehicle, the heading angle of the vehicle, and the relative position coordinates of the vehicle A through a fourth conversion formula to obtain the absolute position point coordinates of the vehicle A;

Among them, the fourth conversion formula is:
(x Av , y Av ) are the relative position coordinates of the vehicle A, and (x Aw , y Aw ) are the coordinates of the driving position point of the vehicle A;

Third conversion matrix
The (x 0 , y 0 ) is the absolute position coordinate of the own vehicle at the current moment, and the θ 0 is the heading angle of the own vehicle at the current moment.
The method according to claim 13 or 14, wherein the determining the type of the vehicle A's position point coordinates according to the longitudinal relative speed and the absolute speed of the vehicle A comprises:

Acquiring the longitudinal absolute speed of the vehicle A according to the longitudinal relative speed of the vehicle A and the absolute speed;

If the longitudinal absolute speed of the vehicle A is greater than the preset speed threshold, determining that the vehicle A can drive position point coordinates are the same direction can drive position point coordinates;

If the longitudinal absolute speed of the vehicle A is less than the preset speed threshold, it is determined that the vehicle A can travel position point coordinates as the reverse travel possible position point coordinates.
A reasoning device for road drivable area, characterized in that it comprises:

The acquisition module is used to acquire the perceived drivable area;

The verification module is used for verifying the perceived drivable area to obtain a first area and a second area, wherein the first area is a travelable area with reliable verification, and the second area is a school Unreliable driving area;

The reasoning module is configured to, if the first region does not cover the region of interest ROI, perform inference on the second region according to the perceptual memory information of the drivable region to obtain the third region and the fourth region, the third region Is an area where the sensory memory area overlaps the second area, and the fourth area is an area in the second area that the sensory memory area cannot cover;

The inference module is further configured to, if the first area and the third area do not cover the ROI, perform inference on the fourth area according to the driving position point to obtain the fifth area; The fifth area is the drivable area in the fourth area;

The determining module is used to determine the first area, the third area, and the fifth area as road-drivable areas.
The device according to claim 16, wherein the verification module is specifically configured to:

Determine whether there is a road boundary on both sides of the perceived drivable area;

If it is determined that the two-sided road boundary of the perceived drivable area exists, then the perceptible drivable area is divided into areas to obtain multiple sub-areas;

Determine that the subregion I satisfies each of the conditions 1 to 4; if the subregion I meets each of the conditions 1 to 4, determine that the subregion I is a subregion with reliable verification; if If the sub-area I does not meet any one of conditions 1 to 4, it is determined that the sub-area I is an area with unreliable verification;

Wherein, the sub-region I is any one of the multiple sub-regions, the first region is an area formed by sub-regions with reliable verification among the multiple sub-regions, and the second region is the multiple sub-regions. The region is composed of sub-regions whose verification is unreliable.
The device according to claim 17, wherein the conditions 1 to 4 are:

Condition 1: The width of the sub-region I satisfies the following conditions:

k min W≤w i ≤k max W

Wherein, the w i is the width of the sub-region I, and the W is determined according to the empirical width of the drivable area and the memory width of the drivable area;

Condition 2: The angle between the boundary of the sub-region I and the boundary of the adjacent sub-region is not greater than the first preset angle;

Condition 3: The distance between the boundary of the sub-region I and the boundary of the perceptual memory area verified before the current moment is not greater than a preset width;

Condition 4: The ratio of the drivable position points in the sub-region I is greater than the preset ratio.
The device according to any one of claims 16-18, wherein the perceptual memory information comprises perceptual memory grid maps of a plurality of historical moments and the perceptual memory grid maps of each perceptual memory grid map. The drivability value, in terms of inferring the second area based on the perceptual memory information to obtain the third area and the fourth area, the inference module is specifically configured to:

Transforming the sensory memory grid maps of the multiple historical moments from the vehicle coordinate system of the vehicle at the historical moment to the world coordinate system to obtain multiple world grid maps;

Acquiring a reasoning area, where the reasoning area is an area where the multiple world grid maps overlap the second area;

Converting the inference area from the world coordinate system to the vehicle coordinate system of the vehicle at the current moment to obtain the first inference grid map;

Calculating the drivability value of each grid in the first reasoning grid map according to the drivability value of the grid in the perceptual memory grid map;

The third area and the fourth area are determined according to the drivability value of each grid in the first inference grid map; the third area is the drivability in the first inference grid map An area composed of grids with a value greater than a first threshold; the fourth area is an area composed of grids with a drivability value not greater than the first threshold in the first inference grid map.
The device according to claim 19, wherein the sensory memory grid maps of the multiple historical moments are respectively converted from the vehicle coordinate system of the own vehicle at the historical moment to the world coordinate system to obtain multiple Regarding the world grid map, the reasoning module is specifically used for:

Convert the sensory memory grid maps of the multiple historical moments from the vehicle coordinate system of the vehicle at the historical moment to the world coordinate system respectively according to the first conversion formula to obtain the multiple world grid maps;

Wherein, the first conversion formula is:
Where (x vt0 ,y vt0 ) is the coordinates of any drivable position P in the sensory memory grid map at historical time t0 in the vehicle coordinate system of the own vehicle, (x wt0 ,y wt0 ) is The coordinates of the drivable position point P in the world coordinate system, the
Is the first conversion matrix, the first conversion matrix
(x t0 , y t0 ) are the coordinates of the vehicle at the historical time t0 in the world standard system, and the θ t0 is the heading angle of the vehicle at the historical time t0.
The device according to claim 20, characterized in that, in the aspect of transforming the inference area from the world coordinate system to the vehicle coordinate system of the vehicle at the current moment to obtain the first inference grid map, the inference area The module is specifically used for:

Converting the inference area from the world coordinate system to the vehicle coordinate system of the vehicle at the current moment according to the second conversion formula to obtain the first inference grid map;

Wherein, the second conversion formula is:
(x wp ,y wp ) is the coordinates of any drivable position point P'in the inference area in the world coordinate system, (x vp ,y vp ) is the drivable position point P'at the current moment The coordinates in the vehicle coordinate system of the own vehicle, the
Is the second conversion matrix;

The second conversion matrix
The (x 0 , y 0 ) is the coordinates of the own vehicle in the world coordinate system at the current moment, and the θ 0 is the heading angle of the own vehicle at the current moment.
The device according to any one of claims 19-21, wherein each grid in the first inference grid map is calculated according to the drivability value of each grid in the perceptual memory grid map In terms of the drivability value of, the reasoning module is specifically used for:

Perform a weighted summation of the drivability values of multiple historical moments corresponding to the grid in the p-th column and the q-th row in the first inference grid map to obtain the value of each grid in the first inference grid map The drivability value; the drivability value of the multiple historical moments is the drivability value of the corresponding grid in the perceptual memory grid map of the p-th column and the q-th row grid in the multiple historical moments ；

Wherein, the drivability value of the grid in the p-th column and the q-th row in the first inference grid map is:

As historic time t 'corresponding to the sensing grid map raster memory may driving capability value, the k' t 'is
the weight of.
The device according to any one of claims 16-22, wherein, in terms of inferring the fourth area according to the drivable position point to obtain the fifth area, the inference module is specifically configured to:

Acquiring a to-be-inferred position point from the drivable position point, where the to-be-inferred position point is a drivable position point located in an area where the fourth area overlaps the ROI;

The coordinates of the location point to be inferred are converted from the world coordinate system to the vehicle coordinate system of the own vehicle to obtain the driveable area to be inferred, and the driveable area to be inferred is in the vehicle coordinate system of the own vehicle The area constituted by the location points to be inferred;

Grid division of the driving area to be inferred to obtain a second inference grid map;

Calculating the drivability value of each grid according to the drivable position point information in each grid in the second inference grid map;

The fifth area is determined according to the drivability value of each grid, where the fifth area is an area composed of grids with drivability values greater than a second threshold in the second inference grid map .
The device according to claim 23, characterized in that, in the aspect of transforming the coordinates of the location point to be inferred from the world coordinate system to the vehicle coordinate system of the own vehicle to obtain the travelable area to be inferred, The reasoning module is specifically used for:

Convert the coordinates of the location point to be inferred according to the third conversion formula to obtain the driveable area to be inferred;

Wherein, the third conversion formula is:
Wherein, (x dw , y dw ) is the coordinates of any position point D to be inferred in the world coordinate system, (x dv , y dv ) is the position point D to be inferred in the The coordinates in the vehicle coordinate system of the own vehicle, the
Is the second conversion matrix,

The second conversion matrix
The (x 0 , y 0 ) is the coordinates of the own vehicle in the world coordinate system at the current moment, and the θ 0 is the heading angle of the own vehicle at the current moment.
The device according to claim 23 or 24, wherein the calculation of the drivability value of each grid based on the drivable position point information in each grid in the second inference grid map On the one hand, the reasoning module is specifically used for:

Calculate the drivability value at different moments according to the drivable position point information in the i-th column and j-th row grid in the second inference grid map;

Perform a weighted summation of the drivability values at different moments to obtain the drivability value of the grid in the i-th column and j-th row;

Wherein, the drivability value of the grid in the i-th column and the j-th row is
Is the drivability value at time t, k t is the
The weight of the

Said
Is the number of travelable location points of surrounding vehicles traveling in the same direction obtained at time t in the grid of the i-th column and the j-th row, and
Is the number of travelable location points of surrounding vehicles in the reverse direction obtained at time t in the grid of the i-th column and the j-th row, the
Is the number of self-driving safe position points acquired at time t in the grid of the i-th column and the j-th row, the
Is the number of dangerous location points of the self-driving vehicle obtained at time t in the grid of the i-th column and the j-th row, the
Is the number of travelable location points of surrounding vehicles traveling in the same direction acquired at time t in the j-th grid, the
Is the number of travelable location points of surrounding vehicles in the reverse direction obtained at time t in the j-th grid, the
Is the number of safe driving position points obtained at time t in the j-th grid, the
Is the number of dangerous location points of the self-driving vehicle acquired at time t in the j-th grid.
The device according to any one of claims 16-25, wherein the drivable position point comprises a self-driving position point, and the acquisition module is further configured to:

Before inferring the fourth area based on the driving position point information, acquiring the driving position point of the own vehicle, where the driving position point information of the own vehicle includes safe driving position points and driving risk position points;

Wherein, the acquiring the driving position point of the self-vehicle includes:

Determine whether the driving mode of the own vehicle at its current position is manual driving mode; if the driving mode of the own vehicle at its current position is manual driving mode, determine that the current position of the own vehicle is the safe driving position point ；

If the driving mode of the self-vehicle at its current position is the automatic driving mode, determine whether the self-vehicle has a risk of collision or abnormal driving behavior at its current position; if it is determined that the self-vehicle has no risk of collision at its current position If there is no abnormal driving behavior, the current position of the own vehicle is determined to be the safe driving position; if it is determined that the own vehicle has a collision risk or abnormal driving behavior at its current position, then the current position of the own vehicle is determined The location point is a dangerous location point for driving.
The device according to claim 26, wherein, in terms of determining whether the own vehicle has a collision risk at its current position, the acquisition module is specifically configured to:

Obtain the angle θ between the driving direction of the own vehicle and the vehicle E; the vehicle E is the surrounding vehicles of the own vehicle;

If the included angle θ is greater than the second preset angle, adopt an intersection mode risk judgment method to determine whether the self-vehicle has a collision risk at its current position;

If the included angle θ is not greater than the second preset angle, a rear-end collision mode risk determination method is adopted to determine whether the own vehicle has a collision risk at its current position.
The device according to claim 26 or 27, wherein the drivable location points include drivable location points of surrounding vehicles, and the acquisition module is further configured to:

Acquire the drivable position point information of the surrounding vehicles, where the drivable position point information of the surrounding vehicles includes the coordinates of the driving position point in the same direction and the coordinates of the driving position point in the reverse direction,

In terms of obtaining information about the driving position points of surrounding vehicles, the obtaining module is further used for:

Acquire driving information of any vehicle A in the surrounding vehicles and driving information of its own vehicle, wherein the driving information of the vehicle A includes relative position coordinates and longitudinal relative speed, and the driving information of the own vehicle includes absolute position coordinates, The absolute speed of the direction and the heading angle of the car;

Obtaining the coordinates of the driving position point of the vehicle A according to the absolute position coordinates of the own vehicle, the heading angle of the vehicle, and the relative position coordinates of the vehicle A;

The type of the drivable position point of the vehicle A is determined according to the longitudinal relative speed of the vehicle A and the absolute speed of the own vehicle; the type of the drivable position point coordinate of the vehicle A includes the reverse drivable position point coordinate or the same Point coordinates to the driving position;

Wherein, the relative position coordinates are coordinates in the vehicle coordinate system, and the drivable position point coordinates of the vehicle A are coordinates in the world coordinate system.
The device according to claim 28, characterized in that, in terms of obtaining the coordinates of the driving position point of the vehicle A based on the absolute position coordinates of the own vehicle, the heading angle of the vehicle, and the relative position coordinates of the vehicle A, The acquisition module is also used for:

Calculate the absolute position coordinates of the own vehicle, the heading angle of the vehicle, and the relative position coordinates of the vehicle A through a fourth conversion formula to obtain the absolute position point coordinates of the vehicle A;

Among them, the fourth conversion formula is:
(x Av , y Av ) are the relative position coordinates of the vehicle A, and (x Aw , y Aw ) are the coordinates of the driving position point of the vehicle A;

Third conversion matrix
The (x 0 , y 0 ) is the absolute position coordinate of the own vehicle at the current moment, and the θ 0 is the heading angle of the own vehicle at the current moment.
The device according to claim 28 or 29, wherein, in terms of determining the type of the vehicle A's drivable position point coordinates according to the longitudinal relative speed of the vehicle A and the absolute speed, the acquisition module Also used for:

Acquiring the longitudinal absolute speed of the vehicle A according to the longitudinal relative speed of the vehicle A and the absolute speed;

If the longitudinal absolute speed of the vehicle A is greater than the preset speed threshold, determining that the vehicle A can drive position point coordinates are the same direction can drive position point coordinates;

If the longitudinal absolute speed of the vehicle A is less than the preset speed threshold, it is determined that the vehicle A can drive position point coordinates as the reverse direction drive position point coordinates.