CN116714669A - Vehicle turning control method and device based on binocular stereoscopic vision - Google Patents

Vehicle turning control method and device based on binocular stereoscopic vision Download PDF

Info

Publication number
CN116714669A
CN116714669A CN202310627877.2A CN202310627877A CN116714669A CN 116714669 A CN116714669 A CN 116714669A CN 202310627877 A CN202310627877 A CN 202310627877A CN 116714669 A CN116714669 A CN 116714669A
Authority
CN
China
Prior art keywords
point cloud
vehicle
cloud data
height threshold
filtering
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310627877.2A
Other languages
Chinese (zh)
Inventor
张芊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jianzhi Technology Co ltd
Original Assignee
Beijing Jianzhi Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jianzhi Technology Co ltd filed Critical Beijing Jianzhi Technology Co ltd
Priority to CN202310627877.2A priority Critical patent/CN116714669A/en
Publication of CN116714669A publication Critical patent/CN116714669A/en
Pending legal-status Critical Current

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B62—LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
    • B62D—MOTOR VEHICLES; TRAILERS
    • B62D6/00—Arrangements for automatically controlling steering depending on driving conditions sensed and responded to, e.g. control circuits
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/24—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view in front of the vehicle
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/31—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles providing stereoscopic vision
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Mechanical Engineering (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Transportation (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the invention provides a vehicle turning control method and device based on binocular stereoscopic vision, and relates to the technical field of vehicle auxiliary driving; comprising the following steps: when the vehicle makes a turn on an unstructured road; acquiring initial image data acquired by the binocular stereo vision sensor; performing three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data; filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space; and determining a turning path according to the drivable space. According to the embodiment of the invention, when the vehicle turns on the unstructured road, the unstructured road is subjected to three-dimensional reconstruction, so that the drivable space capable of accurately avoiding the obstacle can be determined in the unstructured road, and the turning path is determined, so that the vehicle can accurately drive into the target intersection.

Description

Vehicle turning control method and device based on binocular stereoscopic vision
Technical Field
The invention relates to the technical field of prefabricated vehicle auxiliary driving, in particular to a vehicle turning control method based on binocular stereoscopic vision, a vehicle turning control device based on binocular stereoscopic vision, a vehicle and a storage medium.
Background
Currently, for many intersections, no lane is divided for turns. The scheme of path planning by the road structured information is not applicable to unstructured roads. The road turning scene of the unstructured road needs to be determined in a semantic segmentation mode, the method is limited in that a monocular scheme cannot generate space point cloud, a laser radar cannot generate dense point cloud, and the driving path under the scene is difficult to accurately plan. It can be seen that the way the turning path is planned by identifying road structuring information (such as lane lines) and semantic segmentation divisions is not accurate.
Disclosure of Invention
In view of the above problems, embodiments of the present invention have been made to provide a binocular stereo vision-based vehicle turning control method, a binocular stereo vision-based vehicle turning control apparatus, a vehicle, and a storage medium that overcome or at least partially solve the above problems.
In a first aspect of the present invention, an embodiment of the present invention discloses a vehicle turning control method based on binocular stereo vision, a binocular stereo vision sensor is mounted on a vehicle, including:
when the vehicle makes a turn on an unstructured road; acquiring initial image data acquired by the binocular stereo vision sensor;
performing three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data;
filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space;
and determining a turning path according to the drivable space.
Optionally, the step of filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space includes:
taking a flat point cloud in the three-dimensional point cloud data as a reference plane;
and filtering the three-dimensional point cloud data based on the reference plane and in combination with the preset height threshold to obtain a drivable space.
Optionally, the preset height threshold comprises a positive height threshold; the step of filtering the three-dimensional point cloud data based on the reference plane and combined with the preset height threshold value to obtain a drivable space comprises the following steps:
filtering point clouds lower than the positive height threshold value in the three-dimensional point cloud data to be first target point cloud data based on the reference plane;
and determining the space formed by the cloud data of the first target point as the drivable space.
Optionally, the preset height threshold comprises a negative height threshold; the step of filtering the three-dimensional point cloud data based on the reference plane and combined with the preset height threshold value to obtain a drivable space comprises the following steps:
filtering point clouds higher than the negative height threshold value in the three-dimensional point cloud data to be second target point cloud data based on the reference plane;
and determining the space formed by the cloud data of the second target point as the drivable space.
Optionally, the step of determining the turning path according to the drivable space includes:
acquiring vehicle body size information;
and planning in the drivable space according to the car body size information, and generating the turning path.
Optionally, before the step of determining a turning path according to the drivable space, the method further comprises:
when the vehicle runs to a structured road, acquiring three primary color point cloud information of the road;
determining road structure information according to the road three-primary color point cloud information;
and filtering the drivable space by adopting the road structure information.
Optionally, the method further comprises:
and controlling the vehicle to perform turning running along the turning path.
In a second aspect of the present invention, an embodiment of the present invention discloses a vehicle turning control device based on binocular stereo vision, a binocular stereo vision sensor being mounted on a vehicle, the device comprising:
the first acquisition module is used for enabling the vehicle to travel to an unstructured road. Acquiring initial image data acquired by the binocular stereo vision sensor;
the three-dimensional matching module is used for carrying out three-dimensional matching on the initial image data to generate three-dimensional point cloud data;
the filtering module is used for filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space;
a planning module for determining a turning path according to the drivable space
In a third aspect of the present invention, the embodiment of the present invention further discloses a vehicle, including a processor, a memory, and a computer program stored on the memory and capable of running on the processor, the computer program implementing the steps of the binocular stereoscopic vision-based vehicle turning control method as described above when executed by the processor.
In a fourth aspect of the present invention, embodiments of the present invention also disclose a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the binocular stereoscopic vision-based vehicle turning control method as described above.
The embodiment of the invention has the following advantages:
the embodiment of the invention is characterized in that when the vehicle turns on an unstructured road; acquiring initial image data acquired by the binocular stereo vision sensor; performing three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data; filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space; and determining a turning path according to the drivable space. The dense point cloud is obtained through binocular stereoscopic vision, and three-dimensional reconstruction is carried out on the unstructured road, so that a drivable space capable of accurately avoiding an obstacle can be determined in the unstructured road, the structured road information is not relied on, and a turning path is determined, so that a vehicle can accurately drive into a target intersection.
Drawings
FIG. 1 is a flow chart of steps of an embodiment of a binocular stereoscopic vision-based vehicle turn control method of the present invention;
FIG. 2 is a flow chart of steps of another embodiment of a binocular stereoscopic-based vehicle turn control method of the present invention;
FIG. 3 is a schematic illustration of a high-level screening of an example of a binocular stereoscopic-based vehicle turn control method of the present invention;
FIG. 4 is a schematic diagram of a turning path determination for an example of a binocular stereoscopic vision-based vehicle turning control method of the present invention;
FIG. 5 is a block diagram of an embodiment of a binocular stereoscopic vision-based vehicle turn control apparatus of the present invention;
fig. 6 is a block diagram of an electronic device according to an embodiment of the present invention;
fig. 7 is a block diagram of a storage medium according to an embodiment of the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
Referring to fig. 1, there is shown a flow chart of steps of an embodiment of a binocular stereo vision-based vehicle turning control method of the present invention in which a binocular stereo vision sensor is mounted on a vehicle. The specific installation position can be determined according to the actual situation of the vehicle, and the embodiment of the invention is not limited to this. Such as a binocular stereo vision sensor mounted on the inside roof of the front windshield of the vehicle.
The vehicle turning control method based on binocular stereoscopic vision specifically comprises the following steps:
step 101, when the vehicle turns on an unstructured road; and acquiring initial image data acquired by the binocular stereo vision sensor.
In the embodiment of the invention, the unstructured road refers to a road without missing ground marks such as lane lines, lane guide lines and the like. Such as intersections without ground identification, etc. When the vehicle turns on the unstructured road, initial image data collected by the binocular stereo vision sensor aiming at the road can be obtained.
And 102, performing three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data.
After the initial image data acquired by the binocular stereo vision sensor is acquired, the initial image data can be subjected to three-dimensional stereo matching so as to reconstruct the unstructured road in three dimensions, and dense three-dimensional point cloud data are acquired. In the embodiment of the invention, the three-dimensional matching mode of the images is not particularly limited, such as a semi-global matching method, a deep learning method and the like, and a person skilled in the art can select a proper three-dimensional matching mode of the images according to actual conditions to perform three-dimensional matching on initial image data acquired by the binocular stereo vision sensor, so as to reconstruct a three-dimensional scene.
And step 103, filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space.
And filtering the three-dimensional point cloud data based on a preset height threshold value, and determining the drivable space of the vehicle based on the space information corresponding to the three-dimensional point cloud data. The drivable space is a space in which the vehicle can travel without any obstacle. The preset height threshold may be set by those skilled in the art according to requirements, and the embodiment of the present invention is not limited thereto.
And 104, determining a turning path according to the drivable space.
And determining a path which can be traveled by the vehicle according to the drivable space, and selecting an optimal route from the paths as a turning path.
The embodiment of the invention is characterized in that when the vehicle turns on an unstructured road; acquiring initial image data acquired by the binocular stereo vision sensor; performing three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data; filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space; and determining a turning path according to the drivable space. The dense point cloud is obtained through binocular stereoscopic vision, and three-dimensional reconstruction is carried out on the unstructured road, so that a drivable space capable of accurately avoiding an obstacle can be determined in the unstructured road, the structured road information is not relied on, and a turning path is determined, so that a vehicle can accurately drive into a target intersection.
Referring to fig. 2, there is shown a flow chart of steps of another embodiment of a binocular stereo vision based vehicle turning control method of the present invention in which binocular stereo vision sensors are mounted on a vehicle. The vehicle turning control method based on binocular stereoscopic vision specifically comprises the following steps:
step 201, when the vehicle turns on an unstructured road; and acquiring initial image data acquired by the binocular stereo vision sensor.
In the embodiment of the invention, when the vehicle is on an unstructured road, initial image data acquired by the binocular stereoscopic vision sensor is acquired in real time, and vehicle working condition data of the vehicle is acquired in real time.
And 202, performing three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data. The three-dimensional point cloud data may include point cloud data of a road surface, an obstacle, and the like of an unstructured road.
In addition, the initial image data is acquired by using a camera coordinate system of the binocular stereo vision sensor, so that the initial image data can be located in the same coordinate system as the vehicle to facilitate data processing, and the generated three-dimensional point cloud data can be mapped into the vehicle coordinate system of the vehicle.
Specifically, an offset matrix of the camera coordinate system of the binocular stereo vision sensor and the vehicle coordinate system may be calculated, and the three-dimensional point cloud data is mapped into the vehicle coordinate system based on the offset matrix.
And 203, filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space.
After the three-dimensional point cloud data are obtained, filtering can be performed based on the three-dimensional point cloud data and a preset height threshold value, point clouds with obvious height deviation between objects such as pits, road teeth, flower bed and the like and the ground on the point clouds are filtered, and the space corresponding to the rest point clouds is determined as a drivable space.
In an optional embodiment of the present invention, the step of filtering the three-dimensional point cloud data based on a preset height threshold to obtain a travelable space includes:
sub-step S2031, taking a flattened point cloud in the three-dimensional point cloud data as a reference plane;
in the embodiment of the invention, the flat point cloud in the three-dimensional point cloud data can be used as a reference plane, namely, the space corresponding to the flat point cloud in the three-dimensional point cloud data is used as a flat road surface, so that the obstacle can be identified.
Step S2032, filtering the three-dimensional point cloud data based on the reference plane and the preset height threshold, to obtain a drivable space.
And then, on the basis of the reference plane, combining a preset height threshold value, and filtering the space which does not meet the requirement to obtain the drivable space.
Specifically, the preset height threshold includes a positive height threshold; the step of filtering the three-dimensional point cloud data based on the reference plane and combined with the preset height threshold value to obtain a drivable space comprises the following steps: filtering point clouds lower than the positive height threshold value in the three-dimensional point cloud data to be first target point cloud data based on the reference plane; and determining the space formed by the cloud data of the first target point as the drivable space.
In the embodiment of the invention, the preset height threshold is specifically a positive height threshold, namely the height threshold is used for judging the space protruding out of the ground, and determining whether an obstacle blocking the running of the vehicle exists or not, so that the obstacle is filtered. The method is characterized in that a positive height threshold value is added on a reference plane based on the reference plane, and a space higher than the positive height threshold value is an obstacle for obstructing the running of a vehicle, such as a flower bed, road teeth and the like; and taking the point cloud lower than the positive height threshold value as first target point cloud data, and determining the space formed by the first target point cloud data as a drivable space.
Specifically, the preset height threshold includes a negative height threshold; the step of filtering the three-dimensional point cloud data based on the reference plane and combined with the preset height threshold value to obtain a drivable space comprises the following steps: filtering point clouds higher than the negative height threshold value in the three-dimensional point cloud data to be second target point cloud data based on the reference plane; and determining the space formed by the cloud data of the second target point as the drivable space.
In the embodiment of the invention, the preset height threshold is specifically a negative height threshold, namely the height threshold is used for judging the space lower than the ground, and determining whether an obstacle obstructing the running of the vehicle exists or not so as to filter the obstacle. Namely, based on a reference plane, a negative height threshold value is added on the reference plane, and a space below the negative height threshold value is an obstacle which hinders the running of the vehicle, such as a pit and the like; and taking the point cloud higher than the negative altitude threshold value as second target point cloud data, and determining the space formed by the second target point cloud data as a drivable space.
Furthermore, in an alternative embodiment of the present invention, when the vehicle travels to the structured road, the drivable space may be further determined in combination with the information of the structured road, so that the turning path may more conform to the driving specifications. Specifically, the method further comprises the following steps:
and step S1, acquiring three primary color point cloud information of a road when the vehicle runs to the structured road.
When the vehicle travels to the structured road, since the structured road has lane information, three primary color (RGB) point cloud information of the road can be acquired. The three-primary color point cloud information of the road can be acquired and generated through a three-primary color visual sensor on the vehicle. In addition, the binocular vision sensor may be a three primary color vision sensor.
And S2, determining road structure information according to the road three-primary-color point cloud information.
And determining specific road structure information of the road in a three-dimensional space according to the three-primary color point cloud information of the road.
And S3, filtering the drivable space by adopting the road structure information.
And then, based on the road structure information, determining a facing lane and the like, and further filtering the drivable space in a space which cannot be driven in the driving regulation, so that the drivable space can meet the actual driving regulation.
And 204, determining a turning path according to the drivable space.
After the drivable space is obtained, an optimal travel path may be determined as a turning path based on the drivable space.
In an alternative embodiment of the present invention, the step of determining the turning path according to the travelable space includes:
sub-step S2041, obtaining body size information;
in the embodiment of the invention, the vehicle body size information of the vehicle can be further acquired and fused into the path planning.
And a substep S2042 of planning in the drivable space according to the vehicle body size information, and generating the turning path.
The vehicle can be planned in the drivable space according to the vehicle body size information, so that the vehicle can safely pass through the drivable space, and the planned drivable spaces are connected to generate a turning path.
And step 205, controlling the vehicle to perform turning running along the turning path.
After the turning path is obtained, parameters such as the speed, the steering angle and the like of the vehicle are controlled, and the vehicle is controlled to turn along the turning path to travel to the target intersection.
The embodiment of the invention is characterized in that when the vehicle turns on an unstructured road; acquiring initial image data acquired by the binocular stereo vision sensor; performing three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data; filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space; determining a turning path according to the drivable space; and controlling the vehicle to perform turning running along the turning path. The dense point cloud is obtained through binocular stereoscopic vision, and three-dimensional reconstruction is carried out on the unstructured road, so that a drivable space capable of accurately avoiding an obstacle can be determined in the unstructured road, the structured road information is not relied on, and a turning path is determined, so that a vehicle can accurately drive into a target intersection.
In order that those skilled in the art may better understand the embodiments of the present invention, the following description of the embodiments of the present invention is provided by way of example:
1. the binocular stereo vision sensor is arranged in a vehicle, and the coordinate system of the binocular stereo vision sensor can be mapped into the coordinate system of the vehicle in a vehicle body calibration mode and the like.
2. When a vehicle travels to an unstructured intersection, a dense point cloud (three-dimensional point cloud data) of the precise intersection can be obtained by using a binocular stereo vision sensor. The height threshold value can be used to divide the area that can be travelled. As illustrated in fig. 3 below, the ground is a complete plane on the point cloud, and objects such as pits, curbs, flower beds, etc. have significant height deviations from the ground on the point cloud.
3. According to the divided drivable area, the driving path can be planned in the three-dimensional space of the point cloud by combining with the vehicle body size information, and the turning path is obtained, so that the obstacle is avoided, and the vehicle can accurately drive into the next adjacent intersection.
4. For intersections with structured roads such as lane lines, the two road intersections can be combined to obtain RGBD point cloud information, and a driving path conforming to traffic regulations is planned. As shown in the left-turn example of fig. 4, since the turn is defined as a large turn, when the turn path a, the turn path B, and the turn path C are obtained, the turn path a is determined as a traveling turn path.
5. And obtaining point cloud information in the space through a binocular stereoscopic vision camera to obtain a drivable space of the road junction to be turned.
6. The travelable space (e.g., double yellow line, etc.) is further determined by the structured information of the RGB (three primary colors) camera. The turning path of travel is spatially planned.
It should be noted that, for simplicity of description, the method embodiments are shown as a series of acts, but it should be understood by those skilled in the art that the embodiments are not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred embodiments, and that the acts are not necessarily required by the embodiments of the invention.
Referring to fig. 5, a block diagram of an embodiment of a vehicle turning control apparatus based on binocular stereoscopic vision according to the present invention is shown. In the embodiment of the invention, the binocular stereo vision sensor is mounted on a vehicle, and the vehicle turning control device based on binocular stereo vision specifically comprises the following modules:
a first acquisition module 501 is configured to, when the vehicle is traveling to an unstructured road. Acquiring initial image data acquired by the binocular stereo vision sensor;
the three-dimensional matching module 502 is configured to perform three-dimensional matching on the initial image data to generate three-dimensional point cloud data;
a filtering module 503, configured to filter the three-dimensional point cloud data based on a preset height threshold, so as to obtain a drivable space;
a planning module 504, configured to determine a turning path according to the drivable space.
In an alternative embodiment of the present invention, the filtering module 503 includes:
a reference sub-module, configured to use a flattened point cloud in the three-dimensional point cloud data as a reference plane;
and the filtering sub-module is used for filtering the three-dimensional point cloud data based on the reference plane and the preset height threshold value to obtain a drivable space.
In an alternative embodiment of the invention, the preset height threshold comprises a positive height threshold; the filtering submodule includes:
the first filtering unit is used for filtering point clouds lower than the positive height threshold value in the three-dimensional point cloud data to be first target point cloud data based on the reference plane;
and a drivable space determination first unit configured to determine a space composed of the first target point cloud data as the drivable space.
In an alternative embodiment of the invention, the preset height threshold comprises a negative height threshold; the filtering submodule includes:
a second filtering unit, configured to filter, based on the reference plane, a point cloud higher than the negative altitude threshold in the three-dimensional point cloud data as second target point cloud data;
and a drivable space determination first unit configured to determine a space composed of the second target point cloud data as the drivable space.
In an alternative embodiment of the present invention, the planning module 504 includes:
the size acquisition sub-module is used for acquiring the size information of the vehicle body;
and the planning sub-module is used for planning in the drivable space according to the car body size information and generating the turning path.
In an alternative embodiment of the invention, the apparatus further comprises:
the second acquisition module is used for acquiring three primary color point cloud information of the road when the vehicle runs to the structured road;
the road structure information determining module is used for determining road structure information according to the road three-primary-color point cloud information;
and the standard module is used for filtering the drivable space by adopting the road structure information.
In an alternative embodiment of the invention, the apparatus further comprises:
and the control module is used for controlling the vehicle to turn along the turning path.
The embodiment of the invention is characterized in that when the vehicle turns on an unstructured road; acquiring initial point cloud data acquired by the binocular stereo vision sensor; performing three-dimensional stereo matching on the initial point cloud data to generate three-dimensional point cloud data; filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space; and determining a turning path according to the drivable space. The dense point cloud is obtained through binocular stereoscopic vision, and three-dimensional reconstruction is carried out on the unstructured road, so that a drivable space capable of accurately avoiding an obstacle can be determined in the unstructured road, the structured road information is not relied on, and a turning path is determined, so that a vehicle can accurately drive into a target intersection.
For the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points.
Referring to fig. 6, an embodiment of the present invention also provides a vehicle including:
a processor 601 and a storage medium 602, said storage medium 602 storing a computer program executable by said processor 601, said processor 601 executing said computer program when the vehicle is running to perform a binocular stereoscopic vision based vehicle turning control method according to any one of the embodiments of the present invention. The vehicle turning control method based on binocular stereoscopic vision comprises the following steps:
when the vehicle makes a turn on an unstructured road; acquiring initial point cloud data acquired by the binocular stereo vision sensor;
performing three-dimensional stereo matching on the initial point cloud data to generate three-dimensional point cloud data;
filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space;
and determining a turning path according to the drivable space.
Optionally, the step of filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space includes:
taking a flat point cloud in the three-dimensional point cloud data as a reference plane;
and filtering the three-dimensional point cloud data based on the reference plane and in combination with the preset height threshold to obtain a drivable space.
Optionally, the preset height threshold comprises a positive height threshold; the step of filtering the three-dimensional point cloud data based on the reference plane and combined with the preset height threshold value to obtain a drivable space comprises the following steps:
filtering point clouds lower than the positive height threshold value in the three-dimensional point cloud data to be first target point cloud data based on the reference plane;
and determining the space formed by the cloud data of the first target point as the drivable space.
Optionally, the preset height threshold comprises a negative height threshold; the step of filtering the three-dimensional point cloud data based on the reference plane and combined with the preset height threshold value to obtain a drivable space comprises the following steps:
filtering point clouds higher than the negative height threshold value in the three-dimensional point cloud data to be second target point cloud data based on the reference plane;
and determining the space formed by the cloud data of the second target point as the drivable space.
Optionally, the step of determining the turning path according to the drivable space includes:
acquiring vehicle body size information;
and planning in the drivable space according to the car body size information, and generating the turning path.
Optionally, before the step of determining a turning path according to the drivable space, the method further comprises:
when the vehicle runs to a structured road, acquiring three primary color point cloud information of the road;
determining road structure information according to the road three-primary color point cloud information;
and filtering the drivable space by adopting the road structure information.
Optionally, the method further comprises:
and controlling the vehicle to perform turning running along the turning path.
The memory may include a random access memory (Random Access Memory, abbreviated as RAM) or a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
The embodiment of the invention is characterized in that when the vehicle turns on an unstructured road; acquiring initial image data acquired by the binocular stereo vision sensor; performing three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data; filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space; and determining a turning path according to the drivable space. The dense point cloud is obtained through binocular stereoscopic vision, and three-dimensional reconstruction is carried out on the unstructured road, so that a drivable space capable of accurately avoiding an obstacle can be determined in the unstructured road, the structured road information is not relied on, and a turning path is determined, so that a vehicle can accurately drive into a target intersection.
Referring to fig. 7, an embodiment of the present invention further provides a computer readable storage medium 701, where the storage medium 701 stores a computer program, and when the computer program is executed by a processor, the method for controlling turning of a vehicle based on binocular stereo vision according to any one of the embodiments of the present invention is performed. The vehicle turning control method based on binocular stereoscopic vision comprises the following steps:
when the vehicle makes a turn on an unstructured road; acquiring initial image data acquired by the binocular stereo vision sensor;
performing three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data;
filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space;
and determining a turning path according to the drivable space.
Optionally, the step of filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space includes:
taking a flat point cloud in the three-dimensional point cloud data as a reference plane;
and filtering the three-dimensional point cloud data based on the reference plane and in combination with the preset height threshold to obtain a drivable space.
Optionally, the preset height threshold comprises a positive height threshold; the step of filtering the three-dimensional point cloud data based on the reference plane and combined with the preset height threshold value to obtain a drivable space comprises the following steps:
filtering point clouds lower than the positive height threshold value in the three-dimensional point cloud data to be first target point cloud data based on the reference plane;
and determining the space formed by the cloud data of the first target point as the drivable space.
Optionally, the preset height threshold comprises a negative height threshold; the step of filtering the three-dimensional point cloud data based on the reference plane and combined with the preset height threshold value to obtain a drivable space comprises the following steps:
filtering point clouds higher than the negative height threshold value in the three-dimensional point cloud data to be second target point cloud data based on the reference plane;
and determining the space formed by the cloud data of the second target point as the drivable space.
Optionally, the step of determining the turning path according to the drivable space includes:
acquiring vehicle body size information;
and planning in the drivable space according to the car body size information, and generating the turning path.
Optionally, before the step of determining a turning path according to the drivable space, the method further comprises:
when the vehicle runs to a structured road, acquiring three primary color point cloud information of the road;
determining road structure information according to the road three-primary color point cloud information;
and filtering the drivable space by adopting the road structure information.
Optionally, the method further comprises:
and controlling the vehicle to perform turning running along the turning path.
The embodiment of the invention is characterized in that when the vehicle turns on an unstructured road; acquiring initial image data acquired by the binocular stereo vision sensor; performing three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data; filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space; and determining a turning path according to the drivable space. The dense point cloud is obtained through binocular stereoscopic vision, and three-dimensional reconstruction is carried out on the unstructured road, so that a drivable space capable of accurately avoiding an obstacle can be determined in the unstructured road, the structured road information is not relied on, and a turning path is determined, so that a vehicle can accurately drive into a target intersection.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal device comprising the element.
The invention provides a vehicle turning control method and device based on binocular stereoscopic vision, which are described in detail, wherein specific examples are applied to illustrate the principle and the implementation mode of the invention, and the description of the examples is only used for helping to understand the method and the core idea of the invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.

Claims (10)

1. A vehicle turning control method based on binocular stereo vision, a binocular stereo vision sensor is installed on a vehicle, characterized by comprising:
when the vehicle makes a turn on an unstructured road; acquiring initial image data acquired by the binocular stereo vision sensor;
performing three-dimensional stereo matching on the initial image data to generate three-dimensional point cloud data;
filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space;
and determining a turning path according to the drivable space.
2. The method of claim 1, wherein the step of filtering the three-dimensional point cloud data based on a preset height threshold to obtain a travelable space comprises:
taking a flat point cloud in the three-dimensional point cloud data as a reference plane;
and filtering the three-dimensional point cloud data based on the reference plane and in combination with the preset height threshold to obtain a drivable space.
3. The method of claim 2, wherein the preset height threshold comprises a positive height threshold; the step of filtering the three-dimensional point cloud data based on the reference plane and combined with the preset height threshold value to obtain a drivable space comprises the following steps:
filtering point clouds lower than the positive height threshold value in the three-dimensional point cloud data to be first target point cloud data based on the reference plane;
and determining the space formed by the cloud data of the first target point as the drivable space.
4. The method of claim 2, wherein the preset height threshold comprises a negative height threshold; the step of filtering the three-dimensional point cloud data based on the reference plane and combined with the preset height threshold value to obtain a drivable space comprises the following steps:
filtering point clouds higher than the negative height threshold value in the three-dimensional point cloud data to be second target point cloud data based on the reference plane;
and determining the space formed by the cloud data of the second target point as the drivable space.
5. The method of claim 1, wherein the step of determining a turning path from the travelable space comprises:
acquiring vehicle body size information;
and planning in the drivable space according to the car body size information, and generating the turning path.
6. The method of claim 1, wherein prior to the step of determining a turning path from the travelable space, the method further comprises:
when the vehicle runs to a structured road, acquiring three primary color point cloud information of the road;
determining road structure information according to the road three-primary color point cloud information;
and filtering the drivable space by adopting the road structure information.
7. The method according to claim 1, wherein the method further comprises:
and controlling the vehicle to perform turning running along the turning path.
8. A vehicle turning control device based on binocular stereo vision, a binocular stereo vision sensor being mounted on a vehicle, the device comprising:
the first acquisition module is used for enabling the vehicle to travel to an unstructured road. Acquiring initial image data acquired by the binocular stereo vision sensor;
the three-dimensional matching module is used for carrying out three-dimensional matching on the initial image data to generate three-dimensional point cloud data;
the filtering module is used for filtering the three-dimensional point cloud data based on a preset height threshold value to obtain a drivable space;
and the planning module is used for determining a turning path according to the drivable space.
9. A vehicle comprising a processor, a memory and a computer program stored on the memory and executable on the processor, the computer program when executed by the processor implementing the steps of the binocular stereoscopic vision-based vehicle turn control method of any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which when being executed by a processor, implements the steps of the binocular stereoscopic vision-based vehicle turning control method according to any one of claims 1 to 7.
CN202310627877.2A 2023-05-30 2023-05-30 Vehicle turning control method and device based on binocular stereoscopic vision Pending CN116714669A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310627877.2A CN116714669A (en) 2023-05-30 2023-05-30 Vehicle turning control method and device based on binocular stereoscopic vision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310627877.2A CN116714669A (en) 2023-05-30 2023-05-30 Vehicle turning control method and device based on binocular stereoscopic vision

Publications (1)

Publication Number Publication Date
CN116714669A true CN116714669A (en) 2023-09-08

Family

ID=87867107

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310627877.2A Pending CN116714669A (en) 2023-05-30 2023-05-30 Vehicle turning control method and device based on binocular stereoscopic vision

Country Status (1)

Country Link
CN (1) CN116714669A (en)

Similar Documents

Publication Publication Date Title
DE102020110458A1 (en) VEHICLE PATH FORECAST
KR20200101855A (en) Travelling track prediction method and device for vehicle
DE112020003897T5 (en) SYSTEMS AND METHODS FOR MONITORING LANE CONGESTION
US20180203453A1 (en) System and Method of Using Crowd-Sourced Driving Path Data in an Autonomous or Semi-Autonomous Driving System
DE102019201850A1 (en) Vehicle control system and control method
DE102016224604A1 (en) System and method for verifying map data for a vehicle
DE102016209966A1 (en) Autonomous device and vehicle control device
DE102014117751A1 (en) System and method for dynamically focusing vehicle sensors
DE112021002001T5 (en) NAVIGATING A VEHICLE USING AN ELECTRONIC HORIZON
DE112020002764T5 (en) SYSTEMS AND METHODS FOR VEHICLE NAVIGATION
CN111177288A (en) System for deriving autonomous vehicle enabled drivable maps
DE102019119162A1 (en) POSEN ESTIMATE
DE102021103149A1 (en) METHOD AND DEVICE FOR DETERMINING THE OPTIMAL CROSSING LANE IN AN ASSISTED DRIVING SYSTEM
DE102021211781A1 (en) VEHICLE OPERATION USING BEHAVIOR RULES CHECKS
DE102021118316A1 (en) MONOCULAR 3D OBJECT RECOGNITION FROM IMAGE SEMANTIC NETWORK
DE102022102189A1 (en) Multimodal segmentation mesh for improved semantic labeling in map generation
DE102021131489A1 (en) SEMANTIC ANNOTATION OF SENSOR DATA USING UNRELIABLE MAP ANNOTATION INPUTS
DE112021003371T5 (en) Vehicle position estimating device and driving position estimating method
DE102021132082A1 (en) END-TO-END SYSTEM TRAINING USING UNITED IMAGES
DE102022131625A1 (en) CURB-BASED FEATURE EXTRACTION FOR RADAR LOCALIZATION AND LANE DETECTION
DE102018211368A1 (en) A method of describing an environment of a vehicle through the topology of the busy road
DE102022105579A1 (en) GRAPH EXPLORATION FOR RULEBOOK TRAJECTORY GENERATION
CN116714669A (en) Vehicle turning control method and device based on binocular stereoscopic vision
DE102022102187A1 (en) HIGH-FIDELITY DATA-DRIVEN MULTIMODAL SIMULATION
DE102021114756A1 (en) Map-based breakpoint control

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination