WO2021213527A1

WO2021213527A1 - Parallel driving and automatic collision avoidance system for vehicle, storage medium, product and system

Info

Publication number: WO2021213527A1
Application number: PCT/CN2021/089746
Authority: WO
Inventors: 黄踔; 刘渊; 霍舒豪; 张德兆; 王肖; 李晓飞; 张放
Original assignee: 北京智行者科技有限公司
Priority date: 2020-04-24
Filing date: 2021-04-25
Publication date: 2021-10-28
Also published as: CN111497835B; CN111497835A

Abstract

A parallel driving and automatic collision avoidance system for a vehicle, a storage medium, a product and a system. A processor (1) of a vehicle acquires environment video data of the vehicle by means of a video acquisition unit (3) located outside the vehicle, and synthesizes the environment video data to obtain first environment video information. The first environment video information is sent to an external parallel driving background (5) by means of a communication module (11) in the processor. The parallel driving background receives and displays the first environment video information and sends a control instruction to the communication module at the same time. The processor analyzes the control instruction and sends same to a controller (2). The controller correspondingly controls the vehicle according to the instruction. The processor detects, by means of an obstacle detection unit (4), whether there is an obstacle in the vehicle running track. If yes and the distance to the obstacle is smaller than a safety distance, the processor generates a braking instruction to control and stop the vehicle. In this way, remote parallel driving and automatic collision avoidance in the parallel driving are achieved, and the safety of the vehicle is guaranteed.

Description

Parallel driving of vehicles and automatic collision avoidance systems, storage media, products and systems

Technical field

The invention relates to the field of unmanned vehicle driving, in particular to a vehicle parallel driving and automatic collision avoidance system, storage medium, product and system.

Background technique

At present, low-speed unmanned vehicles may have positioning loss or failure to avoid obstacles in some special circumstances. For safety reasons, the vehicle will stop on the spot. At this time, manual intervention is required to make the vehicle leave the area, which requires us to have the ability to remotely control the vehicle. At present, the safety of remote control is purely dependent on the operator's self-identification of risks. Due to the visual blind zone of remote video during remote control and the uncertain delay when video information and remote control commands are transmitted through the network, the probability of vehicle collision will increase. There are great security risks.

Summary of the invention

The purpose of the embodiments of the present invention is to provide a vehicle parallel driving and automatic collision avoidance system, storage medium, product and system to solve the problems existing in the prior art.

In order to achieve the above objective, the present invention provides a vehicle parallel driving and automatic collision avoidance system, the system includes: a processor, a video acquisition unit, an obstacle detection unit, and a controller;

The video acquisition unit is located outside the controlled vehicle and is used to acquire environmental video data of the vehicle;

The obstacle detection unit is located around the vehicle and is used to obtain environmental obstacle data of the vehicle;

The processor is located in the vehicle, is connected to the video acquisition unit and the obstacle detection unit respectively, and is configured to receive environmental video data sent by the video acquisition unit and environmental obstacles sent by the obstacle detection unit Data; the processor has a communication module;

The processor is further configured to perform synthesis processing on the environmental video data to obtain first environmental video information, and send the first environmental video information to an external parallel driving background through the communication module;

The parallel driving background is connected to the communication module via a network, and is configured to receive the first video information and send the received control instruction to the communication module;

The processor parses the control instruction received by the communication module, so that the controller for controlling the vehicle receives it, and performs corresponding control processing of the vehicle;

The processor is further configured to calculate according to the vehicle's own weight, braking torque, and current speed to obtain a first safety distance and a second safety distance;

The processor is further configured to receive and process the environmental obstacle data, and obtain the first obstacle distance and the second obstacle distance in the environmental obstacle data;

When the first obstacle distance is not empty and the first obstacle distance is less than the first safety distance, generating an execution braking instruction, and sending the execution braking instruction to the controller, Carry out the braking treatment of the vehicle; or

When the second obstacle distance is not empty and the second obstacle distance is less than the second safety distance, generating an execution braking instruction, and sending the execution braking instruction to the controller, Perform the vehicle braking process.

Preferably, the video acquisition unit includes 4 cameras, which are respectively arranged on the top of the vehicle, and are respectively used to measure the environmental video data of the front, rear, left, and right parts of the vehicle.

Preferably, the obstacle detection unit includes 12 ultrasonic probes, 4 ultrasonic probes are respectively provided at the front and rear of the vehicle, and 2 ultrasonic probes are respectively provided at the left and right parts of the vehicle.

Further preferably, the environmental obstacle data further includes the probe identification of the ultrasonic probe that detects the obstacle.

Preferably, the parallel driving background is also used to display the received first environmental video information.

Preferably, when the first obstacle distance is not empty and the first obstacle distance is less than the first safety distance, the processor does not parse the control instruction received by the communication module; or

When the second obstacle distance is not empty and the second obstacle distance is less than the second safety distance, the processor does not parse the control instruction received by the communication module.

Preferably, the control instruction includes a first system time stamp of the parallel driving background, and after receiving the control instruction, the processor compares the first system time stamp in the control instruction with the vehicle To compare the timestamps of the second system to obtain communication delay information;

When the communication delay information is greater than the preset instruction transmission cycle, the processor generates an execution braking instruction, and sends the execution braking instruction to the controller to perform the vehicle braking process, and at the same time The communication delay information is sent to the parallel driving background through the communication module.

Preferably, the processor sends the environmental obstacle data to the parallel driving background through the communication module;

The parallel driving background displays the received environmental obstacle data.

In order to achieve the above-mentioned object, the present invention provides a computer-readable storage medium, including a program or instruction, when the program or instruction runs on a computer, the parallel driving and automatic collision avoidance system of the vehicle as described above is realized.

In order to achieve the above objective, the present invention provides a computer program product containing instructions, when the computer program product runs on a computer, the computer realizes the parallel driving and automatic collision avoidance system for vehicles as described above.

In order to achieve the above objective, the present invention provides a chip system, including a processor, the processor is coupled with a memory, the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor Such as the realization of the aforementioned vehicle parallel driving and automatic collision avoidance system.

In order to achieve the above objective, the present invention provides a circuit system including a processing circuit configured to implement the parallel driving and automatic collision avoidance system of the vehicle when executed.

In order to achieve the above objective, the present invention provides a computer system, including a memory, and one or more processors communicatively connected with the memory;

The memory stores instructions that can be executed by the one or more processors, and the instructions are executed by the one or more processors, so that the one or more processors implement the vehicle parallel operation as described above. Driving and automatic collision avoidance system.

In order to achieve the above object, the present invention provides a mobile tool, a server, the server including a memory and one or more processors communicatively connected with the memory;

By applying the vehicle parallel driving and automatic collision avoidance system, storage medium, product and system provided by the present invention, the remote parallel driving of the vehicle can be realized. At the same time, the vehicle can be realized by judging obstacle information and network transmission delay information at the vehicle end. Automatic collision avoidance control. As a result, parallel driving and automatic collision avoidance of unmanned vehicles are realized, and the safety of vehicles in the remote parallel driving state is ensured.

Description of the drawings

FIG. 1 is a schematic diagram of a parallel driving and automatic collision avoidance system for vehicles according to an embodiment of the present invention;

2 is a schematic diagram of first video information provided by an embodiment of the present invention;

3 is a schematic diagram of obstacle detection when a vehicle is traveling straight according to an embodiment of the present invention;

4 is a schematic diagram of obstacle detection when a vehicle turns left according to an embodiment of the present invention;

Fig. 5 is a schematic diagram of obstacle detection when a vehicle turns right according to an embodiment of the present invention.

Detailed ways

The application will be further described in detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described here are only used to explain the related invention, but not to limit the invention. In addition, it should be noted that, for ease of description, only the parts related to the relevant invention are shown in the drawings.

It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other if there is no conflict. Hereinafter, the present application will be described in detail with reference to the drawings and in conjunction with the embodiments.

FIG. 1 is a schematic diagram of a parallel driving and automatic collision avoidance system for vehicles provided by an embodiment of the present invention, which is applied to low-speed unmanned sweeping vehicles. As shown in Figure 1, the parallel driving and automatic collision avoidance system for vehicles includes: a processor 1, a video acquisition unit 3, an obstacle detection unit 4, and a controller 2. The video acquisition unit 3 includes a camera, and the obstacle detection unit 4 includes an ultrasonic probe. The processor 1 can be an on-board computer, which can process and send vehicle and environmental data, and can also receive and analyze control commands and send them to the controller 2, and the controller 2 The work of the vehicle can be controlled according to instructions.

The video acquisition unit 3 is used to collect the environmental video data of the vehicle, including 4 wide-angle cameras, which are installed in four directions outside the vehicle, and are used to collect the environmental video of the front, rear, left and right parts of the vehicle. data.

The obstacle detection unit 4 includes a total of 12 ultrasonic probes, which are installed around the vehicle to detect obstacle information on the traveling track of the vehicle. Among them, the front part and the rear part of the vehicle are respectively equipped with 4 ultrasonic probes, and the left part and the right part of the vehicle are respectively equipped with 2 ultrasonic probes.

The processor 1 is located in the vehicle and is connected to the video acquisition unit 3 and the obstacle detection unit 4 respectively, and is used to receive the environmental video data sent by the video acquisition unit 3 and the environmental obstacle data sent by the obstacle detection unit 4.

A communication module 11 is embedded and installed in the processor 1, and the communication module 11 may be a 4g/5g communication module. The processor 1 communicates with the external parallel driving background 5 through the communication module 11, pushes environmental video data and environmental obstacle data, and receives control instructions sent by the parallel driving background 5 at the same time. Since the communication module 11 adopts 4g/5g network communication, in order to use the communication bandwidth more effectively, in this embodiment, the front camera adopts a resolution of 640*480, and the other three cameras adopt a resolution of 320*240.

After the processor 1 receives the environmental video data in the four directions of the vehicle from the video acquisition unit 3, it synthesizes the environmental video data in the four directions through the GStreamer (a video processing software) tool to obtain the first Environmental video information, as shown in FIG. 2, the first environmental video information includes vehicle front environmental video data 101, vehicle left environmental video data 102, vehicle rear environmental video data 103 and vehicle right environmental video data 104. The processor 1 sends the synthesized first environmental video information to the parallel driving background 5 through the communication module 11.

After receiving the first environment video information, the parallel driving background 5 displays it to the background operator, and the background operator remotely controls the vehicle through the console provided by the parallel driving background 5 according to the first environment video information displayed in real time. The parallel driving background 5 sends the control instruction to the communication module 11 after receiving the control instruction input by the operator through the console.

The transmission of the control command can be transmitted in the socket communication mode, and the data transmission protocol can be in the JSON format. The control commands specifically include: forward, reverse, turn left, turn right, accelerate, decelerate, stop, start sweeping, close sweeping, start parallel driving, and close parallel driving.

The processor 1 parses the control instructions received through the communication module 11, obtains a control code that can be used to control the vehicle, and sends it to the controller 2, and the controller 2 controls the vehicle correspondingly through the control code.

Since there will inevitably be a delay in the transmission of the control command, in order to ensure the safety of the vehicle during parallel driving, the transmission of the control command needs to be monitored. Specifically, the first system time is added to the control command The first system time stamp is the system time stamp of the parallel driving background 5 when the control command starts to be transmitted, and should be at least accurate to the millisecond level. Before the processor 1 receives the control instruction and analyzes it, the processor 1 obtains the second system timestamp, where the second system timestamp is the system timestamp of the vehicle when the control instruction is received, and should be at least accurate to the millisecond level. The communication delay information is obtained by comparing the second system time stamp and the first system time stamp. At the same time, the processor 1 has a preset instruction transmission cycle of 500ms, and the accepted instruction transmission delay shall not exceed 500ms. When the communication delay is greater than 500ms, the processor 1 generates an execution brake command and sends it to the controller 2. The controller 2 controls the vehicle to brake and stop according to the execution brake command, so that the vehicle remains stationary, and at the same time uploads the communication delay information until The communication delay information meets the requirements.

FIG. 3 is a schematic diagram of obstacle detection when a vehicle is traveling straight according to an embodiment of the present invention. As shown in FIG. 3, it includes: a vehicle straight trajectory area 301, a front left probe detection area 201, a front left probe detection area 202, and a front right probe detection area 203 , Right front probe detection area 204, first safety distance 401. Wherein, the first safety distance 401 is calculated in real time according to the weight of the vehicle, the braking torque of the vehicle, and the current speed of the vehicle.

The automatic collision avoidance of a vehicle does not need to use all the ultrasonic probes, but only needs to determine the ultrasonic probe detection data involved in the vehicle trajectory.

When the vehicle is traveling straight, the vehicle's trajectory is always within the detection range of the front left and front right ultrasonic probes of the vehicle. Therefore, it is only necessary to determine the obstacle detection data of the front left and front right ultrasonic probes in the vehicle trajectory. The first obstacle distance is obtained when the front left and front right ultrasonic probes detect obstacles in the driving track of the vehicle. When the first obstacle distance is less than the first safety distance 401, the processor 1 automatically starts the braking process. The braking process is specifically as follows: the processor 1 no longer parses the control instructions sent by the parallel driving background 5 received through the communication module 11, but The braking instruction is automatically generated and sent to the controller 2, and the controller 2 controls the vehicle to stop according to the braking instruction.

When the vehicle is moving backwards, it is only necessary to judge the obstacle detection data of the ultrasonic probes of the rear left and right rear of the vehicle in the vehicle's trajectory.

FIG. 4 is a schematic diagram of obstacle detection when a vehicle turns left according to an embodiment of the present invention. As shown in FIG. 4, it includes: vehicle left turn trajectory area 302, left front probe detection area 201, front left probe detection area 202, and front right probe detection Area 203, front right probe detection area 204, first safety distance 401 and second safety distance 402. Among them, the first safety distance 401 is calculated in real time according to the weight of the vehicle, the braking torque of the vehicle, and the current speed of the vehicle. Since the vehicle has a turning angle when turning, the second safety distance 402 is slightly smaller than the first safety distance 401. The specific calculation formula is: the second safety distance 402 is equal to the first safety distance 401 minus half the vehicle body width.

When the vehicle turns left, the vehicle's trajectory falls into the detection area of the three ultrasonic probes, front left, front left, and front right. Therefore, in addition to judging the first detection area of the vehicle’s front left and front right ultrasonic probes in the vehicle’s trajectory There are no obstacles in the safe distance 401, and it is also determined that there is no obstacle in the second safe distance 402 in the detection area of the ultrasonic probe on the left front of the vehicle in the vehicle driving track. When there is an obstacle in the first safety distance 401 or the obstacle in the second safety distance 402 in the vehicle travel trajectory, the processor 1 automatically starts the above-mentioned braking process to control the vehicle to stop.

FIG. 5 is a schematic diagram of obstacle detection when a vehicle turns right according to an embodiment of the present invention. As shown in FIG. 5, it includes: vehicle right turn trajectory area 303, left front probe detection area 201, front left probe detection area 202, and front right probe detection Area 203, front right probe detection area 204, first safety distance 401 and third safety distance 403. Among them, the first safety distance 401 is calculated in real time according to the weight of the vehicle, the braking torque of the vehicle, and the current speed of the vehicle. Since the vehicle has a turning angle when turning, the third safety distance 403 is slightly smaller than the first safety distance 401. The specific calculation formula is: the third safety distance 403 is equal to the first safety distance 401 minus half the vehicle body width.

When the vehicle turns right, the trajectory of the vehicle falls into the detection area of the three ultrasonic probes of the front left, front right, and front right. Therefore, in addition to judging the first detection area of the front left and front right ultrasonic probes in the vehicle trajectory There are no obstacles in the safe distance 401, and it is also determined that there is no obstacle in the third safe distance 403 in the detection area of the ultrasonic probe on the right front of the vehicle in the vehicle trajectory. When there is an obstacle in the first safety distance 401 or the obstacle in the third safety distance 403 in the driving track of the vehicle, the processor 1 automatically starts the above braking process to control the vehicle to stop.

After the processor 1 of the vehicle detects the obstacle and controls the vehicle to park, the processor 1 sends the environmental obstacle data to the parallel driving background 5 through the communication module 11, and the parallel driving background 5 displays the environmental obstacle data to the background operator. It is convenient for remote obstacle avoidance operations. Among them, the environmental obstacle data also includes the probe identification of the ultrasonic probe that detects the obstacle, which is used to tell the operator the approximate position of the obstacle relative to the vehicle, so as to facilitate the operator to perform obstacle avoidance operations.

The vehicle parallel driving and automatic collision avoidance system provided by the embodiment of the present invention collects video data of the surrounding environment of the vehicle through the video acquisition unit 3 installed on the vehicle, and obtains the surrounding environment video of the vehicle through the communication module 11 on the processor 1 of the vehicle. The data is sent to the external parallel driving background 5, displayed to the background operator, and the control command input by the operator is sent to the processor 1 of the vehicle to control the vehicle and realize the remote parallel driving of the vehicle. At the same time, the obstacle detection unit 4 installed around the vehicle detects in real time whether there are obstacles in the trajectory of the vehicle and the distance between the obstacles and the vehicle. When an obstacle occurs within the safe distance of the vehicle, the processor 1 of the vehicle no longer executes the parallel operation. The control instructions sent by the driving backstage 5 generate and execute braking instructions, control the parking of the vehicle, realize automatic collision avoidance of the vehicle, and send obstacle information to the parallel driving backstage 5 for display to the operator to facilitate the operator to perform obstacle avoidance operations.

The vehicle parallel driving and automatic collision avoidance system provided by the embodiments of the present invention can realize the remote parallel driving of the vehicle when the vehicle is unable to drive autonomously. At the same time, the vehicle can be realized by judging obstacle information and network transmission delay information at the vehicle end. Automatic collision avoidance control. As a result, parallel driving and automatic collision avoidance of unmanned vehicles and vehicle safety in remote parallel driving conditions are realized.

Professionals should be further aware that the units and algorithm steps of the examples described in the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two, in order to clearly illustrate the hardware and software Interchangeability, in the above description, the composition and steps of each example have been generally described in accordance with the function. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered as going beyond the scope of the present invention.

Another embodiment of the present invention also provides a computer-readable storage medium, including a program or instruction, when the program or instruction runs on a computer, the parallel driving and automatic collision avoidance system of the vehicle as described above is realized.

Another embodiment of the present invention also provides a computer program product containing instructions, when the computer program product runs on a computer, the computer realizes the parallel driving and automatic collision avoidance system for vehicles as described above.

Another embodiment of the present invention further provides a chip system, including a processor, the processor is coupled with a memory, the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, such as Realize the aforementioned vehicle parallel driving and automatic collision avoidance system.

Another embodiment of the present invention further provides a circuit system, the circuit system includes a processing circuit configured to implement the parallel driving and automatic collision avoidance system of the vehicle when executed.

Another embodiment of the present invention further provides a computer system, including a memory, and one or more processors communicatively connected with the memory;

Another embodiment of the present invention also provides a mobile tool and a server, the server including a memory and one or more processors communicatively connected with the memory;

The steps of the method or algorithm described in combination with the embodiments disclosed herein can be implemented by hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or all areas in the technical field. Any other known storage media.

The above specific embodiments further describe the purpose, technical solutions and beneficial effects of the present invention in further detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the protection scope of the present invention. Within the spirit and principle of the present invention, any modification, equivalent replacement, improvement, etc., shall be included in the protection scope of the present invention.

Claims

A vehicle parallel driving and automatic collision avoidance system, characterized in that the system includes: a processor, a video acquisition unit, an obstacle detection unit and a controller;

The video acquisition unit is located outside the controlled vehicle and is used to acquire environmental video data of the vehicle;

The obstacle detection unit is located around the vehicle and is used to obtain environmental obstacle data of the vehicle;

The processor is located in the vehicle, is connected to the video acquisition unit and the obstacle detection unit respectively, and is configured to receive environmental video data sent by the video acquisition unit and environmental obstacles sent by the obstacle detection unit Data; the processor has a communication module;

The processor is further configured to perform synthesis processing on the environmental video data to obtain first environmental video information, and send the first environmental video information to an external parallel driving background through the communication module;

The parallel driving background is connected to the communication module via a network, and is configured to receive the first video information and send the received control instruction to the communication module;

The processor parses the control instruction received by the communication module, so that the controller for controlling the vehicle receives it, and performs corresponding control processing of the vehicle;

The processor is further configured to calculate according to the vehicle's own weight, braking torque, and current speed to obtain a first safety distance and a second safety distance;

The processor is further configured to receive and process the environmental obstacle data, and obtain the first obstacle distance and the second obstacle distance in the environmental obstacle data;

When the first obstacle distance is not empty and the first obstacle distance is less than the first safety distance, generating an execution braking instruction, and sending the execution braking instruction to the controller, Carry out the braking treatment of the vehicle; or

When the second obstacle distance is not empty and the second obstacle distance is less than the second safety distance, generating an execution braking instruction, and sending the execution braking instruction to the controller, Perform the vehicle braking process.
The system according to claim 1, wherein the video acquisition unit includes 4 cameras, which are respectively arranged on the top of the vehicle, and are used to measure the front, rear, left, and right parts of the vehicle, respectively. Department’s environmental video data.
The system according to claim 1, wherein the obstacle detection unit includes 12 ultrasonic probes, four ultrasonic probes are respectively provided in the front and rear of the vehicle, and the left and right parts of the vehicle Two ultrasonic probes are installed in each section.
The system according to claim 3, wherein the environmental obstacle data further includes the probe identification of the ultrasonic probe that detects the obstacle.
The system according to claim 1, wherein the parallel driving background is also used to display the received first environmental video information.
The system according to claim 1, wherein when the first obstacle distance is not empty, and the first obstacle distance is less than the first safety distance, the processor does not parse the The control command received by the communication module; or

When the second obstacle distance is not empty and the second obstacle distance is less than the second safety distance, the processor does not parse the control instruction received by the communication module.
The system according to claim 1, wherein the control instruction includes a first system time stamp of the parallel driving background, and after receiving the control instruction, the processor adds the control instruction to Compare the first system timestamp of the vehicle with the second system timestamp of the vehicle to obtain communication delay information;

When the communication delay information is greater than the preset instruction transmission cycle, the processor generates an execution braking instruction, and sends the execution braking instruction to the controller to perform the vehicle braking process, and at the same time The communication delay information is sent to the parallel driving background through the communication module.
The system according to claim 1, wherein the processor sends the environmental obstacle data to the parallel driving background through the communication module;

The parallel driving background displays the received environmental obstacle data.
A computer-readable storage medium, characterized by comprising a program or instruction, when the program or instruction runs on a computer, the system according to any one of claims 1-8 is realized.
A computer program product containing instructions, characterized in that, when the computer program product runs on a computer, the computer realizes the system according to any one of claims 1-8.
A chip system, characterized in that it comprises a processor, the processor is coupled with a memory, the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, claims 1 to 8. The system of any one of them.
A circuit system, characterized in that the circuit system includes a processing circuit, and the processing circuit is configured to implement the system according to any one of claims 1 to 8 when executed.
A computer system, characterized by comprising a memory, and one or more processors communicatively connected with the memory;

The memory stores instructions that can be executed by the one or more processors, and the instructions are executed by the one or more processors, so that the one or more processors implement claims 1 to 8. The system of any one of them.
A mobile tool, characterized by a server, the server comprising a memory and one or more processors communicatively connected with the memory;

The memory stores instructions that can be executed by the one or more processors, and the instructions are executed by the one or more processors, so that the one or more processors implement claims 1 to 8. The system of any one of them.