CN116279538A

CN116279538A - Visualization method and system for assisting vehicle driving

Info

Publication number: CN116279538A
Application number: CN202310170213.8A
Authority: CN
Inventors: 王宏
Original assignee: Shenzhen Binneng Electric Technology Co ltd
Current assignee: Shenzhen Binneng Electric Technology Co ltd
Priority date: 2023-02-27
Filing date: 2023-02-27
Publication date: 2023-06-23

Abstract

The invention provides a visualization method and a system for assisting vehicle driving; the method comprises the steps of acquiring first driving data of the vehicle and second driving data of other vehicles around the vehicle in real time; if the first driving data meets a first preset condition, generating a first visual scheme of the other vehicles related to the second driving data; if the driving data meet a second preset condition, generating a second visual scheme of the other vehicles related to the second driving data; and executing the first visualization scheme or the second visualization scheme. The method and the system can adopt different visual schemes based on the driving condition of the vehicle, so that a driver can intuitively obtain the driving condition of the surrounding vehicle from the displayed visual scheme, thereby making a reasonable driving decision and ensuring driving safety.

Description

Visualization method and system for assisting vehicle driving

Technical Field

The invention relates to the technical field of driving assistance, in particular to a visualization method, a system, electronic equipment and a computer storage medium for assisting vehicle driving.

Background

Driving safety is always an important concern in the driving field, and the driving safety is realized on the one hand based on the safe driving behavior of the driver and on the other hand, the driving behavior of the surrounding traffic participation objects is accurately mastered. Conventional approaches generally only provide threat decisions for other traffic participants associated with a vehicle travel plan, however, such approaches only passively assess the safety of the driver's decision-making behavior, and fail to assist the driver in making optimal driving decisions ahead of time, resulting in insufficient driving safety.

Disclosure of Invention

In order to at least solve the technical problems in the background art, the invention provides a visualization method, a system, electronic equipment and a computer storage medium for assisting driving of a vehicle.

A first aspect of the present invention provides a visualization method of assisting driving of a vehicle, comprising the steps of:

acquiring first driving data of the vehicle and second driving data of other vehicles around the vehicle in real time;

if the first driving data meets a first preset condition, generating a first visual scheme of the other vehicles related to the second driving data;

if the driving data meet a second preset condition, generating a second visual scheme of the other vehicles related to the second driving data;

executing the first visualization scheme or the second visualization scheme;

the first preset condition is related to maintenance of the driving trend of the vehicle, and the second preset condition is related to change of the driving trend of the vehicle.

Further, it is judged that the first driving data satisfies a first preset condition or a second preset condition by:

extracting running plan data and real-time operation data of the vehicle from the first driving data;

carrying out initial prediction on driving trend change in a first adjacent period according to the driving plan data, and calculating the association degree of the real-time operation data and the driving plan data;

and judging that the first driving data meets the first preset condition or the second preset condition according to the initial prediction and the association degree.

Further, the method further comprises:

if the first driving data meets the first preset condition, continuously judging whether the first driving data meets the second preset condition or not in a second adjacent period;

wherein the second immediately adjacent period is included in the first immediately adjacent period.

Further, the range of the second immediately adjacent period is determined by:

calculating residual duration related to driving trend change according to the first driving data, and determining a first range of the second adjacent time period according to the residual duration;

and when the initial prediction is that a second preset condition is met, analyzing whether the association degree of the real-time operation data of the vehicle and the driving plan data before a second adjacent period is higher than an association threshold value, and if so, expanding the first range of the second adjacent period into a second range.

Further, the generating a first visualization of the other vehicle associated with the second driving data includes:

calculating a first movement trend of each of the other vehicles within a first range from the second driving data, the movement trend being described using a first icon with a direction indication;

wherein the first icon with direction indication moves in association with each of the other vehicles.

Further, the generating a second visualization of the other vehicle associated with the second driving data includes:

calculating a second motion trend and an expected track of each other vehicle in a second range according to the second driving data;

the second movement trend is described using a second icon with a directional indication and a different width characteristic, and the predicted trajectory is described using a third icon of variable length.

Further, the first range is of a lower size than the second range.

The second aspect of the invention provides a visualization system for assisting vehicle driving, which comprises an acquisition module, a processing module, a storage module and a visualization module; the processing module is electrically connected with the acquisition module, the storage module and the visualization module;

the memory module is used for storing executable computer program codes;

the acquisition module is used for acquiring first driving data of the vehicle and second driving data of other vehicles around the vehicle and transmitting the first driving data and the second driving data to the processing module;

the processing module for executing the method according to any of the preceding claims by invoking the executable computer program code in the storage module to generate a visualization scheme;

the visualization module is used for executing the visualization scheme.

A third aspect of the present invention provides an electronic device comprising: a memory storing executable program code; a processor coupled to the memory; the processor invokes the executable program code stored in the memory to perform the method of any one of the preceding claims.

A fourth aspect of the invention provides a computer storage medium having stored thereon a computer program which, when executed by a processor, performs a method as claimed in any one of the preceding claims.

The invention has the beneficial effects that: different visual schemes can be adopted based on the driving condition of the vehicle, so that a driver can intuitively obtain the driving condition of surrounding vehicles from the displayed visual scheme, and reasonable driving decisions are made to ensure driving safety.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a visualization method for assisting vehicle driving in accordance with an embodiment of the present invention;

FIG. 2 is a schematic diagram of a visual system for assisting in driving a vehicle according to an embodiment of the present invention;

fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Referring to fig. 1, fig. 1 is a flow chart of a visualization method for assisting driving of a vehicle according to an embodiment of the invention. Referring to fig. 1, an embodiment of the present invention provides a visualization method for assisting driving of a vehicle, including the following steps:

executing the first visualization scheme or the second visualization scheme;

In this embodiment, driving data of the vehicle may be obtained in real time during the driving process of the vehicle, so as to analyze the next driving trend of the vehicle, and different visualization schemes may be adopted for maintaining or changing the current driving state of the vehicle. The driver can intuitively obtain the motion state, motion trend, threat level and the like of the surrounding vehicles from the displayed visual scheme, so that reasonable driving decisions are made to ensure driving safety.

The first driving data of the vehicle can be obtained through direct detection of various vehicle-mounted sensors, for example, a Hall sensor collects real-time vehicle speed, a positioning element collects vehicle longitude and latitude, a corner sensor collects steering wheel corner, a brake sensor detects vehicle brake data, an accelerator pedal sensor detects vehicle acceleration data and the like; the driving gesture of the vehicle on the road can be acquired through an on-vehicle optical sensor (such as an imaging device), and the driving gesture comprises a lane, an intra-lane offset, a head direction and the like; and the vehicle speed, longitude and latitude, running plan/running plan change, lane and other data can be acquired through a navigation system equipped at the vehicle-mounted/mobile terminal. The running data of other surrounding vehicles can be obtained by direct detection through a detection device equipped by the vehicle, such as radar equipped around the vehicle, cameras equipped in the vehicle and the like, and the running data detected by other vehicle self-detection and road side equipment can also be received through the internet of vehicles technology. Specific details are not described in detail.

In addition, the visual presentation may be implemented based on a dashboard screen of a driver's seat equipped in a vehicle, a front-row display screen, a HUD device, a screen of a mobile device communicatively interconnected with a driving computer, and the like, which is not limited to this aspect of the invention.

In this embodiment, due to the gradual popularization of the vehicle-mounted intelligent device and the increasingly powerful functions of the mobile device, the vehicle uses the navigation system at higher frequencies when driving, so that the estimated driving plan of the vehicle in the immediate vicinity of the current moment can be obtained conveniently, and then the situation that the driving trend possibly existing in the vehicle is changed can be judged preliminarily, for example, the vehicle may change lanes rightwards to follow the driving plan. Meanwhile, the actual operation data of the vehicle is analyzed by the driver, and when the degree of association with the running plan is high enough, for example, the driver triggers a right turn light, so that the driver can be judged to be actually preparing for following the running plan; in response to this, when it is initially predicted that the driving tendency does not need to be changed based on the travel plan, the real-time operation data does not match the change plan in the travel plan data (i.e., the driver does not actually follow the travel plan), or the like, it is determined that the first preset condition is satisfied, that is, the driving tendency is maintained. The method and the system accurately analyze the driving trend of the vehicle in the close period based on the initial prediction and the association degree, and are favorable for accurately generating a follow-up visualization scheme.

Further, the method further comprises:

In this embodiment, when the driver changes the driving trend in advance by following the driving plan, there are early and late operations, that is, some drivers may implement the operation corresponding to the driving trend change soon after knowing the lane change prompt of the navigation system, while other drivers implement the operation with relatively late operations based on the actual road condition, personal habits, and the like. Therefore, in a period of time after the first driving data of the vehicle is judged to meet the first preset condition, the invention also continuously judges whether the first driving data meets the second preset condition so as to cope with the actual situation.

Wherein the first immediately adjacent period and the second immediately adjacent period are both future periods with reference to the current time (since the data processing requires a certain period of time, the second immediately adjacent period does not strictly start from the current time), and the second immediately adjacent period is included in the first immediately adjacent period.

It should be noted that, the end time of the second immediately adjacent period should not exceed the allowable limit value related to the change node, in other words, the implementation of the above-mentioned continuous determination is limited only before the vehicle misses the last time of the driving trend change. After the miss, a prompt for keeping the current driving state, updating the driving plan and the like can be output to the driver so as to avoid driving danger caused by urgent change of the driver.

Further, the range of the second immediately adjacent period is determined by:

calculating residual duration related to a driving trend change node according to the first driving data, and determining a first range of the second adjacent time period according to the residual duration;

In this embodiment, when the driver changes the driving trend in advance by following the driving plan, multiple driving trend changes may be required to be thoroughly completed, for example, two separate lane changes are required to complete the driving on the split ramp, and there is a characterization of the conventional straight driving operation in the gap between the two changes (for example, the driver does not trigger the steering lamp again when the lane change condition is not satisfied), which results in that the real-time operation data of the driver is likely to be the unreacted driving trend change before the second lane change.

For the above situation, the present invention calculates the longest remaining duration of the driving trend change according to parameters such as the distance between the vehicle and the change node (for example, the intersection of the ramp and the main road), the vehicle speed, the estimated driving track (the straight track and the change track), and then removes a certain margin (at least including the last opportunity) to determine the first range of the second immediately adjacent period, for example, the longest remaining duration is 15s, and then the first range of the second immediately adjacent period is 12s in the future from the current moment. Meanwhile, if the own vehicle performs the operation matching the driving tendency change related to the change node before the present moment is analyzed, it can be determined that the operation before the own vehicle is a part of the driving tendency change for following the traveling plan, and it is obvious that the driving operation with the driving tendency change will still be performed with a high probability in the subsequent period, and accordingly, the second immediately adjacent period is appropriately enlarged to the second range.

In the present invention, the expansion of the range of the second immediately adjacent period refers to the extension from the current time to the direction of the time corresponding to the change node.

In this embodiment, when it is determined that the host vehicle does not have a driving trend change, only the motion directions included in the motion trends of the surrounding vehicles need to be dynamically visualized in real time, so that a driver can intuitively know whether the surrounding vehicles have threats such as approaching the host vehicle and changing lanes or have actions affecting the original driving trend of the host vehicle and even driving safety, and further can quickly take relevant countermeasures.

In this embodiment, when it is determined that there is a change in the driving trend of the host vehicle, the movement trend and the predicted track of the surrounding vehicles need to be displayed at the same time, where the icon displaying the movement trend may indicate two parameters, such as the aforementioned vehicle movement direction and vehicle acceleration (real-time value and predicted value), the icon may be in the form of an arrow (the first visualization scheme may also be an arrow icon), the arrow direction refers to the vehicle movement direction, and the thickness of the arrow and arrow tail refers to the magnitude of the acceleration. And, the estimated track of the surrounding vehicle may also be indicated using a color bar having a width substantially equivalent to the vehicle size, the length of the color bar may be set according to the confidence of the estimated track, for example, when the estimated track is derived from the navigation system of the surrounding vehicle, the confidence is high, the length of the color bar is set to be long, and the estimated track is predicted according to the traveling data of the surrounding vehicle, the confidence is low, and the length of the color bar is set to be short; and, the color band may be provided to be gradually changed in color from the vehicle end, for example, to be diluted, so that the driver can more intuitively understand the driving trend of the vehicle.

Further, the first range is of a lower size than the second range.

In this embodiment, when the host vehicle does not need to change the driving trend, the surrounding vehicles in the short range are more concerned, and when the host vehicle needs to change the driving trend, the real-time and predicted driving conditions of the vehicles in the larger surrounding range need to be concerned, so that the driver can make a decision on the timing of the driving trend change and how to implement the change. In addition, the second range may be determined according to the specific content of the driving trend change, for example, when the driving trend changes to the left lane, a certain area on the left side is taken as the second range, and when the driving trend changes to the right lane, a certain area on the right side is taken as the second range, which is not described in detail.

Referring to fig. 2, fig. 2 is a schematic structural diagram of a visual system for assisting driving of a vehicle according to an embodiment of the invention. As shown in fig. 2, a visualization system for assisting driving of a vehicle according to an embodiment of the present invention includes an acquisition module (101), a processing module (102), a storage module (103), and a visualization module (104); the processing module (102) is electrically connected with the acquisition module (101), the storage module (103) and the visualization module (104);

-said storage module (103) for storing executable computer program code;

the acquisition module (101) is used for transmitting the first driving data of the vehicle and the second driving data of other vehicles around the vehicle to the processing module (102);

-the processing module (102) for executing the method according to any of the preceding claims by invoking the executable computer program code in the storage module (103) to generate a visualization scheme;

the visualization module (104) is configured to execute the visualization scheme.

The specific function of a visualization system for assisting vehicle driving in this embodiment refers to the foregoing embodiment, and since the system in this embodiment adopts all the technical solutions of the foregoing embodiment, at least all the beneficial effects brought by the technical solutions of the foregoing embodiment are provided, and will not be described in detail herein.

Referring to fig. 3, fig. 3 is an electronic device according to an embodiment of the present invention, including: a memory storing executable program code; a processor coupled to the memory; the processor invokes the executable program code stored in the memory to perform the method as described in the previous embodiment.

The embodiment of the invention also discloses a computer storage medium, and a computer program is stored on the storage medium, and when the computer program is run by a processor, the computer program executes the method according to the previous embodiment.

The processor in the electronic device of the present invention may perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) or a computer program loaded from a memory into a Random Access Memory (RAM). In RAM, various programs and data required for operation can also be stored. The processor, ROM and RAM are connected to each other by a bus. An input/output (I/O) interface is also connected to the bus.

A plurality of components in an electronic device are connected to an I/O interface, comprising: an input unit such as a keyboard, a mouse, etc.; an output unit such as various types of displays, speakers, and the like; a storage unit such as a magnetic disk, an optical disk, or the like; and communication units such as network cards, modems, wireless communication transceivers, and the like. The communication unit allows the device to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

The processor may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of processors include, but are not limited to, central Processing Units (CPUs), graphics Processing Units (GPUs), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processors, controllers, microcontrollers, and the like. The processor performs the various methods and processes described above, such as coping with perceptual methods. For example, in some embodiments, the method of handling awareness may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as a memory. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device via the ROM and/or the communication unit. When the computer program is loaded into RAM and executed by a processor, one or more of the steps of the method of handling awareness described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the coping sense method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application specific integrated circuits (AS ICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A method of visualizing assistance in driving a vehicle, comprising the steps of:

executing the first visualization scheme or the second visualization scheme;

2. A method of visualizing a vehicle driving assistance as in claim 1, wherein: judging that the first driving data meets a first preset condition or a second preset condition by the following method:

3. A method of visualizing a vehicle driving assistance as in claim 2, wherein: the method further comprises the steps of:

4. A method of visualizing assisted vehicle driving according to claim 3, characterized in that: the range of the second immediately adjacent period is determined by:

5. A method of visualizing a vehicle driving assistance as in claim 1, wherein: the generating a first visualization of the other vehicle in relation to the second driving data comprises:

6. A method of visualizing assistance in driving a vehicle in accordance with claim 5, wherein: the generating a second visualization of the other vehicle associated with the second driving data includes:

7. A method of visualizing a vehicle driving assistance as in claim 6, wherein: the first range is of a lower size than the second range.

8. A visualization system for assisting vehicle driving, comprising an acquisition module (101), a processing module (102), a storage module (103) and a visualization module (104); the processing module (102) is electrically connected with the acquisition module (101), the storage module (103) and the visualization module (104);

-said storage module (103) for storing executable computer program code;

-said processing module (102) for executing the method according to any of claims 1-7 by invoking said executable computer program code in said storage module (103) to generate a visualization scheme;

9. An electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; the method is characterized in that: the processor invokes the executable program code stored in the memory to perform the method of any of claims 1-7.

10. A computer storage medium having a computer program stored thereon, characterized in that: the computer program, when executed by a processor, performs the method of any of claims 1-7.