CN117944705A

CN117944705A - Vehicle running information prompting method and system and vehicle

Info

Publication number: CN117944705A
Application number: CN202311699139.5A
Authority: CN
Inventors: 李子伦; 黄晓攀; 杨静; 苏建业
Original assignee: United Automotive Electronic Systems Co Ltd
Current assignee: United Automotive Electronic Systems Co Ltd
Priority date: 2023-12-11
Filing date: 2023-12-11
Publication date: 2024-04-30

Abstract

The invention provides a vehicle running information prompting method and system, and a vehicle, comprising the following steps: sensing a driving environment of the target vehicle by using a sensor; performing visual angle conversion on the perception result according to the visual angle of the driver, and marking an object with the shielded visual angle of the driver or an object with the invisible visual angle of the driver in the visual angle conversion result as a dangerous object by using the visual angle of the driver; and acquiring attribute information of the dangerous object through the sensing result, and outputting prompt information to a driver of the target vehicle based on the attribute information of the dangerous object in the running process of the target vehicle. In the running process of the automobile, the invention obtains the surrounding environment information of the automobile through the sensor, then judges whether the object needing to be noticed by the driver exists in the object outside the automobile in the current running scene according to the sensing result, and displays and broadcasts the dangerous object information through the equipment pre-configured by the automobile so as to prompt the driver of the surrounding situation of the automobile, thereby improving the safety of the driver for driving the automobile.

Description

Vehicle running information prompting method and system and vehicle

Technical Field

The invention relates to the technical field of vehicles, in particular to a vehicle running information prompting method and system and a vehicle.

Background

In the running process of an automobile, dangerous objects which are shielded by dangerous objects or objects which cannot be seen by drivers such as pedestrians which are difficult to identify in the night are frequently encountered, so that the dangerous objects occur in the running process. The dangerous objects mainly comprise objects which are not recognized by a driver but bring risk to the running of the vehicle or cause accidents, such as blocked traffic lights, blocked pedestrians, pedestrians invisible under the condition of poor illumination, and objects of blind areas.

Because the human driver is difficult to concentrate on the surrounding environment at any time or is limited by various factors, the surrounding environment cannot be seen, the invisible dangerous objects cannot be noticed at any time, and the sensor on the intelligent vehicle can monitor the surrounding environment at any time and effectively judge the position, the state and the like of the invisible dangerous objects or the shielding objects. However, at present, no early warning or emphasis action is independently made on the sensor information, so that a human driver cannot effectively recognize invisible dangerous objects, and a correct decision is difficult to make in the driving process, so that dangers are caused. In addition, in a human-based driving behavior, the existing intelligent vehicle may display the currently recognized state information on the visual central control screen, but no explicit or special prompt is provided for invisible dangerous objects, so that the driver may not intuitively recognize the dangerous information.

Disclosure of Invention

In view of the above drawbacks of the prior art, an object of the present invention is to provide a vehicle driving information prompting method and system, and a vehicle, for solving the technical problems in the prior art.

To achieve the above and other related objects, the present invention provides a vehicle running information prompting method, including the steps of:

sensing the running environment of the target vehicle by utilizing a sensor which is configured on the target vehicle in advance or in real time to obtain a corresponding sensing result;

Performing visual angle conversion on the perception result according to the visual angle of a driver on the target vehicle, and marking an object with the shielded visual angle of the driver or an object with the invisible visual angle of the driver in the visual angle conversion result as a dangerous object by using the visual angle of the driver; and

Acquiring attribute information of the dangerous object through the sensing result, and outputting prompt information to a driver of the target vehicle based on the attribute information of the dangerous object in the running process of the target vehicle; wherein the attribute information of the dangerous object includes: the type of the dangerous object, the volume of the dangerous object, the speed of the dangerous object, and the position information of the dangerous object relative to the target vehicle.

In an embodiment of the present invention, before performing the viewing angle conversion on the sensing result according to the viewing angle of the driver on the target vehicle, the method further includes:

Carrying out driving scene matching on the target vehicle based on the perception result, and determining whether a target object exists in the perception result in the current driving scene; wherein the target object includes an object that requires attention of a driver on the target vehicle;

if the perception result has a target object, converting the perception result into a visual angle according to the visual angle of a driver on the target vehicle;

And if the sensing result does not include the target object, continuing to sense the running environment of the target vehicle by using a sensor which is configured on the target vehicle in advance or in real time.

In an embodiment of the present invention, the process of performing the viewing angle conversion on the perception result according to the viewing angle of the driver on the target vehicle includes:

Acquiring a position visual angle of a sensor on the target vehicle, comparing the position visual angle of the sensor with a driver visual angle, and determining whether the position visual angle of the sensor is consistent with the driver visual angle;

If the position visual angle of the sensor is consistent with the visual angle of the driver, directly taking the sensing result as a visual angle conversion result;

If the position visual angle of the sensor is inconsistent with the driver visual angle, based on the position visual angle of the sensor and the driver visual angle, performing visual angle conversion on the perception result according to the driver visual angle on the target vehicle to obtain a corresponding visual angle conversion result.

In an embodiment of the present invention, after the object with the blocked viewing angle of the driver or the object with the invisible viewing angle of the driver in the viewing angle conversion result is marked as the dangerous object by the viewing angle of the driver, the method further includes:

Continuously sensing the position information of the dangerous object relative to the target vehicle by using a sensor which is configured on the target vehicle in advance or in real time, and determining whether the dangerous object exceeds a preset prompt range of the target vehicle or not based on the position information;

If the dangerous object exceeds the preset prompt range of the target vehicle, stopping sensing the dangerous object, and removing the dangerous object from the corresponding sensing result;

If the dangerous object does not exceed the preset prompt range of the target vehicle, the dangerous object is perceived by using a sensor which is configured in advance or in real time on the target vehicle continuously until the dangerous object is removed from the corresponding perception result.

In an embodiment of the present invention, if the dangerous object does not exceed the preset prompting range of the target vehicle, the process of continuously sensing the dangerous object by using the sensor configured in advance or in real time on the target vehicle includes:

Detecting whether the sensor senses the dangerous object at the current moment;

If the sensor senses the dangerous object at the current moment, determining whether the dangerous object exceeds a preset prompt range of the target vehicle or not based on the position information of the dangerous object relative to the target vehicle;

If the sensor does not sense the dangerous object at the current moment, determining whether the dangerous object exceeds a preset prompt range of the target vehicle based on the position information of the dangerous object relative to the target vehicle at the previous moment, the speed of the dangerous object and the running speed of the target vehicle.

In an embodiment of the present invention, if the current driving scene includes an illumination condition less than a preset illumination threshold, determining whether the perceived result has the target object in the current driving scene includes:

Taking a moving object with the relative position with the target vehicle smaller than or equal to a preset distance in the perception result as the target object; wherein the moving object includes: pedestrians, vehicles, and/or animals;

and/or taking a stationary object with a relative position with the target vehicle smaller than or equal to a preset distance in the sensing result as the target object; wherein the stationary object comprises: traffic signs, plants and/or buildings.

In an embodiment of the present invention, the process of outputting the prompt information to the driver of the target vehicle based on the attribute information of the dangerous object during the driving of the target vehicle includes:

Outputting sound prompt information to a driver of the target vehicle based on attribute information of the dangerous object in the running process of the target vehicle; and/or outputting and displaying prompt information to a driver of the target vehicle based on the attribute information of the dangerous object in the running process of the target vehicle;

Wherein, the display prompt message includes: prompt information generated based on a head-up display, prompt information generated based on an augmented reality head display and prompt information generated based on a vehicle-mounted display screen.

In an embodiment of the present invention, the sensor configured in advance or in real time on the target vehicle includes at least one of the following: laser radar, camera, millimeter wave radar, ultrasonic radar, inertial measurement unit, global navigation satellite system, infrared sensor.

The invention also provides a vehicle running information prompt system, which comprises:

the sensing module is used for sensing the running environment of the target vehicle by utilizing a sensor which is configured on the target vehicle in advance or in real time to obtain a corresponding sensing result;

The visual angle conversion module is used for carrying out visual angle conversion on the perception result according to the visual angle of the driver on the target vehicle, and marking an object with the blocked visual angle of the driver or an object with the invisible visual angle of the driver in the visual angle conversion result as a dangerous object;

The information prompt module is used for acquiring attribute information of the dangerous object through the sensing result and outputting prompt information to a driver of the target vehicle based on the attribute information of the dangerous object in the running process of the target vehicle; wherein the attribute information of the dangerous object includes: the type of the dangerous object, the volume of the dangerous object, the speed of the dangerous object, and the position information of the dangerous object relative to the target vehicle.

The invention also provides a vehicle, which comprises a sensor and a processor; wherein,

The sensor is used for sensing the running environment of the vehicle to obtain a corresponding sensing result;

The processor is used for converting the perception result according to the visual angle of a driver on the vehicle, and marking an object with the blocked visual angle of the driver or an object with the invisible visual angle of the driver in the visual angle conversion result as a dangerous object; acquiring attribute information of the dangerous object through the sensing result, and outputting prompt information to a driver of the vehicle based on the attribute information of the dangerous object in the running process of the vehicle; wherein the attribute information of the dangerous object includes: the type of the dangerous object, the volume of the dangerous object, the speed of the dangerous object, and the position information of the dangerous object relative to the vehicle.

As described above, the present invention provides a vehicle running information prompting method and system, a vehicle, and has the following beneficial effects: the invention senses the running environment of the target vehicle by utilizing the sensor which is configured on the target vehicle in advance or in real time to obtain a corresponding sensing result; then, carrying out visual angle conversion on the perception result according to the visual angle of a driver on the target vehicle, and marking an object with the shielded visual angle of the driver or an object with the invisible visual angle of the driver in the visual angle conversion result as a dangerous object by using the visual angle of the driver; and acquiring attribute information of the dangerous object through the sensing result, and outputting prompt information to a driver of the target vehicle based on the attribute information of the dangerous object in the running process of the target vehicle. Therefore, in the running process of the automobile, the sensor senses and acquires the surrounding environment information of the automobile for a period of time, then judges whether objects needing attention of a driver exist on the outside of the automobile in the current running scene according to the sensing result, judges whether the objects are invisible dangerous objects such as shielded or invisible to human eyes according to the visual angle of the driver, simultaneously sends the attribute corresponding to the invisible dangerous objects and the tag to the automobile display system, and carries out enhanced display and broadcasting on the information of the dangerous objects through the pre-configured equipment of the automobile so as to prompt the driver of the surrounding situation of the automobile, thereby improving the safety of the driver driving the automobile. Therefore, the invention can pre-warn or emphasize the driver aiming at invisible dangerous objects according to the sensing result of the sensor on the vehicle, reduce the risk of the driver caused by the unfocused driving environment or energy and the like in the driving process of the vehicle, and reduce the probability of the direct contact or collision of the vehicle and the external object.

Drawings

FIG. 1 is a flowchart of a method for prompting vehicle driving information according to an embodiment of the present invention;

FIG. 2 is a flowchart of a method for prompting vehicle driving information according to another embodiment of the present invention;

FIG. 3 is a schematic hardware structure of a vehicle driving information prompt system according to an embodiment of the present invention;

fig. 4 is a schematic hardware structure of a vehicle running information prompting system according to another embodiment of the present invention;

fig. 5 is a schematic diagram of a hardware architecture of a vehicle suitable for implementing one or more embodiments of the invention.

Detailed Description

Other advantages and effects of the present invention will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present invention with reference to specific examples. The invention may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present invention. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict.

It should be noted that, the illustrations provided in the present embodiment merely illustrate the basic concept of the present invention by way of illustration, and only the components related to the present invention are shown in the drawings and are not drawn according to the number, shape and size of the components in actual implementation, and the form, number and proportion of the components in actual implementation may be arbitrarily changed, and the layout of the components may be more complex.

Fig. 1 is a schematic flow chart of a vehicle driving information prompting method according to an embodiment of the invention. Specifically, in an exemplary embodiment, as shown in fig. 1, the present embodiment provides a vehicle running information prompting method, which includes the steps of:

S110, sensing the running environment of the target vehicle by utilizing a sensor which is configured on the target vehicle in advance or in real time, and obtaining a corresponding sensing result. In this embodiment, the sensors configured in advance or in real time on the target vehicle may be set according to actual conditions, and the present embodiment does not limit the types and the number of the sensors. As an example, sensors configured in advance or in real time on the target vehicle include, but are not limited to, lidar, cameras, millimeter wave radar, ultrasonic radar, inertial measurement units, global navigation satellite systems, infrared sensors. In the present embodiment, the target vehicle includes, but is not limited to, an L0 class manual driving vehicle, an L1 class assisted driving vehicle, an L2 class partially automatic driving vehicle, an L3 class conditional automatic driving vehicle, an L4 class highly automatic driving vehicle, an L5 class fully automatic driving vehicle, and the like. The vehicles such as L0 level manual driving vehicle, L1 level auxiliary driving vehicle, L2 level partial automatic driving vehicle, L3 level conditional automatic driving vehicle, L4 level high automatic driving vehicle, L5 level full automatic driving vehicle and the like can be new energy vehicles or fuel oil vehicles.

S120, carrying out visual angle conversion on the perception result according to the visual angle of a driver on the target vehicle, and marking an object with the shielded visual angle of the driver or an object with the invisible visual angle of the driver in the visual angle conversion result as a dangerous object;

S130, acquiring attribute information of the dangerous object through a sensing result, and outputting prompt information to a driver of the target vehicle based on the attribute information of the dangerous object in the running process of the target vehicle; wherein the attribute information of the dangerous object includes, but is not limited to: the type of the dangerous object, the volume of the dangerous object, the speed of the dangerous object, and the position information of the dangerous object relative to the target vehicle. In this embodiment, dangerous objects may be set according to actual situations, and the type and number of the sensors are not limited in this embodiment. As an example, dangerous objects in the present embodiment include, but are not limited to: a traffic light that is blocked, a vehicle that is traveling toward the host vehicle, a pedestrian in the night, a vehicle front or vehicle side that is blocked by another object, a pedestrian that is traveling toward the host vehicle, a vehicle front or vehicle side that is blocked by another object, an animal that is traveling toward the host vehicle, a vehicle rear or vehicle side that is blocked by another object, a pedestrian that is traveling toward the host vehicle, a vehicle rear or vehicle side that is blocked by another object, a traffic sign that is blocked, a road surface pothole that is blocked, and the like.

Therefore, in the driving process of the automobile, the sensor senses and acquires the surrounding environment information of the automobile for a period of time, then judges whether objects needing attention of the driver exist in the objects outside the automobile in the current driving scene according to the sensing result, judges whether the objects are invisible dangerous objects such as shielded or invisible to human eyes according to the visual angle of the driver, and simultaneously sends the attribute corresponding to the invisible dangerous objects and the tag to the automobile display system, and the vehicle preconfigured equipment is used for carrying out enhanced display and broadcasting of the information of the dangerous objects so as to prompt the driver of the surrounding situation of the automobile, thereby improving the safety of the driver driving the automobile. Therefore, the embodiment can perform early warning or emphasis action on the driver aiming at invisible dangerous objects according to the sensing result of the sensor on the vehicle, reduce the risk of the driver caused by the driving environment or the lack of concentration in the driving process of the vehicle, and reduce the probability of the direct contact or collision of the vehicle and the external object.

In an exemplary embodiment of the present invention, before performing the viewing angle conversion on the perception result according to the viewing angle of the driver on the target vehicle, the embodiment may further include: performing driving scene matching on the target vehicle based on the sensing result, and determining whether a target object exists in the sensing result in the current driving scene; if the perception result has the target object, converting the perception result into a visual angle according to the visual angle of a driver on the target vehicle; if the sensing result does not include the target object, the driving environment of the target vehicle is sensed by continuously utilizing a sensor which is configured in advance or in real time on the target vehicle. Wherein the target object includes an object that requires attention of a driver on the target vehicle. As an example, in this embodiment, if the current driving scene includes an illumination condition less than a preset illumination threshold, the process of determining whether the perceived result has the target object in the current driving scene includes: taking a moving object with the relative position with the target vehicle smaller than or equal to a preset distance in the sensing result as a target object; wherein the moving object includes: pedestrians, vehicles, and/or animals; and/or taking a stationary object with the relative position with the target vehicle smaller than or equal to a preset distance in the sensing result as a target object; wherein the stationary object comprises: traffic signs, plants and/or buildings. As an example, the present embodiment may perform scene pre-matching with the database according to time sequence information of the sensing result or information transmitted by the interconnected vehicles and roads such as V2X (Vehicle to everything, vehicle wireless communication, V2X for short), and then determine whether there is an object that may need to be noticed by the driver in the current driving scene according to the matching result, so as to send out a reminder. If the object is matched with the object to be noticed by the driver and is currently shielded or invisible, the perception result is converted according to the visual angle of the driver on the target vehicle. In this embodiment, the object to be noticed by the driver is stored in the storage unit EMMC (Embedded Multi MEDIA CARD, EMMC for short) or Flash of the controller in a configuration file manner, and can be updated and updated so as to screen out the object to be continuously tracked and to be noticed by the driver.

In an exemplary embodiment of the present invention, a process of converting a perception result according to a driver's viewing angle on a target vehicle includes: acquiring a position visual angle of a sensor on a target vehicle, comparing the position visual angle of the sensor with a driver visual angle, and determining whether the position visual angle of the sensor is consistent with the driver visual angle; if the position visual angle of the sensor is consistent with the visual angle of the driver, directly taking the sensing result as a visual angle conversion result; if the position visual angle of the sensor is inconsistent with the driver visual angle, based on the position visual angle of the sensor and the driver visual angle, performing visual angle conversion on the perception result according to the driver visual angle on the target vehicle to obtain a corresponding visual angle conversion result. As an example, in the present embodiment, if the arrangement position of the sensor on the target vehicle is not at the same position as the driver position, it is necessary to convert all objects in the sensing result to the driver viewing angle according to the conversion matrix and the solid geometry calculation to determine whether the sensing result is a dangerous object that is blocked in the driver viewing angle or is invisible to the driver, and to give information on the position, size, speed, and the like of the determined dangerous object or invisible dangerous object.

In an exemplary embodiment of the present invention, after an object whose viewing angle is blocked by the driver or an object whose viewing angle is not visible by the driver in the viewing angle conversion result is recorded as a dangerous object with the driver viewing angle, the embodiment may further include: continuously sensing position information of the dangerous object relative to the target vehicle by using a sensor which is configured on the target vehicle in advance or in real time, and determining whether the dangerous object exceeds a preset prompting range of the target vehicle based on the position information; if the dangerous object exceeds the preset prompt range of the target vehicle, stopping sensing the dangerous object, and removing the dangerous object from the corresponding sensing result; if the dangerous object does not exceed the preset prompt range of the target vehicle, the dangerous object is sensed by the sensor which is configured in advance or in real time on the target vehicle continuously until the dangerous object is removed from the corresponding sensing result. The preset prompting range of the target vehicle can be calibrated and set in advance or in real time, and the embodiment is not described herein. In addition, in this embodiment, if the dangerous object does not exceed the preset prompting range of the target vehicle, the process of continuously sensing the dangerous object by using the sensor configured in advance or in real time on the target vehicle includes: detecting whether a sensor senses a dangerous object at the current moment; if the sensor senses a dangerous object at the current moment, determining whether the dangerous object exceeds a preset prompting range of the target vehicle or not based on the position information of the dangerous object relative to the target vehicle; if the sensor does not sense the dangerous object at the current moment, determining whether the dangerous object exceeds the preset prompt range of the target vehicle based on the position information of the dangerous object relative to the target vehicle at the previous moment, the speed of the dangerous object and the running speed of the target vehicle. In this or other embodiments, the hazardous object may also be referred to as an invisible hazardous object.

Specifically, in the present embodiment, after the dangerous object is identified, it is also necessary to continuously track the state of the perception identification target object, such as whether it is lost, whether it reaches the detection edge, or the like. Specifically, firstly judging whether the invisible dangerous object of the target reaches the edge of the range to be detected of the target vehicle, namely, whether the invisible dangerous object of the target reaches the preset prompting range of the target vehicle; if the detection range is close to the edge of the detection range, the judgment and the prompt of the target object are not performed in the subsequent processing. If the target object is still in the range to be detected, continuous tracking is needed, and the continuous tracking mode is as above. After the target is converted to the visual angle of the driver, judging that the dangerous object is invisible if the situation that the target is blocked by other target objects occurs; the driver and the vehicle sensor cannot sense the current moment, but the related information exists at the previous moment, and the dangerous object is judged to be invisible. A typical scene is a traffic light that is obscured by a large vehicle in front. The invisible dangerous object is firstly judged according to the information of a period of time before and after the current moment, and when the abnormal disappearance of the target object judged by the perception of a certain place (the perception of a vehicle or the information acquired by V2X) at a certain moment, the invisible dangerous object can be judged as the invisible dangerous object blocked at the current moment. Another way of discriminating is to directly discriminate by an algorithm such as deep learning, for example, a transducer, etc., to give a feature of whether the identified target is occluded or invisible to the human eye. In this embodiment, the invisible hazard includes, but is not limited to: a traffic light that is blocked, a vehicle that is traveling toward the host vehicle, a pedestrian in the night, a vehicle front or vehicle side that is blocked by another object, a pedestrian that is traveling toward the host vehicle, a vehicle front or vehicle side that is blocked by another object, an animal that is traveling toward the host vehicle, a vehicle rear or vehicle side that is blocked by another object, a pedestrian that is traveling toward the host vehicle, a vehicle rear or vehicle side that is blocked by another object, a traffic sign that is blocked, a road surface pothole that is blocked, and the like.

According to the above description, in an exemplary embodiment of the present invention, a process of outputting a prompt message to a driver of a target vehicle based on attribute information of a dangerous object during traveling of the target vehicle includes: outputting sound prompt information to a driver of the target vehicle based on attribute information of the dangerous object in the running process of the target vehicle; and/or outputting and displaying prompt information to a driver of the target vehicle based on attribute information of the dangerous object in the running process of the target vehicle; wherein, the display prompt message includes: prompt information generated based on a head-up display, prompt information generated based on an augmented reality head display and prompt information generated based on a vehicle-mounted display screen. Specifically, the present embodiment may prompt the driver of information of dangerous objects in various ways. For example, the driver can select whether to start the invisible object prompt function through interaction modes such as a vehicle-mounted central control display screen and the like, and the vehicle-mounted central control system judges whether to start the prompt function; if the prompting module is started, the prompting module is started; if the device is closed, only judgment is performed, and no prompt is performed. The prompt mode of the embodiment can be divided into a display prompt and a sound prompt. The Display prompt mainly comprises a prompt generated and output by a virtual space of a vehicle-mounted central control Display screen, a HUD (Head Up Display, HUD for short) and the like; the invisible objects can be displayed in the corresponding positions in a differentiated mode to prompt. In addition, since the technology of the car cabin is also rapidly developing, the technologies of HUD, AR-HUD (Augmented REALITY HEAD Up Display), etc. can also be applied to the Display prompt of the present embodiment. For example, when the shielded traffic light is rendered through the vehicle-mounted central control display screen, the HUD and the like, the traffic light is highlighted by thickening, reddening, flashing and the like to prompt the driver. The voice prompt prompts the driver of the type and the azimuth of the shielded object according to the judgment result of the shielded object, for example, the right front is shielded by a traffic light, so as to assist the driver in judging the surrounding environment and obtain more comprehensive information by combining display.

In another exemplary embodiment of the present invention, as shown in fig. 2, the embodiment further provides a vehicle driving information prompting method, including the steps of:

The driving environment of the target vehicle is perceived through the perception module, and information such as the category, the position, the size, the speed and the like of the perception target is provided. The sensing module in this embodiment mainly includes a bicycle intelligent sensing technology of peripheral devices such as a laser radar, a camera, a millimeter wave radar, an ultrasonic radar, a GNSS (Global Navigation SATELLITE SYSTEM, global navigation satellite system, abbreviated as GNSS)/IMU (Inertial Measurement Unit, inertial measurement unit, abbreviated as IMU), and the like, and uses deep learning or a transducer algorithm to sense and identify the surrounding environment, and also includes a technology of V2X interconnection communication. With the development of technologies such as a perception algorithm and V2X, the identification of invisible dangerous objects is more and more accurate at present, more scenes are derived from the identification, and the embodiment mainly utilizes the scenes derived from the shielding and invisible human eyes. For example, the BEV (Bird's Eye View) algorithm can more accurately identify the shielding object, the laser radar is not limited by the night, and the like, and more dangerous scenes which are invisible or not attended by human eyes can be seen, so that better performance can be provided, and better input is provided for a downstream algorithm.

Judging whether invisible dangerous objects needing to be noticed by a driver exist in the current driving scene according to time sequence information of a sensing result or information transmitted by interconnected vehicles, such as V2X and the like and roads. If invisible dangerous objects needing to be noticed by the driver exist, when the arrangement position of the sensor is inconsistent with the position of the driver, the sensing result is calculated according to the conversion matrix and the solid geometry, and the invisible dangerous objects needing to be noticed by the driver in the sensing result are converted into the view angle of the driver, so as to judge whether the sensing result is the dangerous objects which are shielded by the view angle of the driver or are invisible to the driver. And if the invisible dangerous objects which need to be noticed by the driver are not available, the driving environment of the target vehicle is perceived again through the perception module. If the sensor arrangement position is consistent with the driver position, the driver visual angle conversion is not performed, and whether dangerous objects which are blocked by the driver visual angle or are invisible to the driver are judged directly according to the perception result.

The state of perceptually identified invisible hazards, such as whether lost, reaching a detection edge, etc., is continuously tracked. Specifically, firstly, whether the invisible dangerous object of the target reaches the edge of the range to be detected is judged, and if the invisible dangerous object of the target approaches the edge of the range to be detected, the judgment and the prompt of the target are not performed in the subsequent processing. If the target invisible dangerous object is still in the range to be detected, continuous tracking is needed. Specifically, after the invisible dangerous object of the target is converted to the visual angle of the driver, judging the invisible dangerous object if the invisible dangerous object is blocked by other target objects; the driver and the vehicle sensor cannot sense the current moment, but the related information exists at the previous moment, and the dangerous object is judged to be invisible. A typical scene is a traffic light that is obscured by a large vehicle in front. The invisible dangerous object is firstly judged according to the information of a period of time before and after the current moment, and when the abnormal disappearance of the target object judged by the perception of a certain place (the perception of a vehicle or the information acquired by V2X) at a certain moment, the invisible dangerous object can be judged as the invisible dangerous object blocked at the current moment. Another way of discriminating is to directly discriminate by an algorithm such as deep learning, for example, a transducer, etc., to give a feature of whether the identified target is occluded or invisible to the human eye. In addition, whether the current illumination condition is lower than a preset illumination threshold value can be judged, and if the current illumination condition is lower than the preset illumination threshold value, pedestrians, vehicles and the like approaching the vehicle are judged to be invisible dangerous objects.

As an example, the invisible hazardous materials in the present embodiment are mainly shown in the following table 1, but the invisible hazardous materials that can be applied in the present embodiment are not limited to the following table 1. Vehicles may continuously update such information of invisible hazards through OTA (Over-the-Air Technology, OTA for short).

TABLE 1 invisible hazardous materials

The driver is prompted by various means that dangerous information is not visible. For example, the driver can select whether to start the invisible object prompt function through interaction modes such as a vehicle-mounted central control display screen and the like, and the vehicle-mounted central control system judges whether to start the prompt function; if the prompting module is started, the prompting module is started; if the device is closed, only judgment is performed, and no prompt is performed. The prompt mode of the embodiment can be divided into a display prompt and a sound prompt. The Display prompt mainly comprises a prompt generated and output by a virtual space of a vehicle-mounted central control Display screen, a HUD (Head Up Display, HUD for short) and the like; the invisible objects can be displayed in the corresponding positions in a differentiated mode to prompt. In addition, since the technology of the car cabin is also rapidly developing, the technologies of HUD, AR-HUD (Augmented REALITY HEAD Up Display), etc. can also be applied to the Display prompt of the present embodiment. For example, when the shielded traffic light is rendered through the vehicle-mounted central control display screen, the HUD and the like, the traffic light is highlighted by thickening, reddening, flashing and the like to prompt the driver. The voice prompt prompts the driver of the type and the azimuth of the shielded object according to the judgment result of the shielded object, for example, the right front is shielded by a traffic light, so as to assist the driver in judging the surrounding environment and obtain more comprehensive information by combining display.

Therefore, the embodiment can consider the differential prompt of the invisible dangerous objects, and the differential prompt mainly means that the invisible dangerous objects can be highlighted and prompted through the intelligent display mode in the cabin. Therefore, this embodiment can provide more accurate, kind is richer by invisible dangerous object information, after converting the visual angle to the driver visual angle, this kind of invisible dangerous object suggestion is more accurate, practical to the driver. The invisible dangerous object type warehouse can be updated through later stage; meanwhile, the existing cabin technology which is rapidly developed is combined, and invisible dangerous objects are highlighted by means of display, voice prompt and the like, so that a driver is prompted more effectively, and driving safety is improved.

In summary, the present invention provides a vehicle driving information prompting method, which senses a driving environment of a target vehicle by using a sensor configured in advance or in real time on the target vehicle to obtain a corresponding sensing result; then, carrying out visual angle conversion on the perception result according to the visual angle of a driver on the target vehicle, and marking an object with the shielded visual angle of the driver or an object with the invisible visual angle of the driver in the visual angle conversion result as a dangerous object by using the visual angle of the driver; and acquiring attribute information of the dangerous object through the sensing result, and outputting prompt information to a driver of the target vehicle based on the attribute information of the dangerous object in the running process of the target vehicle. Therefore, in the automobile driving process, the method is characterized in that the surrounding environment information of the automobile is obtained through the sensor in a sensing mode, whether objects needing to be noticed by a driver exist on the outside of the automobile in the current driving scene is judged according to the sensing result, whether the objects are invisible dangerous objects such as shielded or invisible to human eyes is judged according to the visual angle of the driver, meanwhile, the attribute corresponding to the invisible dangerous objects and the tag are sent to the automobile display system, and the information of the dangerous objects is displayed and broadcasted in an intensified mode through the equipment preset by the automobile, so that the surrounding situation of the automobile is prompted to the driver, and the safety of the automobile driving by the driver is improved. Therefore, the method can pre-warn or emphasize the driver according to the sensing result of the sensor on the vehicle to aim at invisible dangerous objects, reduce the risk of the driver due to the fact that the driving environment or energy is not concentrated in the driving process of the vehicle, and reduce the probability of direct contact or collision of the vehicle and external objects.

In an exemplary implementation of the present invention, as shown in fig. 3, this embodiment further provides a vehicle running information prompting system, including:

The sensing module 310 is configured to sense a driving environment of the target vehicle by using a sensor configured in advance or in real time on the target vehicle, so as to obtain a corresponding sensing result. In this embodiment, the sensors configured in advance or in real time on the target vehicle may be set according to actual conditions, and the present embodiment does not limit the types and the number of the sensors. As an example, sensors configured in advance or in real time on the target vehicle include, but are not limited to, lidar, cameras, millimeter wave radar, ultrasonic radar, inertial measurement units, global navigation satellite systems, infrared sensors. In the present embodiment, the target vehicle includes, but is not limited to, an L0 class manual driving vehicle, an L1 class assisted driving vehicle, an L2 class partially automatic driving vehicle, an L3 class conditional automatic driving vehicle, an L4 class highly automatic driving vehicle, an L5 class fully automatic driving vehicle, and the like. The vehicles such as L0 level manual driving vehicle, L1 level auxiliary driving vehicle, L2 level partial automatic driving vehicle, L3 level conditional automatic driving vehicle, L4 level high automatic driving vehicle, L5 level full automatic driving vehicle and the like can be new energy vehicles or fuel oil vehicles.

The visual angle conversion module 320 is configured to perform visual angle conversion on the perception result according to the visual angle of the driver on the target vehicle, and record an object with the blocked visual angle of the driver or an object with the invisible visual angle of the driver in the visual angle conversion result as a dangerous object;

The information prompt module 330 is configured to obtain attribute information of the dangerous object according to the sensing result, and output prompt information to a driver of the target vehicle based on the attribute information of the dangerous object during the driving process of the target vehicle; wherein, the attribute information of the dangerous object includes: the type of the dangerous object, the volume of the dangerous object, the speed of the dangerous object, and the position information of the dangerous object relative to the target vehicle. In this embodiment, dangerous objects may be set according to actual situations, and the type and number of the sensors are not limited in this embodiment. As an example, dangerous objects in the present embodiment include, but are not limited to: a traffic light that is blocked, a vehicle that is traveling toward the host vehicle, a pedestrian in the night, a vehicle front or vehicle side that is blocked by another object, a pedestrian that is traveling toward the host vehicle, a vehicle front or vehicle side that is blocked by another object, an animal that is traveling toward the host vehicle, a vehicle rear or vehicle side that is blocked by another object, a pedestrian that is traveling toward the host vehicle, a vehicle rear or vehicle side that is blocked by another object, a traffic sign that is blocked, a road surface pothole that is blocked, and the like.

It should be noted that, the vehicle running information prompting system provided in this embodiment and the vehicle running information prompting method provided in the foregoing embodiments belong to the same concept, and a specific manner in which each module in the vehicle running information prompting system performs an operation has been described in detail in the method embodiment shown in fig. 1, which is not repeated herein. In practical application, the vehicle running information prompting system provided in this embodiment may allocate the functions to different functional modules according to needs, that is, the internal structure of the system is divided into different functional modules, so as to complete all or part of the functions described above. For example, the sensing module 310 in this embodiment may be used to perform step S110 described in the above embodiment, the perspective conversion module 320 in this embodiment may be used to perform step S120 described in the above embodiment, and the information prompting module 330 in this embodiment may be used to perform step S130 described in the above embodiment.

In another exemplary embodiment of the present invention, as shown in fig. 4, the embodiment further provides a vehicle running information prompting system, including: the system comprises a sensing module, an invisible dangerous object judging module and an invisible dangerous object prompting module. Wherein,

The sensing module is used for sensing the running environment of the target vehicle and providing information such as the category, the position, the size, the speed and the like of the sensing target. The sensing module in this embodiment mainly includes technologies for intelligent sensing of a bicycle by using peripheral devices such as a laser radar, a camera, a millimeter wave radar, an ultrasonic radar, a GNSS/IMU, and the like, and uses deep learning or a transducer algorithm to sense and identify a surrounding environment, and also includes technologies for interconnecting communication such as V2X, and the like. With the development of technologies such as a perception algorithm and V2X, the identification of invisible dangerous objects is more and more accurate at present, more scenes are derived from the identification, and the embodiment mainly utilizes the scenes derived from the shielding and invisible human eyes. For example, BEV algorithms can more accurately identify occlusions, lidar is not limited by night, etc., can see more dangerous scenes that are not visible or noticeable to the human eye, and thus can provide better performance, providing better input to downstream algorithms.

The invisible dangerous object judging module is used for carrying out invisible dangerous object pretreatment, visual angle conversion and invisible dangerous object judgment on the sensing result. In particular, the method comprises the steps of,

The invisible dangerous object judging module mainly comprises the following parts:

a. the first part is an invisible dangerous object matching unit, and judges whether invisible dangerous objects which possibly need to be noticed by a driver exist or not according to time sequence information of a sensing result or information transmitted by the interconnected vehicles such as V2X and the like and the road so as to send out reminding. If the object is judged to be the blocked or invisible object, searching the locally stored object which needs to be noticed by the driver. If the invisible dangerous object of the target is matched as an object to be noticed by the driver and is shielded or invisible at present, the attribute information of the invisible dangerous object of the target is sent to the invisible dangerous object prompting module. The object to be noticed by the driver is stored in the memory unit EMMC or Flash of the controller in the form of a profile, and can be updated and updated. The module can screen out target invisible dangerous objects which need to be continuously tracked and need to be noted by a system.

B. The second part is a visual angle conversion unit, the sensor arrangement position and the driver position are not in the same position, and the perception result is required to be converted into the visual angle of the driver according to the conversion matrix and the solid geometry calculation, so as to judge whether the perception result is a dangerous object which is shielded by the visual angle of the driver or is invisible to the driver. And gives information such as the position, the size, the speed and the like of the invisible dangerous objects which are judged in advance.

C. The third part is an invisible dangerous object judging unit. The discrimination of invisible dangerous objects requires continuous tracking of the state of the perceived invisible dangerous objects, such as whether the dangerous objects are lost, whether the dangerous objects reach the detection edge, etc. The module is used for selecting information to be prompted to the driver from the information to display and broadcast to the driver, and mainly comprises the following steps c11-c12.

C11 Firstly judging whether the invisible dangerous object of the target reaches the edge of the range to be detected, and if the invisible dangerous object of the target is close to the edge of the range to be detected, judging and prompting the target object in the subsequent processing are not performed; the edges of the range to be detected can be calibrated and set in advance or in real time.

C12 If the target invisible dangerous object is still in the range to be detected, continuous tracking is required. Specifically, after the invisible dangerous object of the target is converted to the visual angle of the driver, judging the invisible dangerous object if the invisible dangerous object is blocked by other target objects; the driver and the vehicle sensor cannot sense the current moment, but the related information exists at the previous moment, and the dangerous object is judged to be invisible. A typical scene is a traffic light that is obscured by a large vehicle in front. The invisible dangerous object is firstly judged according to the information of a period of time before and after the current moment, and when the abnormal disappearance of the target object judged by the perception of a certain place (the perception of a vehicle or the information acquired by V2X) at a certain moment, the invisible dangerous object can be judged as the invisible dangerous object blocked at the current moment. Another way of discriminating is to directly discriminate by an algorithm such as deep learning, for example, a transducer, etc., to give a feature of whether the identified target is occluded or invisible to the human eye. In addition, whether the current illumination condition is lower than a preset illumination threshold value is judged, and if the current illumination condition is lower than the preset illumination threshold value, pedestrians, vehicles and the like approaching the vehicle are judged to be invisible dangerous objects.

The invisible dangerous objects in this embodiment are mainly shown in the following table 2, but the invisible dangerous objects that can be applied in this embodiment are not limited to the following table 2. The vehicle can continuously update information of such invisible dangerous objects through the OTA.

TABLE 2 invisible hazardous materials

The invisible dangerous object prompting module is used for receiving information of the invisible dangerous object judging module and prompting information of invisible dangerous objects of a driver in various modes by combining prompting modes. For example, the driver can select whether to start the invisible object prompt function through interaction modes such as a vehicle-mounted central control display screen and the like, and the vehicle-mounted central control system judges whether to start the prompt function; if the prompting module is started, the prompting module is started; if the device is closed, only judgment is performed, and no prompt is performed. The prompt mode of the embodiment can be divided into a display prompt and a sound prompt. The Display prompt mainly comprises a prompt generated and output by a virtual space of a vehicle-mounted central control Display screen, a HUD (Head Up Display, HUD for short) and the like; the invisible objects can be displayed in the corresponding positions in a differentiated mode to prompt. In addition, since the technology of the car cabin is also rapidly developing, the technologies of HUD, AR-HUD (Augmented REALITY HEAD Up Display), etc. can also be applied to the Display prompt of the present embodiment. For example, when the shielded traffic light is rendered through the vehicle-mounted central control display screen, the HUD and the like, the traffic light is highlighted by thickening, reddening, flashing and the like to prompt the driver. The voice prompt prompts the driver of the type and the azimuth of the shielded object according to the judgment result of the shielded object, for example, the right front is shielded by a traffic light, so as to assist the driver in judging the surrounding environment and obtain more comprehensive information by combining display.

In summary, the present invention provides a vehicle driving information prompting system, which senses a driving environment of a target vehicle by using a sensor configured in advance or in real time on the target vehicle to obtain a corresponding sensing result; then, carrying out visual angle conversion on the perception result according to the visual angle of a driver on the target vehicle, and marking an object with the shielded visual angle of the driver or an object with the invisible visual angle of the driver in the visual angle conversion result as a dangerous object by using the visual angle of the driver; and acquiring attribute information of the dangerous object through the sensing result, and outputting prompt information to a driver of the target vehicle based on the attribute information of the dangerous object in the running process of the target vehicle. Therefore, in the running process of the automobile, the system senses and acquires the surrounding environment information of the automobile for a period of time through the sensor, then judges whether objects needing attention of a driver exist on the outer side of the automobile in the current running scene according to the sensing result, judges whether the objects are invisible dangerous objects such as shielded or invisible to human eyes according to the visual angle of the driver, simultaneously sends the attribute corresponding to the invisible dangerous objects and the tag to the automobile display system, and carries out enhanced display and broadcasting on the information of the dangerous objects through the pre-configured equipment of the automobile so as to prompt the driver of the surrounding situation of the automobile, thereby improving the safety of the driver driving the automobile. Therefore, the system can pre-warn or emphasize the driver according to the sensing result of the sensor on the vehicle to aim at invisible dangerous objects, reduce the risk of the driver due to the fact that the driving environment or energy is not concentrated in the driving process of the vehicle, and reduce the probability of direct contact or collision of the vehicle and external objects.

In another exemplary embodiment of the present invention, as shown in fig. 5, the embodiment further provides a vehicle including a sensor and a processor; the sensor is used for sensing the running environment of the vehicle to obtain a corresponding sensing result; the processor is used for converting the visual angle of the perceived result according to the visual angle of a driver on the vehicle, and recording an object with the shielded visual angle of the driver or an object with the invisible visual angle of the driver in the visual angle conversion result as a dangerous object; acquiring attribute information of the dangerous object through a sensing result, and outputting prompt information to a driver of the vehicle based on the attribute information of the dangerous object in the running process of the vehicle; wherein, the attribute information of the dangerous object includes: the type of the dangerous object, the volume of the dangerous object, the speed of the dangerous object, and the position information of the dangerous object relative to the vehicle.

It should be noted that, the vehicle provided in this embodiment is the same concept as the vehicle running information prompting method and the vehicle running information prompting system provided in the foregoing embodiments, and a specific manner in which the vehicle running information prompting method and the vehicle running information prompting system execute operations has been described in detail in the foregoing embodiments, which is not repeated herein. In practical application, the vehicle provided in this embodiment may distribute the functions to different functional modules according to needs, that is, the internal structure of the vehicle is divided into different functional modules, so as to complete all or part of the functions described above. As an example, the sensor in the present embodiment may be used to perform step S110 described in the above embodiment and steps associated with before and after step S110, for example; the processor may be used to perform step S120 and steps associated with before and after step S120 described in the above embodiments, and the processor may be used to perform step S130 and steps associated with before and after step S130 described in the above embodiments. As another example, for example, the sensor in the present embodiment may be used to perform the sensing module 310 described in the above embodiment and the performing actions associated with the sensing module 310 in front and back; the processor may be used to perform the view angle conversion module 320 and the performing actions associated with the front and rear of the view angle conversion module 320 described in the above embodiments, and the processor may be used to perform the information presentation module 330 and the performing actions associated with the front and rear of the information presentation module 330 described in the above embodiments.

It should be noted that, in the above embodiment, when processing related data (for example, sensing data obtained by sensors such as a laser radar, a camera, a millimeter wave radar, an ultrasonic radar, an inertial measurement unit, a global navigation satellite system, an infrared sensor, etc.), for example, processing such as collection, storage, use, processing, transmission, provision, disclosure, deletion, etc., is performed with or without approval of a user. Sensing data of sensors such as laser radar, cameras, millimeter wave radar, ultrasonic radar, inertial measurement units, global navigation satellite systems, infrared sensors and the like are authorized to be obtained under the condition that users know and agree; either the user is actively provided after reading the relevant description, or the user is actively authorized/provided/uploaded, or otherwise obtained through or informed of the user's consent, while using some or all of the functions described in the above embodiments.

The above embodiments are merely illustrative of the principles of the present invention and its effectiveness, and are not intended to limit the invention. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the invention. Accordingly, it is intended that all equivalent modifications and variations of the invention be covered by the claims, which are within the ordinary skill of the art, be within the spirit and scope of the present disclosure.

Claims

1. A vehicle travel information presentation method, characterized by comprising the steps of:

2. The vehicle travel information presentation method according to claim 1, characterized in that before converting the perception result in accordance with the driver's angle of view on the target vehicle, the method further comprises:

3. The vehicle travel information presentation method according to claim 1 or 2, characterized in that the process of performing viewing angle conversion of the perception result according to the viewing angle of the driver on the target vehicle includes:

4. The vehicle travel information presentation method according to claim 3, wherein after an object whose driver viewing angle is blocked or an object whose driver viewing angle is not visible in the viewing angle conversion result is recorded as a dangerous object at the driver viewing angle, the method further comprises:

5. The vehicle travel information presentation method according to claim 4, wherein if the dangerous object does not exceed the preset presentation range of the target vehicle, the process of continuing to sense the dangerous object by using a sensor configured in advance or in real time on the target vehicle includes:

Detecting whether the sensor senses the dangerous object at the current moment;

6. The vehicle driving information prompting method according to claim 2, wherein the process of determining whether the perceived result has the target object in the current driving scene if the current driving scene includes an illumination condition less than a preset illumination threshold value includes:

7. The vehicle travel information presentation method according to claim 1, characterized in that the process of outputting the presentation information to the driver of the target vehicle based on the attribute information of the dangerous object during the travel of the target vehicle includes:

8. The vehicle travel information presentation method according to claim 1, wherein the sensor configured in advance or in real time on the target vehicle includes at least one of: laser radar, camera, millimeter wave radar, ultrasonic radar, inertial measurement unit, global navigation satellite system, infrared sensor.

9. A vehicle travel information presentation system, the system comprising:

10. A vehicle comprising a sensor and a processor; wherein,