CN111157014A - Road condition display method and device, vehicle-mounted terminal and storage medium - Google Patents
Road condition display method and device, vehicle-mounted terminal and storage medium Download PDFInfo
- Publication number
- CN111157014A CN111157014A CN201911394907.XA CN201911394907A CN111157014A CN 111157014 A CN111157014 A CN 111157014A CN 201911394907 A CN201911394907 A CN 201911394907A CN 111157014 A CN111157014 A CN 111157014A
- Authority
- CN
- China
- Prior art keywords
- road condition
- vehicle
- real
- road
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/34—Route searching; Route guidance
- G01C21/36—Input/output arrangements for on-board computers
- G01C21/3605—Destination input or retrieval
- G01C21/3617—Destination input or retrieval using user history, behaviour, conditions or preferences, e.g. predicted or inferred from previous use or current movement
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/34—Route searching; Route guidance
- G01C21/36—Input/output arrangements for on-board computers
- G01C21/3626—Details of the output of route guidance instructions
- G01C21/365—Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/34—Route searching; Route guidance
- G01C21/36—Input/output arrangements for on-board computers
- G01C21/3691—Retrieval, searching and output of information related to real-time traffic, weather, or environmental conditions
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Automation & Control Theory (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Ecology (AREA)
- Environmental Sciences (AREA)
- Environmental & Geological Engineering (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biodiversity & Conservation Biology (AREA)
- Atmospheric Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Social Psychology (AREA)
- Traffic Control Systems (AREA)
Abstract
The application discloses a road condition display method and device, a vehicle-mounted terminal and a storage medium, and relates to the technical field of vehicles. Determining whether a real obstacle target exists in the real-time road condition image or not according to the real-time road condition image, a road obstacle analysis model and height data of an object in front of a vehicle, wherein the road obstacle analysis model is trained in advance according to a plurality of historical road condition images with classification marks; if the real obstacle target exists in the real-time road condition image, processing the real-time road condition image into a virtual road condition image; the virtual road condition image is projected to the front windshield of the vehicle to be displayed by controlling the projection equipment, so that a real obstacle target appears in the front of the road, a driver can clearly see a visual field blind area in the front of the engine cover, the driving direction is adjusted in time to avoid obstacles, potential safety hazards during vehicle driving are reduced, the accuracy of determining the real obstacle target is high, and visual interference caused to the driver during normal road condition driving is avoided.
Description
Technical Field
The present application relates to the field of vehicle technologies, and in particular, to a road condition display method and apparatus, a vehicle-mounted terminal, and a storage medium.
Background
With the continuous development of the vehicle industry, vehicles gradually enter thousands of households, and become a necessary tool for life of people. The driving strategy of a vehicle is different for drivers in different weather, different road conditions and even different driving experiences. For example, when the vehicle is driven at night, in a foggy day, in a rainy day, or on a steep mountain road, a blind area may be formed in front of the vehicle, the driver may often not see the road condition ahead clearly, the driver has a safety hazard during driving, and the driving experience is also poor.
Disclosure of Invention
The embodiment of the application provides a road condition display method and device, a vehicle-mounted terminal and a storage medium, so that the problem that a driver cannot clearly see the road condition in front when a blind field of view exists in front of a vehicle is solved.
In a first aspect, an embodiment of the present application provides a road condition display method, including:
receiving a real-time road condition image in front of the vehicle sent by the image acquisition equipment and receiving height data of an object in front of the vehicle transmitted by the height detector in a road condition display mode;
determining whether a real obstacle target exists in the real-time road condition image according to the real-time road condition image, a road obstacle analysis model and height data of an object in front of the vehicle, wherein the road obstacle analysis model is trained in advance according to a plurality of historical road condition images with classification marks;
if the real road condition image has a real obstacle target, processing the real road condition image into a virtual road condition image;
and controlling projection equipment to project the virtual road condition image to a front windshield of the vehicle for display.
In a second aspect, an embodiment of the present application further provides a road condition display device, including:
the information receiving unit is configured to receive a real-time road condition image in front of the vehicle sent by the image acquisition equipment and receive height data of an object in front of the vehicle transmitted by the height detector in a road condition display mode;
the obstacle analysis unit is configured to determine whether a real obstacle target exists in the real-time road condition image according to the real-time road condition image, a road obstacle analysis model and height data of an object in front of the vehicle, wherein the road obstacle analysis model is trained in advance according to a plurality of historical road condition images carrying classification identifications;
an image processing unit configured to process the real-time road condition image into a virtual road condition image if a real obstacle target exists in the real-time road condition image;
and the image projection unit is configured to control the projection device to project the virtual road condition image to a front windshield display of the vehicle.
In a third aspect, an embodiment of the present application further provides a vehicle-mounted terminal, which includes a processor, a memory, and a computer program that is stored in the memory and is executable on the processor, and when the computer program is executed by the processor, the steps of the road condition displaying method are implemented.
In a fourth aspect, an embodiment of the present application further provides a storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the steps of the road condition displaying method.
The embodiment of the application adopts at least one technical scheme which can achieve the following beneficial effects: determining whether a real obstacle target exists in the real-time road condition image or not according to the real-time road condition image, a road obstacle analysis model and height data of an object in front of a vehicle, wherein the road obstacle analysis model is trained in advance according to a plurality of historical road condition images with classification marks; if the real obstacle target exists in the real-time road condition image, processing the real-time road condition image into a virtual road condition image; the virtual road condition image is projected to the front windshield of the vehicle to be displayed by controlling the projection equipment, so that when a real obstacle target appears in front of a road, the real-time road condition image is converted into the virtual road condition image to be displayed on the engine cover of the vehicle, a driver can clearly see visual field blind areas in front of and below the engine cover, the display effect of the transparent engine cover is realized, the driving direction is adjusted in time to avoid obstacles, the potential safety hazard during vehicle driving is reduced, and the driving experience is improved; the real obstacle target is determined by combining the real-time road condition image and the height data of the object in front of the vehicle, the accuracy is high, the virtual road condition image is displayed on the front windshield of the vehicle only when the real obstacle target is determined to exist, and visual interference to a driver is avoided when the driver drives under a normal road condition.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is an interaction schematic diagram of a vehicle-mounted terminal, a height detector, an image acquisition device, and a projection device, respectively, according to an embodiment of the present disclosure;
fig. 2 is a flowchart of a road condition displaying method according to an embodiment of the present application;
fig. 3 is a flowchart of a road condition displaying method according to an embodiment of the present application;
fig. 4 is a flowchart of a road condition displaying method according to an embodiment of the present application;
fig. 5 is a flowchart of a road condition displaying method according to an embodiment of the present application;
fig. 6 is a flowchart of a road condition displaying method according to an embodiment of the present application;
fig. 7 is a flowchart of a road condition displaying method according to an embodiment of the present application;
fig. 8 is a functional block diagram of a road condition display device according to an embodiment of the present application;
fig. 9 is a functional block diagram of a road condition display device according to an embodiment of the present application;
fig. 10 is a functional block diagram of a road condition display device according to an embodiment of the present application;
fig. 11 is a functional block diagram of a road condition display device according to an embodiment of the present application;
fig. 12 is a functional block diagram of a road condition display device according to an embodiment of the present application;
fig. 13 is a block diagram of an in-vehicle terminal according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.
Referring to fig. 1, an embodiment of the present application provides a road condition displaying method, which is applied to a vehicle-mounted terminal 101, wherein as shown in fig. 2, the vehicle-mounted terminal 101 is respectively connected to an image acquisition device 102, a height detector 104, and a projection device 103 through a CAN bus in a communication manner. The method comprises the following steps:
s11: in the road condition display mode, the real-time road condition image in front of the vehicle sent by the image acquisition device 102 and the height data of the object in front of the vehicle transmitted by the height detector 104 are received.
Specifically, after the driver inputs the road condition display instruction, the vehicle enters the road condition display mode. The road condition display instruction input mode can be as follows: triggering a target button (for example, an option for inputting a road condition presentation instruction set In an In-Vehicle Infotainment (IVI)) on an instrument panel of a Vehicle, and inputting the road condition presentation instruction; or, the voice "road condition display" is input, and after the vehicle is monitored, the voice is converted into a road condition display instruction and the like, which is not limited herein.
The driver can decide whether to input the road condition display instruction according to the current weather or road condition. For example, the road condition display instruction may be input when a blind area is likely to occur in front of the vehicle during driving on a foggy day, at night, on a mountain road, or the like. Under normal conditions, a road condition display instruction does not need to be input, so that the phenomenon that a virtual road condition image is generated subsequently and the vision of a driver is interfered is avoided.
In addition, the image capturing device 102 and the height detector 104 may be, but not limited to, disposed at intervals at the bottom of the front air grille of the vehicle so as to capture the image and the height data, respectively, the image capturing device 102 may be, but not limited to, a high-definition camera, and the height detector 104 may be, but not limited to, a laser detector.
S12: and determining whether a real obstacle target exists in the real-time road condition image according to the real-time road condition image, the road obstacle analysis model and the height data of the object in front of the vehicle, and if so, executing S13.
The road obstacle analysis model is formed by training in advance according to a plurality of historical road condition images carrying classification marks, and the classification marks are used for representing whether historical real obstacle targets are included in the historical road condition images or not. The plurality of historical road condition images with the classification identifications can be input into training networks such as a neural network and a capsule network to train the road obstacle analysis model.
S13: and processing the real-time road condition image into a virtual road condition image.
Specifically, the real-time traffic image may be converted into a planar virtual traffic image or a 3D virtual traffic image by using an Augmented Reality (AR) display technology, which is not limited herein.
S14: and controlling the projection device 103 to project the virtual road condition image to the front windshield of the vehicle for display.
The projection device 103 can project the virtual road condition image through a screen and display the virtual road condition image on a front windshield of the vehicle. The projection device 103 comprises a projection module, wherein the projection module is a micro-projection module based on DLP technology or a micro-projection module based on laser projection or a high-brightness liquid crystal screen; the projection module projects the image onto a front windshield display of the vehicle opposite the projection module.
The virtual road condition image is located in the position of the front windshield within the visual field range of the driver in the normal sitting posture head-up front, so that the driver can conveniently observe the virtual road condition image. In addition, if the driver drives the vehicle to go around the real obstacle target, the image capturing device 102 cannot capture the image of the real obstacle target in front of the vehicle and the height detector 104 cannot capture the height data of the real obstacle target in front of the vehicle, and at this time, the virtual road condition image displayed on the front windshield may be controlled to disappear, or the virtual road condition image displayed on the front windshield may be controlled to disappear in response to a closing instruction input by the driver.
The road condition display method comprises the steps of determining whether a real obstacle target exists in a real-time road condition image or not according to the real-time road condition image, a road obstacle analysis model and height data of an object in front of a vehicle, wherein the road obstacle analysis model is trained according to a plurality of historical road condition images with classification marks in advance; if the real obstacle target exists in the real-time road condition image, processing the real-time road condition image into a virtual road condition image; the projection equipment 103 is controlled to project the virtual road condition image to the front windshield of the vehicle for display, so that when a real obstacle target appears in front of a road, the real-time road condition image is converted into the virtual road condition image to be displayed on the engine cover of the vehicle, a driver can clearly see visual field blind areas in front of and below the engine cover, the display effect of the transparent engine cover is realized, the driving direction is adjusted in time to avoid obstacles, the potential safety hazard during vehicle driving is reduced, and the driving experience is improved; the real obstacle target is determined by combining the real-time road condition image and the height data of the object in front of the vehicle, the accuracy is high, the virtual road condition image is displayed on the front windshield of the vehicle only when the real obstacle target is determined to exist, and visual interference to a driver is avoided when the driver drives under a normal road condition.
Specifically, as shown in fig. 3, S12 includes:
s31: inputting the real-time road condition image into the road obstacle analysis model, determining whether a suspected obstacle target exists in the real-time road condition image, and if so, executing S32.
The road obstacle analysis model can only identify whether a suspected obstacle target exists in the real-time road condition image, the suspected obstacle target may be a real obstacle such as a small animal, a stone, a branch, a construction board and the like, and may also be a pool of sewage, a plastic bag, a paper sheet and the like on the ground, and therefore the suspected obstacle target needs to be further identified.
S33: it is determined whether the height of the suspected obstacle target in the height data of the preceding object is greater than a preset threshold value, and if so, S34 is performed.
S34: and determining that the real obstacle target exists in the real-time road condition image.
It is understood that an object with a large height value may obstruct the vehicle from passing through, and therefore, it is possible to determine whether the suspected obstacle target is a real obstacle target by identifying the height of the suspected obstacle target.
Optionally, as shown in fig. 4, as an embodiment, the method further includes:
s41: and receiving the distance of the chassis of the vehicle relative to the current ground transmitted by the distance sensor.
The distance sensor is mounted at the bottom of the vehicle so that the distance of the chassis of the vehicle relative to the current ground can be acquired.
S42: and estimating the height of the chassis of the vehicle in the off-road mode according to the distance between the chassis of the vehicle and the current ground.
It can be understood that the vehicle-mounted terminal 101 stores the height difference between the normal driving mode and the off-road mode, and calculates the sum of the received distance between the chassis of the vehicle and the current ground and the height difference between the normal driving mode and the off-road mode, that is, the height of the chassis of the vehicle in the off-road mode can be estimated.
S43: it is determined whether the height of the real obstacle target is between the distance of the chassis from the current ground and the height of the chassis when the vehicle is in the off-road mode, and if so, S44 is performed.
S44: and controlling the driving mode of the vehicle to be switched to the off-road mode.
When the height of the real obstacle target is between the distance between the chassis and the current ground and the height of the chassis when the vehicle is in the off-road mode, the driving mode of the vehicle can be controlled to be switched to the off-road mode (such as heightening the air suspension or the electronic suspension of the vehicle), so that the vehicle can directly pass through the real obstacle target without obstacle avoidance operation.
Optionally, as shown in fig. 5, as an embodiment, the method further includes:
s51: and receiving the distance of the chassis of the vehicle relative to the current ground transmitted by the distance sensor.
The distance sensor is mounted at the bottom of the vehicle so that the distance of the chassis of the vehicle relative to the current ground can be acquired.
S52: and estimating the height of the chassis of the vehicle in the off-road mode according to the distance between the chassis of the vehicle and the current ground.
It can be understood that the vehicle-mounted terminal 101 stores the height difference between the normal driving mode and the off-road mode, and calculates the sum of the received distance between the chassis of the vehicle and the current ground and the height difference between the normal driving mode and the off-road mode, that is, the height of the chassis of the vehicle in the off-road mode can be estimated.
S53: it is determined whether the height of the real obstacle target is between the distance of the chassis from the current ground and the height of the chassis when the vehicle is in the off-road mode, and if so, S54 is performed.
S54: and generating an off-road operation prompt.
The off-road operation prompt may be, but is not limited to, a text prompt of "obstacle ahead, please switch driving mode" on the instrument panel.
S55: and controlling the driving mode of the vehicle to be switched into the off-road mode in response to an off-road command input by a driver.
Whether the height of the real obstacle target is between the distance between the chassis and the current ground and the height of the chassis when the vehicle is in the off-road mode or not can control the driving mode of the vehicle to be switched to the off-road mode (such as heightening the air suspension or the electronic suspension of the vehicle), so that the vehicle can directly pass through the real obstacle target without obstacle avoidance operation.
In addition, when the driver sees the off-road operation prompt, the off-road instruction (which can be input by voice or by buttons on an instrument panel) can be input according to the off-road operation prompt, and whether the driving mode is switched or the driver controls the vehicle to bypass the real obstacle target is determined according to the instruction of the driver, so that the driving satisfaction of the driver is improved.
In addition, the method may further include:
s56: and acquiring the distance between the real obstacle target and the vehicle while determining that the real obstacle target exists in the real-time road condition image.
It is understood that the distance between the real obstacle target and the vehicle may be collected using a distance sensor provided at the front end.
S57: and determining the time interval of the vehicle running to the real obstacle target according to the distance between the real obstacle target and the vehicle and the current speed of the vehicle.
Specifically, the time interval is determined according to the formula T ═ S/V, where T is the time interval, S is the distance between the real obstacle target and the vehicle, and V is the current vehicle speed of the vehicle.
After S54, the method further comprises:
s58: and if the off-road instruction input by the driver is not responded, after the time interval, controlling the projection equipment to project the virtual road condition image to the front windshield display of the vehicle again.
If the off-road command input by the driver is not responded to, it may be because the driver is not paying attention to the real obstacle target in the previously displayed virtual road condition image. Causing the vehicle to travel just above the real obstacle target after a time interval, possibly causing the real obstacle target to collide with the chassis of the vehicle. At the moment, the projection equipment is controlled to project the virtual road condition image to the front windshield of the vehicle for displaying, so that the driver can be guided to avoid a real obstacle target.
The virtual road condition image should also be located in the front windshield within the field of view of the driver in the sitting position looking straight ahead. Specifically, the view field blind area of the front windshield in the view field range can be projected (for example, the view field of the front windshield and the view field of the engine cover can be projected at the moment, and the effect of transparent engine cover can be given to the driver).
Optionally, as shown in fig. 6, the method further includes:
s61: and generating the image information of the virtual road condition and generating the prompt information of the roadblock.
For example, the roadblock prompting message may be a text prompt of "ahead roadblock, please drive with caution", or a prompt of "exclamation mark", or an audible and visual alarm, etc., which is not limited herein.
S62: and transmitting the roadblock prompt information to a display interface for display.
Specifically, the barrier prompt information is projected to a front windshield display of the vehicle or output to an instrument panel display.
Optionally, as shown in fig. 7, the method further includes:
s71: and uploading the roadblock prompt information to a cloud server or transmitting the roadblock prompt information to a vehicle display associated with the current vehicle.
If the roadblock prompt information is uploaded to the cloud server, the cloud server can issue the roadblock prompt information to the vehicles located in the preset distance range behind the current vehicle so as to prompt the driver of the rear vehicle to drive carefully; if the vehicle (the ID of the passing vehicle is bound before the vehicle needs to go out again) which is transmitted to the current vehicle is displayed, the driver of the passing vehicle can be reminded to drive carefully.
Referring to fig. 8, an embodiment of the present application further provides a road condition displaying apparatus 800, which is applied to a vehicle-mounted terminal 101, wherein as shown in fig. 2, the vehicle-mounted terminal 101 is respectively connected to an image capturing device 102, a height detector 104, and a projection device 103 through a CAN bus in a communication manner. It should be noted that the basic principle and the generated technical effects of the road condition displaying apparatus 800 provided in the embodiment of the present application are the same as those of the above embodiment, and for the sake of brief description, no part of the embodiment of the present application is mentioned, and reference may be made to the corresponding contents in the above embodiment. The apparatus 800 comprises an information receiving unit 801, an obstacle analyzing unit 802, an image processing unit 803, and an image projecting unit 804, wherein,
the information receiving unit 801 is configured to receive the real-time road condition image in front of the vehicle sent by the image capturing device 102 and the height data of the object in front of the vehicle transmitted by the height detector 104 in the road condition displaying mode.
And an obstacle analysis unit 802 configured to determine whether a real obstacle target exists in the real-time road condition image according to the real-time road condition image, the road obstacle analysis model, and height data of an object in front of the vehicle.
The road obstacle analysis model is formed by training in advance according to a plurality of historical road condition images carrying classification marks.
An image processing unit 803 configured to process the real-time road condition image into a virtual road condition image if a real obstacle target exists in the real-time road condition image.
And an image projection unit 804 configured to control the projection device 103 to project the virtual road condition image to a front windshield display of the vehicle.
The road condition display device 800 can realize the following functions when being executed: determining whether a real obstacle target exists in the real-time road condition image or not according to the real-time road condition image, a road obstacle analysis model and height data of an object in front of a vehicle, wherein the road obstacle analysis model is trained in advance according to a plurality of historical road condition images with classification marks; if the real obstacle target exists in the real-time road condition image, processing the real-time road condition image into a virtual road condition image; the projection equipment 103 is controlled to project the virtual road condition image to the front windshield of the vehicle for display, so that when a real obstacle target appears in front of a road, the real-time road condition image is converted into the virtual road condition image to be displayed on the front windshield of the vehicle, a driver can clearly see visual field blind areas in front of and below the engine cover, the display effect of the transparent engine cover is realized, the driving direction is adjusted in time to avoid obstacles, the potential safety hazard during vehicle driving is reduced, and the driving experience is improved; the real-time road condition image and the height data of the object in front of the vehicle are combined to determine the real obstacle target, the accuracy is high, the virtual road condition image is displayed on the engine hood of the vehicle only when the real obstacle target is determined to exist, and visual interference to a driver is avoided when the driver drives under normal road conditions.
As shown in fig. 9, the obstacle analysis unit 801 includes: the suspected target determining module 901 is configured to input the real-time traffic image into the road obstacle analysis model, and determine whether a suspected obstacle target exists in the real-time traffic image.
The determining module 902 is configured to determine whether the height of the suspected obstacle target in the height data of the front object is greater than a preset threshold value if it is determined that the suspected obstacle target exists in the real-time road condition image.
A real target determination module 903 configured to determine that a real obstacle target exists in the real-time road condition image if the height of the suspected obstacle target is greater than a preset threshold.
Optionally, as one of the embodiments, the information receiving unit 801 is further configured to receive the distance of the chassis of the vehicle relative to the current ground, which is transmitted by the distance sensor. As shown in fig. 10, the apparatus 800 further includes:
a chassis height estimation unit 1001 configured to estimate a chassis height when the vehicle is in an off-road mode according to a distance of a chassis of the vehicle with respect to a current ground.
A driving mode switching unit 1002 configured to control the driving mode of the vehicle to be switched to the off-road mode if the height of the real obstacle target is between the distance of the chassis from the current ground and the height of the chassis when the vehicle is in the off-road mode.
Optionally, as another embodiment, the information receiving unit is further configured to receive the distance of the chassis of the vehicle relative to the current ground transmitted by the distance sensor. As shown in fig. 11, the apparatus 800 further includes:
a ride height estimation unit 1101 configured to estimate a ride height when the vehicle is in an off-road mode based on a distance of a chassis of the vehicle relative to a current ground.
An operation prompt generation unit 1102 configured to generate an off-road operation prompt if the height of the real obstacle target is between the distance of the chassis from the current ground and the height of the chassis when the vehicle is in the off-road mode.
A driving mode switching unit 1103 configured to control the driving mode of the vehicle to be switched to the off-road mode in response to an off-road instruction input by the driver.
Optionally, as shown in fig. 12, the apparatus 800 further includes:
a road block prompt generating unit 1201 configured to generate road block prompt information while generating the virtual road condition image information.
And an information sending unit 1202 configured to transmit the roadblock prompting information to a display interface display.
For example, the barrier prompt information is projected onto a front windshield display of the vehicle or output to an instrument panel display.
The information sending unit 801 is further configured to upload the barrier prompt information to a cloud server or to a vehicle display associated with the current vehicle.
It should be noted that the execution subjects of the steps of the method provided in embodiment 1 may be the same device, or different devices may be used as the execution subjects of the method. For example, the execution subject of steps 11 and 12 may be device 1, and the execution subject of step 13 may be device 2; for another example, the execution subject of step 11 may be device 1, and the execution subjects of step 12 and step 13 may be device 2; and so on.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
Fig. 13 is a schematic structural diagram of the in-vehicle terminal 101 according to an embodiment of the present application. Referring to fig. 13, in a hardware level, the vehicle-mounted terminal 101 includes a processor, and optionally further includes an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory, such as at least 1 disk Memory. Of course, the in-vehicle terminal may also include hardware required for other services.
The processor, the network interface, and the memory may be connected to each other via an internal bus, which may be an ISA (Industry standard architecture) bus, a PCI (Peripheral component interconnect) bus, an EISA (Extended Industry standard architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 13, but that does not indicate only one bus or one type of bus.
And the memory is used for storing programs. In particular, the program may include program code comprising computer operating instructions. The memory may include both memory and non-volatile storage and provides instructions and data to the processor.
The processor reads the corresponding computer program from the nonvolatile memory to the memory and then runs the computer program to form the road condition display device on the logic level. The processor is used for executing the program stored in the memory and is specifically used for executing the following operations:
receiving a real-time road condition image in front of the vehicle sent by the image acquisition equipment and receiving height data of an object in front of the vehicle transmitted by the height detector in a road condition display mode;
determining whether a real obstacle target exists in the real-time road condition image according to the real-time road condition image, a road obstacle analysis model and height data of an object in front of the vehicle, wherein the road obstacle analysis model is trained in advance according to a plurality of historical road condition images with classification marks;
if the real road condition image has a real obstacle target, processing the real road condition image into a virtual road condition image;
and controlling projection equipment to project the virtual road condition image to a front windshield of the vehicle for display.
The method executed by the road condition displaying apparatus according to the embodiment shown in fig. 1 of the present application can be applied to a processor, or implemented by the processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware component. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.
The vehicle-mounted terminal can also execute the method shown in fig. 1 and realize the functions of the road condition display device in the embodiment shown in fig. 1, which is not described herein again in the embodiments of the present application.
Of course, the vehicle-mounted terminal of the present application does not exclude other implementations, such as a logic device or a combination of software and hardware, besides a software implementation, that is, the execution main body of the following processing flow is not limited to each logic unit, and may also be hardware or a logic device.
An embodiment of the present application also proposes a computer-readable storage medium storing one or more programs, the one or more programs including instructions, which when executed by a portable in-vehicle terminal including a plurality of application programs, can cause the in-vehicle terminal to perform the method of the embodiment shown in fig. 1, and is specifically configured to perform the following operations:
receiving a real-time road condition image in front of the vehicle sent by the image acquisition equipment and receiving height data of an object in front of the vehicle transmitted by the height detector in a road condition display mode;
determining whether a real obstacle target exists in the real-time road condition image according to the real-time road condition image, a road obstacle analysis model and height data of an object in front of the vehicle, wherein the road obstacle analysis model is trained in advance according to a plurality of historical road condition images with classification marks;
if the real road condition image has a real obstacle target, processing the real road condition image into a virtual road condition image;
and controlling projection equipment to project the virtual road condition image to a front windshield of the vehicle for display.
In short, the above description is only a preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include transitory computer readable media (transmyedia) such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
Claims (10)
1. A road condition display method is characterized by comprising the following steps:
receiving a real-time road condition image in front of the vehicle sent by the image acquisition equipment and receiving height data of an object in front of the vehicle transmitted by the height detector in a road condition display mode;
determining whether a real obstacle target exists in the real-time road condition image according to the real-time road condition image, a road obstacle analysis model and height data of an object in front of the vehicle, wherein the road obstacle analysis model is trained in advance according to a plurality of historical road condition images with classification marks;
if the real road condition image has a real obstacle target, processing the real road condition image into a virtual road condition image;
and controlling projection equipment to project the virtual road condition image to a front windshield of the vehicle for display.
2. The road condition displaying method according to claim 1, wherein the determining whether the real obstacle target exists in the real-time road condition image according to the real-time road condition image, the road obstacle analysis model and the height data of the object in front of the vehicle comprises:
inputting the real-time road condition image into the road obstacle analysis model, and determining whether a suspected obstacle target exists in the real-time road condition image;
if the suspected obstacle target exists in the real-time road condition image, determining whether the height of the suspected obstacle target in the height data of the front object is larger than a preset threshold value or not;
and if the height of the suspected obstacle target is larger than a preset threshold value, determining that a real obstacle target exists in the real-time road condition image.
3. The road condition displaying method according to claim 1, wherein after the controlling the projection device to project the virtual road condition image to a front windshield of a vehicle for display, the method further comprises:
receiving the distance of the chassis of the vehicle relative to the current ground transmitted by the distance sensor;
according to the distance between the chassis of the vehicle and the current ground, the height of the chassis of the vehicle in the off-road mode is estimated;
controlling the driving mode of the vehicle to switch to an off-road mode if the height of the real obstacle target is between the distance of the chassis from the current ground and the height of the chassis when the vehicle is in the off-road mode.
4. The road condition displaying method according to claim 1, wherein after the controlling the projection device to project the virtual road condition image to a front windshield of a vehicle for display, the method further comprises:
receiving the distance of the chassis of the vehicle relative to the current ground transmitted by the distance sensor;
estimating the height of the chassis of the vehicle in an off-road mode according to the distance between the chassis of the vehicle and the current ground;
generating an off-road operation cue if the height of the real obstacle target is between the distance of the chassis from the current ground and the chassis height when the vehicle is in an off-road mode;
and controlling the driving mode of the vehicle to be switched to the off-road mode if responding to an off-road instruction input by a driver.
5. The road condition display method according to claim 4, further comprising:
the method comprises the steps of acquiring the distance between a real obstacle target and a vehicle while determining that the real obstacle target exists in a real-time road condition image;
determining the time interval from the vehicle to the real obstacle target according to the distance between the real obstacle target and the vehicle and the current speed of the vehicle;
and if the off-road instruction input by the driver is not responded after the off-road operation prompt is generated, controlling the projection equipment to project the virtual road condition image to the front windshield display of the vehicle again after the time interval.
6. The road condition displaying method according to claim 1, wherein the road block prompt information is generated while the virtual road condition image information is generated;
and projecting the roadblock prompt information to a front windshield of a vehicle for display or outputting the roadblock prompt information to an instrument screen for display.
7. The road condition displaying method according to claim 6, further comprising: and uploading the roadblock prompt information to a cloud server or transmitting the roadblock prompt information to a vehicle display associated with the current vehicle.
8. A road condition display device, characterized in that, the device includes:
the information receiving unit is configured to receive a real-time road condition image in front of the vehicle sent by the image acquisition equipment and receive height data of an object in front of the vehicle transmitted by the height detector in a road condition display mode;
the obstacle analysis unit is configured to determine whether a real obstacle target exists in the real-time road condition image according to the real-time road condition image, a road obstacle analysis model and height data of an object in front of the vehicle, wherein the road obstacle analysis model is trained in advance according to a plurality of historical road condition images carrying classification identifications;
an image processing unit configured to process the real-time road condition image into a virtual road condition image if a real obstacle target exists in the real-time road condition image;
and the image projection unit is configured to control the projection device to project the virtual road condition image to a front windshield display of the vehicle.
9. An in-vehicle terminal, comprising a processor, a memory and a computer program stored on the memory and operable on the processor, wherein the computer program, when executed by the processor, implements the steps of the road condition displaying method according to any one of claims 1 to 7.
10. A storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the road condition presentation method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911394907.XA CN111157014A (en) | 2019-12-30 | 2019-12-30 | Road condition display method and device, vehicle-mounted terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911394907.XA CN111157014A (en) | 2019-12-30 | 2019-12-30 | Road condition display method and device, vehicle-mounted terminal and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111157014A true CN111157014A (en) | 2020-05-15 |
Family
ID=70559154
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911394907.XA Pending CN111157014A (en) | 2019-12-30 | 2019-12-30 | Road condition display method and device, vehicle-mounted terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111157014A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111710181A (en) * | 2020-05-30 | 2020-09-25 | 芜湖扬宇机电技术开发有限公司 | Safe driving method in cluster fog |
| CN112606935A (en) * | 2020-11-08 | 2021-04-06 | 泰州镭昇光电科技有限公司 | Cross-country bicycle safety control system |
| CN112977469A (en) * | 2021-03-12 | 2021-06-18 | 乐视致新信息技术(武汉)有限公司 | Navigation device based on vehicle-mounted holographic projection and control method thereof |
| CN113034607A (en) * | 2021-03-03 | 2021-06-25 | 恒大新能源汽车投资控股集团有限公司 | Method and system for installing and calibrating vehicle-mounted projection equipment and electronic equipment |
| CN113391368A (en) * | 2021-06-30 | 2021-09-14 | 山东国瑞新能源有限公司 | Road exploration method and equipment based on virtual imaging technology |
| CN114368342A (en) * | 2021-04-16 | 2022-04-19 | 长城汽车股份有限公司 | Driving assistance method, storage medium, and vehicle |
| CN115134491A (en) * | 2022-05-27 | 2022-09-30 | 深圳市有方科技股份有限公司 | Image processing method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101549683A (en) * | 2009-04-23 | 2009-10-07 | 上海交通大学 | Vehicle intelligent method for automatically identifying road pit or obstruction |
| US20130144521A1 (en) * | 2011-12-02 | 2013-06-06 | GM Global Technology Operations LLC | Optimum driving path on full windshield display |
| CN103534560A (en) * | 2011-03-15 | 2014-01-22 | 捷豹路虎有限公司 | Wading vehicle control system |
| CN105196918A (en) * | 2014-06-12 | 2015-12-30 | 通用汽车环球科技运作有限责任公司 | Process For Representing Vehicle Surroundings Information Of A Motor Vehicle |
| CN107985443A (en) * | 2017-12-23 | 2018-05-04 | 长安大学 | A kind of wheeled more ground adaptability vehicles and its obstacle-detouring method |
| CN108515909A (en) * | 2018-04-04 | 2018-09-11 | 京东方科技集团股份有限公司 | A kind of automobile head-up-display system and its barrier prompt method |
| JP2019087127A (en) * | 2017-11-09 | 2019-06-06 | サカエ理研工業株式会社 | Vehicle traffic determination device |
-
2019
- 2019-12-30 CN CN201911394907.XA patent/CN111157014A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101549683A (en) * | 2009-04-23 | 2009-10-07 | 上海交通大学 | Vehicle intelligent method for automatically identifying road pit or obstruction |
| CN103534560A (en) * | 2011-03-15 | 2014-01-22 | 捷豹路虎有限公司 | Wading vehicle control system |
| US20130144521A1 (en) * | 2011-12-02 | 2013-06-06 | GM Global Technology Operations LLC | Optimum driving path on full windshield display |
| CN105196918A (en) * | 2014-06-12 | 2015-12-30 | 通用汽车环球科技运作有限责任公司 | Process For Representing Vehicle Surroundings Information Of A Motor Vehicle |
| JP2019087127A (en) * | 2017-11-09 | 2019-06-06 | サカエ理研工業株式会社 | Vehicle traffic determination device |
| CN107985443A (en) * | 2017-12-23 | 2018-05-04 | 长安大学 | A kind of wheeled more ground adaptability vehicles and its obstacle-detouring method |
| CN108515909A (en) * | 2018-04-04 | 2018-09-11 | 京东方科技集团股份有限公司 | A kind of automobile head-up-display system and its barrier prompt method |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111710181A (en) * | 2020-05-30 | 2020-09-25 | 芜湖扬宇机电技术开发有限公司 | Safe driving method in cluster fog |
| CN112606935A (en) * | 2020-11-08 | 2021-04-06 | 泰州镭昇光电科技有限公司 | Cross-country bicycle safety control system |
| CN112606935B (en) * | 2020-11-08 | 2021-12-17 | 庄景江 | Cross-country bicycle safety control system |
| CN113034607A (en) * | 2021-03-03 | 2021-06-25 | 恒大新能源汽车投资控股集团有限公司 | Method and system for installing and calibrating vehicle-mounted projection equipment and electronic equipment |
| CN112977469A (en) * | 2021-03-12 | 2021-06-18 | 乐视致新信息技术(武汉)有限公司 | Navigation device based on vehicle-mounted holographic projection and control method thereof |
| CN114368342A (en) * | 2021-04-16 | 2022-04-19 | 长城汽车股份有限公司 | Driving assistance method, storage medium, and vehicle |
| CN113391368A (en) * | 2021-06-30 | 2021-09-14 | 山东国瑞新能源有限公司 | Road exploration method and equipment based on virtual imaging technology |
| CN115134491A (en) * | 2022-05-27 | 2022-09-30 | 深圳市有方科技股份有限公司 | Image processing method and device |
| CN115134491B (en) * | 2022-05-27 | 2023-11-24 | 深圳市有方科技股份有限公司 | Image processing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111157014A (en) | Road condition display method and device, vehicle-mounted terminal and storage medium | |
| US20210365696A1 (en) | Vehicle Intelligent Driving Control Method and Device and Storage Medium | |
| US9507345B2 (en) | Vehicle control system and method | |
| US11318886B2 (en) | Interactive safety system for vehicles | |
| US20190315368A1 (en) | Assisted driving method and apparatus, computing device, computer readable storage medium and computer program product | |
| CN107487258A (en) | Blind area detecting system and method | |
| US11837093B2 (en) | Lane change notification | |
| CN107487333A (en) | Blind area detecting system and method | |
| JP2015139128A (en) | Vehicular periphery monitoring device | |
| CN113205088B (en) | Obstacle image presentation method, electronic device, and computer-readable medium | |
| JP2009231938A (en) | Surrounding monitoring device for vehicle | |
| US11393223B2 (en) | Periphery monitoring device | |
| US11580695B2 (en) | Method for a sensor-based and memory-based representation of a surroundings, display device and vehicle having the display device | |
| JP2019185105A (en) | Vehicle system, space area estimation method and space area estimation apparatus | |
| JPWO2018150888A1 (en) | Information recording apparatus, information recording method, and information recording program | |
| KR20180130201A (en) | Apparatus and method of support safe driving considering rear vehicle | |
| CN111645705A (en) | Method for issuing driving route adjustment and server | |
| KR20210147410A (en) | Method and apparatus for providing information using vehicle's camera | |
| TWI573713B (en) | Indicating device and method for driving distance with vehicles | |
| US11383733B2 (en) | Method and system for detecting a dangerous driving condition for a vehicle, and non-transitory computer readable medium storing program for implementing the method | |
| JP2019074862A (en) | Recommendable driving output device, recommendable driving output system, and recommendable driving output method | |
| CN111756987B (en) | Control method and device of vehicle-mounted camera and vehicle-mounted image capturing system | |
| CN114511834A (en) | Method and device for determining prompt information, electronic equipment and storage medium | |
| JP6983335B2 (en) | Operation judgment device and operation judgment method | |
| JP2022056153A (en) | Temporary stop detection device, temporary stop detection system, and temporary stop detection program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20201214 Address after: Room 1402, basement and building T7, Nansha Jinmao Bay, No.01, jinmaozhong 2nd Street, Huangge Town, Nansha District, Guangzhou City, Guangdong Province Applicant after: Hengda hengchi new energy automobile technology (Guangdong) Co.,Ltd. Address before: Room 1403, basement and building T7, Nansha Jinmao Bay, No.01, Jinmao middle street, Huangge Town, Nansha District, Guangzhou, Guangdong Province Applicant before: Evergrande new energy vehicle technology (Guangdong) Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200515 |