CN110254349B - Vehicle collision early warning method and system, vehicle and storage medium - Google Patents
Vehicle collision early warning method and system, vehicle and storage medium Download PDFInfo
- Publication number
- CN110254349B CN110254349B CN201910582457.0A CN201910582457A CN110254349B CN 110254349 B CN110254349 B CN 110254349B CN 201910582457 A CN201910582457 A CN 201910582457A CN 110254349 B CN110254349 B CN 110254349B
- Authority
- CN
- China
- Prior art keywords
- obstacle
- early warning
- camera
- radar sensor
- vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
- B60Q9/008—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling for anti-collision purposes
Abstract
A vehicle collision early warning method, a system, a vehicle and a storage medium are provided, the method comprises the following steps: detecting an obstacle by using a radar sensor and a camera; determining an early warning level according to the detection result of the radar sensor and the detection result of the camera; determining a virtual image of the obstacle corresponding to the early warning level; outputting and displaying a real environment image shot by the camera, and superposing and displaying the virtual image on the real environment image; and the virtual images of the obstacles corresponding to different early warning levels are different. By implementing the embodiment of the invention, the detection blind area of the barrier can be reduced, and the risk of vehicle collision is reduced.
Description
Technical Field
The invention relates to the technical field of driving assistance, in particular to a vehicle collision early warning method, a vehicle collision early warning system, a vehicle and a storage medium.
Background
At present, most vehicles are equipped with a collision early warning system (such as a reversing radar), can detect obstacles in a low-speed reversing scene, and carry out prompt early warning through sound or reversing images when detecting that the obstacles exist.
In practice, it is found that when designing a collision warning system, the number of radars to be used is generally reduced as much as possible based on the concept of cost and beautiful appearance. Further, the radar is installed at a position as beautiful as possible, and such an installation position may sacrifice a part of the radar performance. In addition, radar sensors themselves have certain limitations. Taking the ultrasonic radar as an example, the coverage of the acoustic wave envelope emitted by the ultrasonic radar is limited, and is generally 120 ° in the horizontal direction and 60 ° in the vertical direction.
Under the comprehensive influence of factors such as the number of radars, the installation position, the envelope characteristic and the like, the reversing radar system may have a large detection blind area. Referring to fig. 1, fig. 1 is a diagram illustrating an exemplary detection range that can be covered by a collision warning system. The vehicle shown in fig. 1 is equipped with two radar sensors, one mounted on the outside of the bumper and the other mounted on the inside of the bumper. H1 in FIG. 1 is the minimum height of the vehicle from the ground, which is generally 200 mm-300 mm; h2 is the ground clearance of the lowest point of the envelope coverage range of the radar outside the bumper, and is generally 280 mm-350 mm; h3 is the height above the ground of the lowest point of the envelope coverage range of the radar on the inner side of the bumper, and is generally 130-160 mm. It can be seen that the collision early warning system shown in fig. 1 has a larger detection blind area in a region close to the ground, and the area of the detection blind area is larger as the collision early warning system is closer to the vehicle. The existence of the detection blind area can lead to the missed detection of short obstacles by a collision early warning system, thereby leading to the rise of the risk of vehicle collision.
Disclosure of Invention
The embodiment of the invention discloses a vehicle collision early warning method, a vehicle collision early warning system, a vehicle and a storage medium, which can reduce a detection blind area of an obstacle and reduce the risk of vehicle collision.
The embodiment of the invention discloses a vehicle collision early warning method in a first aspect, which comprises the following steps:
detecting an obstacle by using a radar sensor and a camera;
determining an early warning level according to the detection result of the radar sensor and the detection result of the camera;
determining a virtual image of the obstacle corresponding to the early warning level;
outputting and displaying a real environment image shot by the camera, and superposing and displaying the virtual image on the real environment image;
and the virtual images of the obstacles corresponding to different early warning levels are different.
As an optional implementation manner, in the first aspect of the embodiment of the present invention, the determining an early warning level according to the detection result of the radar sensor and the detection result of the camera includes:
if the detection result of the radar sensor and the detection result of the camera indicate that an obstacle exists in the process that the vehicle moves for a preset distance, determining the early warning level as high;
if the detection result of the radar sensor indicates that the obstacle disappears along with the movement of the vehicle in the process that the vehicle moves the preset distance, and meanwhile, the detection result of the camera indicates that the obstacle exists, the early warning grade is determined to be a medium grade;
and if the detection result of the radar sensor indicates that no obstacle exists and the detection result of the camera indicates that the obstacle exists in the process that the vehicle moves the preset distance, determining the early warning level as low.
As an optional implementation manner, in the first aspect of the embodiment of the present invention, the displaying the virtual image on the real environment image in an overlapping manner includes:
identifying an image position of the obstacle in the real environment image;
and superposing and displaying the virtual image on the image position.
As an optional implementation manner, in the first aspect of the embodiment of the present invention, the virtual images of the obstacles corresponding to different warning levels are highlighted to different degrees.
As an optional implementation manner, in the first aspect of the embodiment of the present invention, before the performing obstacle detection by using a radar sensor and a camera, the method further includes:
detecting whether the vehicle is in a reverse gear;
and if so, executing the step of detecting the obstacle by using the radar sensor and the camera.
As an optional implementation manner, in the first aspect of the embodiment of the present invention, the obstacle detection using a radar sensor and a camera includes:
detecting a plurality of pre-divided early warning planning areas around the vehicle by using a radar sensor;
if the fact that an obstacle exists in one early warning planning area is detected, shooting the early warning planning area with the obstacle by using a camera;
and detecting whether an obstacle exists in the image shot by the camera.
The second aspect of the embodiment of the present invention discloses a vehicle collision early warning system, including: a first detection unit for performing obstacle detection using a radar sensor and a camera;
the first determining unit is used for determining an early warning level according to the detection result of the radar sensor and the detection result of the camera;
a second determination unit configured to determine a virtual image of the obstacle corresponding to the early warning level;
the first display unit is used for outputting and displaying a real environment image shot by the camera;
the second display unit is used for displaying the virtual image on the real environment image in an overlapping mode;
and the virtual images of the obstacles corresponding to different early warning levels are different.
As an alternative implementation, in the second aspect of the embodiment of the present invention:
the first determining unit is specifically configured to determine that an early warning level is high if the detection result of the radar sensor and the detection result of the camera both indicate that an obstacle exists in the process that the vehicle moves by a preset distance; and if the detection result of the radar sensor indicates that the obstacle disappears along with the movement of the vehicle and the detection result of the camera indicates that the obstacle exists in the process of the movement of the vehicle by the preset distance, determining the early warning level as a middle level; and in the process that the vehicle moves for the preset distance, if the detection result of the radar sensor indicates that no obstacle exists and the detection result of the camera indicates that the obstacle exists, determining the early warning level as low.
As an alternative implementation, in a second aspect of the embodiment of the present invention, the second display unit includes:
the identification subunit is used for identifying the image position of the obstacle in the real environment image;
and the superposition subunit is used for superposing and displaying the virtual image on the image position.
As an optional implementation manner, in the second aspect of the embodiment of the present invention, the virtual images of the obstacles corresponding to different warning levels are highlighted to different degrees.
As an optional implementation manner, in the second aspect of the embodiment of the present invention, the system further includes:
a second detection unit configured to detect whether the vehicle is in a reverse gear before the first detection unit detects an obstacle using a radar sensor and a camera; and when the fact that the vehicle is in a reverse gear is detected, triggering the first detection unit to execute the operation of detecting the obstacle by using the radar sensor and the camera.
As an optional implementation manner, in a second aspect of the embodiment of the present invention, the first detection unit includes:
the first detection subunit is used for detecting a plurality of pre-divided early warning planning areas around the vehicle by using a radar sensor;
the shooting subunit is configured to, when the first detection subunit detects that an obstacle exists in a certain early warning planning area, shoot the early warning planning area where the obstacle exists by using a camera;
and the second detection subunit is used for detecting whether an obstacle exists in the image shot by the camera.
A third aspect of the embodiments of the invention discloses a vehicle, including: any one of the vehicle collision warning systems disclosed in the second aspect of the embodiment of the invention.
A fourth aspect of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to execute any one of the methods disclosed in the first aspect of the embodiments of the present invention.
A fifth aspect of the embodiments of the present invention discloses a computer program product, which, when running on a computer, causes the computer to execute any one of the methods disclosed in the first aspect of the embodiments of the present invention.
Compared with the prior art, the embodiment of the invention has the following beneficial effects:
according to the embodiment of the invention, the information of the obstacles detected by the radar sensor and the camera is fused, and the detection blind area of the radar sensor can be supplemented by the visual information provided by the camera, so that the detection blind area of the obstacles can be reduced. And according to the difference of the detection results of the radar sensor and the camera on the obstacle, different early warning levels are divided, and different virtual images are correspondingly displayed, so that a driver can quickly identify the current early warning level, and then adopt corresponding obstacle avoidance operation, and the risk of vehicle collision is reduced.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is an exemplary diagram of a detection range that can be covered by a collision warning system according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a vehicle collision warning method according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart of another vehicle collision warning method disclosed in the embodiment of the invention;
FIG. 4 is an exemplary diagram of an early warning planning area around a vehicle according to an embodiment of the disclosure;
FIG. 5 is a schematic structural diagram of a vehicle collision warning system according to an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of another vehicle collision warning system disclosed in the embodiment of the invention;
fig. 7 is a schematic structural diagram of another vehicle collision warning system according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It is to be noted that the terms "comprises" and "comprising" and any variations thereof in the embodiments and drawings of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
The embodiment of the invention discloses a vehicle collision early warning method which can reduce a detection blind area of an obstacle and reduce the risk of vehicle collision. The following are detailed below.
Example one
Referring to fig. 2, fig. 2 is a schematic flow chart of a vehicle collision warning method according to an embodiment of the present invention. As shown in fig. 2, the vehicle collision warning method may include the steps of:
101. the early warning system utilizes radar sensor and camera to carry out the obstacle detection.
In the embodiment of the present invention, the radar sensor may be an ultrasonic radar sensor, a millimeter wave radar sensor, a laser radar sensor, or the like, which is not limited in the embodiment of the present invention. The radar sensors may be installed at the head of the vehicle, the tail of the vehicle, and/or the door positions at both sides of the vehicle, and the number of the radar sensors may be one or more. The camera may be a common optical camera or a 360 ° panoramic camera, and the embodiment of the present invention is not limited. Further, the viewing range of the camera may partially or completely overlap with the detection range of the radar sensor, which is not limited in the embodiment of the present invention.
Because the scope of framing of camera and radar sensor's detection range probably is different, therefore the camera probably shoots the region that radar sensor can't detect, and radar sensor can detect the region that the camera can't shoot, can be each other supplementary between camera and the radar sensor. When the camera is used for detecting the obstacle, the obstacle can be extracted from the image shot by the camera through image recognition algorithms such as deep learning.
102. And the early warning system determines an early warning level according to the detection result of the radar sensor and the detection result of the camera.
In the embodiment of the present invention, the detection result of the radar sensor may specifically indicate the presence or absence of an obstacle; the detection result of the camera may also indicate the presence or absence of an obstacle. Therefore, there are the following cases:
1) the detection result of the radar sensor indicates that an obstacle exists, and the detection result of the camera also indicates that the obstacle exists;
2) the detection result of the radar sensor indicates that no obstacle exists, but the detection result of the camera indicates that the obstacle exists;
3) the detection result of the radar sensor indicates that an obstacle exists, but the detection result of the camera indicates that the obstacle does not exist;
4) the detection result of the radar sensor indicates that no obstacle exists, and the detection result of the camera indicates that no obstacle exists.
Each of the above situations may correspond to an early warning level. Optionally, the early warning level corresponding to the case 1) may be high; the early warning levels corresponding to the cases 2) and 3) can be intermediate levels; case 4) the corresponding warning level may be low. The higher the warning level, the higher the probability that an obstacle is actually present.
103. The early warning system determines a virtual image of the obstacle corresponding to the early warning level.
In the embodiment of the invention, the virtual images of the obstacles corresponding to different early warning levels are different. Therefore, the user can intuitively understand the current early warning level through different virtual images, so as to determine what coping strategies need to be taken.
104. The early warning system outputs and displays a real environment image shot by the camera, and a virtual image is superposed and displayed on the real environment image.
As an alternative embodiment, the warning system may display the virtual image in an overlapping manner at any position of the real environment image. Preferably, the first and second liquid crystal materials are,
embodiments of step 104 may also include the steps of:
identifying the image position of the obstacle in the real environment image;
the virtual image is superimposed and displayed on the image position.
That is, the warning system directly marks the position of the obstacle in the real environment image to prompt the driver to pay attention. The driver can directly locate which object in the real environment is the obstacle with collision risk through the virtual image, so that obstacle avoidance measures can be implemented more easily.
The vehicle collision early warning method described in fig. 2 can be applied to scenes of low-speed driving such as reversing and parking, and the detection blind area of the radar sensor is supplemented by the camera, so that the detection blind area of the obstacle is reduced. And according to the difference of the detection results of the radar sensor and the camera on the obstacle, different early warning levels are divided, and different virtual images are correspondingly displayed, so that a driver can quickly identify the current early warning level, and then adopt corresponding obstacle avoidance operation, and the risk of vehicle collision is reduced.
Example two
Referring to fig. 3, fig. 3 is a schematic flow chart of another vehicle collision warning method according to an embodiment of the present invention. As shown in fig. 3, the vehicle collision warning method may include the steps of:
301. the early warning system detects whether the vehicle is in a reverse gear; if yes, go to step 302; if not, the flow is ended.
Because the number of vision blind areas of the driver is large during reversing, the embodiment of the invention executes the following steps 302-307 when detecting that the vehicle needs to be backed, so as to improve the obstacle detection accuracy during reversing and improve the driving safety during reversing.
302. The early warning system utilizes the radar sensor to detect a plurality of early warning planning areas which are divided in advance around the vehicle.
In the embodiment of the present invention, please refer to fig. 4, in which fig. 4 is an exemplary diagram of an early warning planning area around a vehicle according to the embodiment of the present invention. As an alternative, each pre-warning planning area may be a detection area of a single radar sensor, and the data of the pre-warning planning area is determined based on the number of radar sensors installed on the vehicle.
303. If an obstacle exists in a certain early warning planning area, the early warning system shoots the early warning planning area with the obstacle by using the camera.
In the embodiment of the invention, the radar sensor is used for carrying out primary obstacle detection, and if the radar sensor detects that the obstacle exists, the camera is used for further obstacle identification. That is, image recognition is performed on the early warning planning area emphasis where the radar sensor detects the existence of the obstacle. By implementing the embodiment, the data processing amount of image recognition can be reduced, so that the response speed of the early warning system is improved.
304. And the early warning system determines an early warning level according to the detection result of the radar sensor and the detection result of the camera.
In the embodiment of the present invention, as an optional implementation manner, the detection result of the camera is taken as the standard, and if the detection result of the camera indicates that no obstacle exists, it may be considered that collision warning is not required, and at this time, the warning level is not classified. Accordingly, case 1) and case 2) shown in embodiment one can be further subdivided into the following cases:
in the process that the vehicle moves for a preset distance, a detection result of a radar sensor indicates that an obstacle exists, and a detection result of a camera also indicates that the obstacle exists;
in the process that the vehicle moves for the preset distance, the detection result of the radar sensor indicates that the obstacle disappears along with the movement of the vehicle, and meanwhile, the detection result of the camera indicates that the obstacle exists;
and thirdly, in the process that the vehicle moves for the preset distance, the detection result of the radar sensor indicates that the obstacle does not exist, and meanwhile, the detection result of the camera indicates that the obstacle exists.
The preset distance may be set with reference to a detection range of the radar sensor, and may be set to be smaller than a total length of the detection range of the radar sensor. The reason why the radar sensor does not detect the obstacle may be: the vehicle is too far from the obstacle; or the vehicle is too close to the obstacle (as described in the background). If the vehicle is moving a long distance, it is likely that the distance between the vehicle and the obstacle is long, and there is practically no risk of collision. Therefore, the embodiment of the invention detects the obstacle with a short vehicle moving distance as a detection cycle, thereby detecting any one of the situations (i) to (iii). The following are discussed separately:
in the case that both the radar sensor and the camera can detect the obstacle all the time, the corresponding early warning level can be high; the situation that the obstacle disappears along with the movement of the vehicle due to the detection blind area of the radar sensor is high in probability, but the situation that the vehicle is far away from the obstacle is not eliminated, so that the corresponding early warning level can be a middle level; the situation that the obstacle is not detected all the time by the radar sensor, and the obstacle is mistakenly recognized probably due to the fact that an error exists when the image recognition is carried out on the picture shot by the camera, so that the corresponding early warning level can be low.
305. The early warning system determines the highlighting degree of the virtual image of the obstacle corresponding to the early warning level.
In the embodiment of the invention, different early warning levels correspond to different highlighting degrees of the virtual image. The specific form of the highlighting may include, but is not limited to: adjusting the color of the virtual image, adjusting the size of the virtual image, and the like. Generally, the higher the warning level, the higher the degree of highlighting of the virtual image.
Taking color as an example, when the early warning level is high, the early warning level can correspond to a red virtual image; when the early warning level is a middle level, the early warning level can correspond to a yellow virtual image; when the early warning level is low, the early warning level can correspond to a green virtual image. The frequency of red, yellow and green colors appearing in the real environment decreases in order, and the degree of highlighting increases as the color of the obstacle contrasts with the real environment. For another example, taking the size of the virtual image as an example, the higher the warning level is, the larger the size of the virtual image is, and the higher the degree of highlighting of the virtual image is.
Further, the higher the highlighting degree of the virtual image is, the more likely the driver is to notice the virtual image of the obstacle, and by observing the color or size of the virtual image, the driver can intuitively know the current warning level to help the driver decide what obstacle avoidance measures to take.
306. The early warning system identifies the image position of the obstacle in the real environment image.
307. And the early warning system displays the virtual image of the barrier in an overlapping manner at the corresponding highlighting degree on the image position.
In the embodiment of the present invention, as an optional implementation manner, the shape of the virtual image of the obstacle may be a preset warning symbol, an outline of the obstacle, or a virtual frame surrounding the obstacle, which is not limited in the embodiments of the present invention. Preferably, the virtual image of the obstacle is superimposed on the image position of the obstacle, so that the obstacle is not blocked when the virtual image of the obstacle is displayed, and the driver can observe the actual situation of the obstacle through the real environment image while seeing the virtual image.
And step 306 to step 307 are executed, after the image position of the obstacle in the real environment image is identified, the virtual image is superimposed and displayed on the image position by using a highlighting mode such as color, size and the like, and meanwhile, the driver can quickly position the obstacle by combining the real environment image, so that the safety of driving auxiliary service provided by the vehicle collision system is enhanced, and the subjective driving experience of the driver is optimized.
In addition, as an optional implementation manner, the early warning system may further determine an alarm sound effect corresponding to the early warning level, and control a speaker of the vehicle to output the corresponding alarm sound effect, so as to prompt the driver to pay attention by sound.
It can be seen that in the method described in fig. 3, the early warning system highlights the virtual image of the obstacle in the real environment image by adjusting the color, the size, and the like, so as to remind the driver of the obstacle. And different early warning grades correspond different highlighting degrees, so that the driver can quickly locate the barrier and know the current specific early warning grade, thereby enhancing the safety of the driving auxiliary service provided by the vehicle collision system and optimizing the subjective driving experience of the driver. In addition, if the shape of the virtual image adopts the outline of the obstacle or a virtual picture frame surrounding the obstacle, the obstacle in the real environment image can not be shielded while the virtual image is displayed in a superposed mode, so that the driver can observe the actual situation of the obstacle through the real environment image while seeing the virtual image.
Furthermore, when the vehicle needs to be backed up, collision early warning is triggered, so that the obstacle detection accuracy during backing up can be improved, and the driving safety during backing up is improved. Furthermore, the method described in fig. 3 performs image recognition on the early warning planning region where the radar sensor detects that the obstacle exists, so that the data processing amount required for image recognition can be reduced, and the reflection speed of the early warning system can be increased.
Example four
Referring to fig. 5, fig. 5 is a schematic structural diagram of a vehicle collision warning system according to an embodiment of the present invention. As shown in fig. 5, the vehicle collision warning system may include:
a first detection unit 501 for performing obstacle detection using a radar sensor and a camera; the camera may be a 360-degree panoramic camera, and the embodiment of the present invention is not limited. Specifically, the obstacle can be extracted from the image captured by the camera through an image recognition algorithm such as deep learning.
A first determining unit 502, configured to determine an early warning level according to a detection result of the radar sensor and a detection result of the camera;
if the detection result of the radar sensor indicates that an obstacle exists and the detection result of the camera also indicates that the obstacle exists, determining the early warning level as high; if the detection result of the radar sensor indicates that no obstacle exists and the detection result of the camera indicates that the obstacle exists, or if the detection result of the radar sensor indicates that the obstacle exists and the detection result of the camera indicates that the obstacle does not exist, determining the early warning level as a middle level; if the detection result of the radar sensor indicates that no obstacle exists and the detection result of the camera indicates that no obstacle exists, determining the early warning level as high;
a second determining unit 503, configured to determine a virtual image of an obstacle corresponding to the early warning level; the virtual images of the obstacles corresponding to different early warning levels are different.
A first display unit 504, configured to output and display a real environment image captured by a camera;
and a second display unit 505, configured to display a virtual image on the real environment image in an overlapping manner.
As an optional implementation manner, the second display unit 505 specifically includes:
an identifying subunit 5051, configured to identify an image position of the obstacle in the real environment image;
an overlay sub-unit 5052 is used for displaying the virtual image in an overlay manner at the above-mentioned image position.
That is, the second display unit 505 may directly mark the position of the obstacle in the real environment image to prompt the driver's attention.
Therefore, by implementing the vehicle collision early warning system shown in fig. 5, the detection blind area of the radar sensor can be supplemented by using the camera, so that the detection blind area of the obstacle is reduced. And according to the difference of the detection results of the radar sensor and the camera on the obstacle, different early warning levels are divided, and different virtual images are correspondingly displayed, so that a driver can quickly identify the current early warning level, and then adopt corresponding obstacle avoidance operation, and the risk of vehicle collision is reduced.
Example four
Referring to fig. 6, fig. 6 is a schematic structural diagram of another vehicle collision warning system disclosed in the embodiment of the present invention. The vehicle collision warning system shown in fig. 6 is obtained by optimizing the vehicle collision warning system shown in fig. 5. In the vehicle collision warning system shown in fig. 6:
the first determining unit 502 is specifically configured to determine the early warning level as high level if the detection result of the radar sensor and the detection result of the camera both indicate that an obstacle exists during the movement of the vehicle by the preset distance; and if the detection result of the radar sensor indicates that the obstacle disappears along with the movement of the vehicle and the detection result of the camera indicates that the obstacle exists in the process of the movement of the vehicle by the preset distance, determining the early warning grade as a middle grade; and in the process that the vehicle moves for the preset distance, if the detection result of the radar sensor indicates that no obstacle exists and the detection result of the camera indicates that the obstacle exists, determining the early warning level as low.
Accordingly, in the vehicle collision warning system shown in fig. 6, the degrees of highlighting of the obstacles are different for different warning levels. The specific form of the highlighting may include, but is not limited to: adjusting the color of the virtual image, adjusting the size of the virtual image, and the like. Generally, the higher the warning level, the higher the degree of highlighting of the virtual image.
Taking color as an example, when the early warning level is high, the early warning level can correspond to a red virtual image; when the early warning level is a middle level, the early warning level can correspond to a yellow virtual image; when the early warning level is low, the early warning level can correspond to a green virtual image. The frequency of red, yellow and green colors appearing in the real environment decreases in order, and the degree of highlighting increases as the color of the obstacle contrasts with the real environment.
Taking the size of the virtual image as an example, the higher the warning level is, the larger the size of the virtual image is, and the higher the highlighting degree of the virtual image is.
Further optionally, the shape of the virtual image of the obstacle may be an outline of the obstacle, or a virtual frame surrounding the obstacle;
that is, the above-mentioned superposition sub-unit 5052 may be specifically configured to outline the obstacle with a conspicuous color at the image position of the obstacle or to circle the obstacle with a virtual frame; or, drawing the outline of the obstacle through different sizes at the image position of the obstacle or circling the obstacle by using a virtual picture frame;
optionally, in the vehicle collision warning system shown in fig. 6, the vehicle collision warning system may further include:
a second detection unit 506 for detecting whether the vehicle is in a reverse gear before the first detection unit 501 detects an obstacle using the radar sensor and the camera; and when it is detected that the vehicle is in the reverse gear, the first detection unit 501 is triggered to perform an operation of detecting an obstacle using the radar sensor and the camera. The operation corresponding to the second detection unit 506 is executed, and the collision early warning can be triggered when the vehicle needs to be backed up, so that the obstacle detection accuracy during backing up is improved, and the driving safety during backing up is improved.
Further optionally, in the vehicle collision warning system shown in fig. 6, the first detecting unit 501 may specifically include:
the first detection subunit 5011 is configured to detect a plurality of pre-divided early warning planning areas around the vehicle by using a radar sensor;
the shooting and shooting subunit 5012 is configured to, when the first detecting subunit 5011 detects that an obstacle exists in a certain early warning planning area, shoot the early warning planning area where the obstacle exists by using the camera;
the second detecting subunit 5013 is configured to detect whether an obstacle exists in the image captured by the camera.
That is to say, image recognition is performed on the early warning planning region in which the radar sensor detects that the obstacle exists, so that the data processing amount required for image recognition can be reduced, and the reflecting speed of the early warning system is increased.
Therefore, by implementing the vehicle collision early warning system shown in fig. 6, the virtual image of the obstacle can be highlighted in the real environment image by adjusting the color, the size and the like to remind the driver of paying attention, so that the driver can quickly locate the obstacle and know the current specific early warning level, the safety of the driving auxiliary service provided by the vehicle collision system is enhanced, and the subjective driving experience of the driver is optimized. In addition, if the shape of the virtual image adopts the outline of the obstacle or a virtual picture frame surrounding the obstacle, the obstacle in the real environment image can not be shielded while the virtual image is displayed in a superposed mode, so that the driver can observe the actual situation of the obstacle through the real environment image while seeing the virtual image. Furthermore, when the vehicle needs to be backed up, collision early warning is triggered, so that the obstacle detection accuracy during backing up can be improved, and the driving safety during backing up is improved. Furthermore, image recognition is performed on the early warning planning area in which the radar sensor detects the existence of the obstacle, so that the data processing amount of the image recognition can be reduced, and the reflecting speed of the early warning system is increased.
EXAMPLE five
Referring to fig. 7, fig. 7 is a schematic structural diagram of another vehicle collision warning system according to an embodiment of the present invention. As shown in fig. 7, the vehicle collision warning system may include:
the system comprises a collision early warning control unit 701, a radar sensor 702, a camera 703, a display 704, a sound alarm module 705 and a power supply module 706. The collision warning control Unit 701 may be an Electronic Control Unit (ECU); the radar sensor 702 and the collision warning control unit 701 can communicate with each other through an Local Interconnect Network (LIN); the camera 703 and the collision warning control unit 701 may communicate with each other through a Low Voltage Differential Signaling (LVDS) interface; the display 704 and the collision warning control unit 701 may be connected through a Push to Talk over cell (Poc) communication; the sound alarm module 705 and the collision warning control unit 701 may communicate with each other through a Controller Area Network (CAN) bus.
The collision early warning control unit 701 may be configured to analyze radar data acquired by the radar sensor 702 to obtain a detection result of the radar sensor detecting an obstacle; the collision early warning control unit 701 acquires visual information acquired by the camera 703 to obtain a detection result obtained by detecting an obstacle by the camera;
the collision early warning control unit 701 may be further configured to determine an early warning level according to a detection result of the radar sensor 702 and a detection result of the camera 703, and determine a virtual image of an obstacle corresponding to the early warning level; optionally, the collision warning control unit 701 may also determine an alarm sound effect corresponding to the warning level; the virtual images of the obstacles corresponding to different early warning levels are different; the alarm sound effects of the barriers corresponding to different early warning levels are different;
a display 704, configured to obtain and output a real environment image captured by the camera 703, and receive a virtual image determined by the collision warning control unit 701, so as to superimpose and display a virtual image of an obstacle on the real environment image;
the sound alarm module 705 is used for outputting an alarm sound effect corresponding to the early warning level;
and the power supply module 706 is used for supplying power to each unit or module included in the vehicle collision warning system so that each unit or module performs corresponding operation.
The embodiment of the invention discloses a vehicle which comprises a vehicle collision early warning system shown in any one of figures 5-7.
An embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to execute any one of the vehicle collision warning methods shown in fig. 2 or fig. 4.
An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute any one of the vehicle collision warning methods shown in fig. 2 or fig. 4.
It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Those skilled in the art should also appreciate that the embodiments described in this specification are exemplary and alternative embodiments, and that the acts and modules illustrated are not required in order to practice the invention.
In various embodiments of the present invention, it should be understood that the sequence numbers of the above-mentioned processes do not imply an inevitable order of execution, and the execution order of the processes should be determined by their functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated units, if implemented as software functional units and sold or used as a stand-alone product, may be stored in a computer accessible memory. Based on such understanding, the technical solution of the present invention, which is a part of or contributes to the prior art in essence, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for causing a computer device (which may be a personal computer, a server, a network device, or the like, and may specifically be a processor in the computer device) to execute part or all of the steps of the above-described method of each embodiment of the present invention.
It will be understood by those skilled in the art that all or part of the steps in the methods of the embodiments described above may be implemented by hardware instructions of a program, and the program may be stored in a computer-readable storage medium, where the storage medium includes Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM), or other Memory, such as a magnetic disk, or a combination thereof, A tape memory, or any other medium readable by a computer that can be used to carry or store data.
The vehicle collision warning method, system, vehicle and storage medium disclosed in the embodiments of the present invention are described in detail above, and the principle and implementation of the present invention are explained herein by applying specific examples, and the description of the above embodiments is only used to help understanding the method and the core idea of the present invention. Meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (12)
1. A vehicle collision warning method is characterized by comprising the following steps:
detecting an obstacle by using a radar sensor and a camera;
determining an early warning level according to the detection result of the radar sensor and the detection result of the camera;
determining a virtual image of the obstacle corresponding to the early warning level;
outputting and displaying a real environment image shot by the camera, and superposing and displaying the virtual image on the real environment image;
the virtual images of the obstacles corresponding to different early warning levels are different;
if the detection result of the radar sensor and the detection result of the camera indicate that an obstacle exists in the process that the vehicle moves for a preset distance, determining the early warning level as high;
if the detection result of the radar sensor indicates that the obstacle disappears along with the movement of the vehicle in the process that the vehicle moves the preset distance, and meanwhile, the detection result of the camera indicates that the obstacle exists, the early warning grade is determined to be a medium grade;
and if the detection result of the radar sensor indicates that no obstacle exists and the detection result of the camera indicates that the obstacle exists in the process that the vehicle moves the preset distance, determining the early warning level as low.
2. The method according to claim 1, wherein said displaying the virtual image superimposed on the real environment image comprises:
identifying an image position of the obstacle in the real environment image;
and superposing and displaying the virtual image on the image position.
3. The method of claim 2, wherein the virtual images of the obstacles are highlighted to different degrees for different warning levels.
4. The method of claim 1, wherein prior to the obstacle detection using a radar sensor and a camera, the method further comprises:
detecting whether the vehicle is in a reverse gear;
and if so, executing the step of detecting the obstacle by using the radar sensor and the camera.
5. The method according to any one of claims 1 to 4, wherein the obstacle detection using a radar sensor and a camera comprises:
detecting a plurality of pre-divided early warning planning areas around the vehicle by using a radar sensor;
if the fact that an obstacle exists in one early warning planning area is detected, shooting the early warning planning area with the obstacle by using a camera;
and detecting whether an obstacle exists in the image shot by the camera.
6. A vehicle collision warning system, comprising:
a first detection unit for performing obstacle detection using a radar sensor and a camera;
the first determining unit is used for determining an early warning level according to the detection result of the radar sensor and the detection result of the camera;
a second determination unit configured to determine a virtual image of the obstacle corresponding to the early warning level;
the first display unit is used for outputting and displaying a real environment image shot by the camera;
the second display unit is used for displaying the virtual image on the real environment image in an overlapping mode;
the virtual images of the obstacles corresponding to different early warning levels are different;
the first determining unit is specifically configured to determine that an early warning level is high if the detection result of the radar sensor and the detection result of the camera both indicate that an obstacle exists in the process that the vehicle moves by a preset distance; and if the detection result of the radar sensor indicates that the obstacle disappears along with the movement of the vehicle and the detection result of the camera indicates that the obstacle exists in the process of the movement of the vehicle by the preset distance, determining the early warning level as a middle level; and in the process that the vehicle moves for the preset distance, if the detection result of the radar sensor indicates that no obstacle exists and the detection result of the camera indicates that the obstacle exists, determining the early warning level as low.
7. The system of claim 6, wherein the second display unit comprises:
the identification subunit is used for identifying the image position of the obstacle in the real environment image;
and the superposition subunit is used for superposing and displaying the virtual image on the image position.
8. The system of claim 7, wherein the pre-warning levels are different for different obstacles.
9. The system of claim 6, further comprising:
a second detection unit configured to detect whether the vehicle is in a reverse gear before the first detection unit detects an obstacle using a radar sensor and a camera; and when the fact that the vehicle is in a reverse gear is detected, triggering the first detection unit to execute the operation of detecting the obstacle by using the radar sensor and the camera.
10. The system according to any one of claims 6 to 9, wherein the first detection unit comprises:
the first detection subunit is used for detecting a plurality of pre-divided early warning planning areas around the vehicle by using a radar sensor;
the shooting subunit is configured to, when the first detection subunit detects that an obstacle exists in a certain early warning planning area, shoot the early warning planning area where the obstacle exists by using a camera;
and the second detection subunit is used for detecting whether an obstacle exists in the image shot by the camera.
11. A vehicle, characterized by comprising: a vehicle collision warning system as claimed in any one of claims 6 to 10.
12. A computer-readable storage medium on which a computer program is stored, characterized in that: the computer program causes a computer to execute the vehicle collision warning method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910582457.0A CN110254349B (en) | 2019-06-28 | 2019-06-28 | Vehicle collision early warning method and system, vehicle and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910582457.0A CN110254349B (en) | 2019-06-28 | 2019-06-28 | Vehicle collision early warning method and system, vehicle and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110254349A CN110254349A (en) | 2019-09-20 |
| CN110254349B true CN110254349B (en) | 2021-02-19 |
Family
ID=67923397
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910582457.0A Active CN110254349B (en) | 2019-06-28 | 2019-06-28 | Vehicle collision early warning method and system, vehicle and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110254349B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111427331B (en) * | 2020-03-24 | 2022-03-04 | 新石器慧通（北京）科技有限公司 | Perception information display method and device of unmanned vehicle and electronic equipment |
| CN113734048A (en) * | 2020-05-29 | 2021-12-03 | 广州汽车集团股份有限公司 | Backing warning method and device |
| CN111994064A (en) * | 2020-08-31 | 2020-11-27 | 中国第一汽车股份有限公司 | Vehicle control method, device, equipment, system and storage medium |
| CN114194108A (en) * | 2020-09-16 | 2022-03-18 | 宝能汽车集团有限公司 | Safety early warning device and method for vehicle and vehicle |
| CN112105113A (en) * | 2020-11-05 | 2020-12-18 | 宁波市奉化浩轩光电有限公司 | LED light-emitting system for flashlight and light-emitting method thereof |
| CN112406707B (en) * | 2020-11-24 | 2022-10-21 | 上海高德威智能交通系统有限公司 | Vehicle early warning method, vehicle, device, terminal and storage medium |
| CN113682259B (en) * | 2021-09-22 | 2023-07-04 | 海南大学 | Door opening early warning anti-collision system for vehicle and control method |
| CN114063613A (en) * | 2021-10-25 | 2022-02-18 | 浙江小族智能科技有限公司 | Amusement vehicle and control method and device thereof, storage medium and terminal |
| CN115092136B (en) * | 2022-07-27 | 2023-09-12 | 广州小鹏自动驾驶科技有限公司 | Vehicle speed planning method and device, vehicle and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104627105A (en) * | 2015-03-05 | 2015-05-20 | 江苏新安电器有限公司 | Event-driven-based anti-collision device and event-driven-based anti-collision method for automobile |
| JP2019046251A (en) * | 2017-09-04 | 2019-03-22 | 株式会社デンソーテン | Target detection equipment, driving assist system, and target detection method |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6882287B2 (en) * | 2001-07-31 | 2005-04-19 | Donnelly Corporation | Automotive lane change aid |
| DE102004062459A1 (en) * | 2004-10-13 | 2006-04-27 | Daimlerchrysler Ag | Method and device for avoiding collisions when opening vehicle doors |
| JP2011118482A (en) * | 2009-11-30 | 2011-06-16 | Fujitsu Ten Ltd | In-vehicle device and recognition support system |
| JP6481846B2 (en) * | 2014-03-27 | 2019-03-13 | 日本精機株式会社 | Vehicle alarm device |
| CN204821326U (en) * | 2015-07-03 | 2015-12-02 | 河南工业技术研究院 | Car microwave radar anticollision early warning system based on video fuses |
| CN204895296U (en) * | 2015-08-19 | 2015-12-23 | 天津中环创新科技有限公司 | Driving obstacle target alarm system |
| CN106291535B (en) * | 2016-07-21 | 2018-12-28 | 触景无限科技（北京）有限公司 | A kind of obstacle detector, robot and obstacle avoidance system |
-
2019
- 2019-06-28 CN CN201910582457.0A patent/CN110254349B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104627105A (en) * | 2015-03-05 | 2015-05-20 | 江苏新安电器有限公司 | Event-driven-based anti-collision device and event-driven-based anti-collision method for automobile |
| JP2019046251A (en) * | 2017-09-04 | 2019-03-22 | 株式会社デンソーテン | Target detection equipment, driving assist system, and target detection method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110254349A (en) | 2019-09-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110254349B (en) | Vehicle collision early warning method and system, vehicle and storage medium | |
| CN109455177B (en) | Automatic parking safety distance warning method and vehicle-mounted terminal | |
| US10102438B2 (en) | Information display device | |
| JP4933658B2 (en) | In-vehicle image display device | |
| US11321911B2 (en) | Method for representing the surroundings of a vehicle | |
| EP3886075A1 (en) | Parking space detection method and device | |
| JP5177515B2 (en) | Peripheral recognition support system | |
| US20070053551A1 (en) | Driving support apparatus | |
| US20080198226A1 (en) | Image Processing Device | |
| US20110228980A1 (en) | Control apparatus and vehicle surrounding monitoring apparatus | |
| KR20020066214A (en) | Picture synthesizing apparatus | |
| JP2009040272A (en) | Vehicle periphery monitoring device | |
| CN110831818B (en) | Parking assist method and parking assist device | |
| KR101729486B1 (en) | Around view monitor system for detecting blind spot and method thereof | |
| US20210300402A1 (en) | Method and Control Device for Warning a Driver of a Motor Vehicle and Motor Vehicle with Such a Control Device | |
| CN110533945B (en) | Early warning method and system of traffic signal lamp, vehicle and storage medium | |
| US20180162274A1 (en) | Vehicle side-rear warning device and method using the same | |
| JP4795281B2 (en) | Vehicle safety device | |
| JP2018198071A (en) | On-vehicle image display device | |
| JP2009117978A (en) | Vehicle surroundings display device | |
| CN114298908A (en) | Obstacle display method and device, electronic equipment and storage medium | |
| KR20220073535A (en) | Parking assistance apparatus and method for vehicle | |
| JP2017076431A (en) | Vehicle control device | |
| JP6372556B2 (en) | In-vehicle image display device | |
| JP2014078271A (en) | On-vehicle image display device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP02 | Change in the address of a patent holder |
Address after: 510000 No.8 Songgang street, Cencun, Tianhe District, Guangzhou City, Guangdong Province Patentee after: GUANGZHOU XPENG AUTOMOBILE TECHNOLOGY Co.,Ltd. Address before: Room 245, No. 333, jiufo Jianshe Road, Zhongxin Guangzhou Knowledge City, Guangzhou City, Guangdong Province Patentee before: GUANGZHOU XPENG AUTOMOBILE TECHNOLOGY Co.,Ltd. |
|
| CP02 | Change in the address of a patent holder |