CN112489425A - Vehicle anti-collision early warning method and device, vehicle-mounted terminal equipment and storage medium - Google Patents
Vehicle anti-collision early warning method and device, vehicle-mounted terminal equipment and storage medium Download PDFInfo
- Publication number
- CN112489425A CN112489425A CN202011337072.7A CN202011337072A CN112489425A CN 112489425 A CN112489425 A CN 112489425A CN 202011337072 A CN202011337072 A CN 202011337072A CN 112489425 A CN112489425 A CN 112489425A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- early warning
- data
- collision
- driver
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0125—Traffic data processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0137—Measuring and analyzing of parameters relative to traffic conditions for specific applications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Chemical & Material Sciences (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Analytical Chemistry (AREA)
- Traffic Control Systems (AREA)
Abstract
The application is suitable for the technical field of artificial intelligence, and provides a vehicle anti-collision early warning method and device, vehicle-mounted terminal equipment and a storage medium. The vehicle anti-collision early warning method comprises the following steps: acquiring state data of a vehicle driver and driving data of the vehicle; inputting the state data and the driving data into a pre-trained anti-collision early warning model, and predicting whether the vehicle is likely to collide or not through the anti-collision early warning model, wherein the anti-collision early warning model is a neural network model obtained by training with driver state data and vehicle driving data corresponding to the vehicle when the vehicle collides as a training set; and if the vehicle is likely to collide, determining an early warning mode according to the state data, and early warning the vehicle driver according to the early warning mode. By adopting the vehicle anti-collision early warning method, a proper early warning mode can be formulated according to the state of a driver, and the safety of vehicle driving is further improved.
Description
Technical Field
The application belongs to the technical field of artificial intelligence, and particularly relates to a vehicle anti-collision early warning method and device, vehicle-mounted terminal equipment and a storage medium.
Background
With the continuous increase of the vehicle inventory, great potential safety hazards are brought when people go out conveniently, and endless traffic accidents seriously threaten the life and property safety of the people. Aiming at the problem, a plurality of vehicles can be provided with vehicle anti-collision early warning systems on vehicle-mounted terminal equipment, and the system can provide services such as driving monitoring, danger early warning and the like when a driver drives the vehicle.
Specifically, through the vehicle anti-collision early warning system, a sound or visual alarm can be given to a driver according to data such as distance and vehicle speed before the vehicle is about to collide, so that possible collision events are reminded. However, sometimes the driver may be aware of the danger, and the warning issued by the pre-crash warning system may be distracting to the driver, resulting in a collision accident.
Disclosure of Invention
In view of this, the present application provides a vehicle anti-collision warning method and apparatus, a vehicle-mounted terminal device, and a storage medium, which can formulate a suitable warning mode according to a state of a driver, so as to further improve the safety of vehicle driving.
In a first aspect, an embodiment of the present application provides a vehicle anti-collision early warning method, including:
acquiring state data of a vehicle driver and driving data of the vehicle;
inputting the state data and the driving data into a pre-trained anti-collision early warning model, and predicting whether the vehicle is likely to collide or not through the anti-collision early warning model, wherein the anti-collision early warning model is a neural network model obtained by training with driver state data and vehicle driving data corresponding to the vehicle when the vehicle collides as a training set;
and if the vehicle is likely to collide, determining an early warning mode according to the state data, and early warning the vehicle driver according to the early warning mode.
According to the anti-collision early warning method and the anti-collision early warning device, the state data of the vehicle driver and the running data of the vehicle are input into the designed anti-collision early warning model, whether the vehicle is likely to collide can be predicted, and when the vehicle is predicted to be likely to collide, the corresponding early warning mode can be determined according to the state of the driver, and then the vehicle driver is early warned according to the determined early warning mode, so that the driving safety of the vehicle is further improved.
In one embodiment of the present application, the acquiring of the state data of the vehicle driver may include:
acquiring a head image of the vehicle driver through a camera;
inputting the head image into a pre-trained head posture estimation model, and obtaining head posture data of the vehicle driver through the head posture estimation model, wherein the head posture estimation model determines the head posture data by detecting key points of human faces in the head image;
and acquiring the eye movement track data of the vehicle driver through an eye tracker.
The head image of the vehicle driver can be shot through a camera arranged in the carriage, and then the obtained head image is input into a head posture estimation model which is trained in advance, and the head posture estimation model can determine the head posture data of the driver by detecting key points of the human face in the head image. The eye tracker is a device for recording eye movement track characteristics of a person when processing visual information, and can be used for collecting eye movement track data of a vehicle driver so as to determine a watching area of the vehicle driver more accurately.
Further, the determining an early warning mode according to the state data may include:
determining a gaze area of the vehicle driver from the head pose data and the eye movement trajectory data;
and determining a corresponding early warning mode according to the gazing area and the vehicle collision position predicted by the anti-collision early warning model.
Studies have shown that a person's prediction of gaze comes from a combination of head pose and eye direction. Because the eye fixation point is collected by the eye tracker, the fixation area judgment with higher precision can be completed by combining the eye fixation coordinate data after the judgment of the head posture is completed. After the gaze area of the driver is determined, the corresponding early warning mode can be determined according to the relative relationship between the vehicle collision position predicted by the model and the gaze area.
Further, determining a gaze region of the vehicle driver based on the head pose data and the eye movement trajectory data may include:
calculating a maximum view range area of the vehicle driver in front of the vehicle according to the head posture data;
and positioning the gaze region from the maximum visual field range region in combination with the eye movement trajectory data.
The eye watching approximate range of the human face model can be obtained based on the head posture and the position information of the key points of the eyes, so that the maximum visual field range area of the driver is obtained, and the watching area of the driver at the moment can be calculated by combining the eye movement track data for relocation.
Further, determining a corresponding early warning mode according to the gazing area and the collision position predicted by the anti-collision early warning model may include:
if the watching area continuously covers the collision position within a preset time length, determining that the early warning mode is a first mode;
if the watching area is switched back and forth between the collision position and other positions within a preset time, determining that the early warning mode is a second mode;
and if the gazing area does not cover the collision position, determining that the early warning mode is a third mode.
According to the position relation between the watching area and the collision position, the method can be used for judging whether a driver is aware of the imminent collision danger and the collision position, further respectively determining the corresponding early warning modes, and can effectively improve the safety and flexibility of anti-collision early warning.
Further, the warning the vehicle driver according to the warning mode may include:
if the early warning mode is the first mode, no early warning prompt in any form is output;
if the early warning mode is the second mode, projecting preset early warning information to the front glass of the vehicle through a projector;
and if the early warning mode is the third mode, projecting preset early warning information to the front glass of the vehicle through a projector, and controlling a buzzer of the vehicle to play warning sound.
The first mode is a mild early warning mode, and at the moment, simple early warning information can be displayed on a display screen of the vehicle-mounted terminal only, or early warning prompts in any form are not output. The second mode is a moderate early warning mode, and preset early warning information can be projected to the front glass of the vehicle through a projector arranged in the vehicle cabin at the moment and is used for indicating the time and the collision position of possible collision. The third mode is the severe early warning mode, can be through the preset early warning information of projecting to the glass before the carriage projection that sets up this moment to control appointed buzzer broadcast warning sound, in order to remind the driver to be about to the danger of collision.
In an embodiment of the present application, the anti-collision early warning model may be obtained by training in the following manner:
acquiring sample data, wherein the sample data comprises corresponding driver state data and vehicle running data when a vehicle collides;
and inputting the sample data into an automatic machine learning module to design and train a model, so as to obtain the anti-collision early warning model.
When sample data of the training anti-collision early warning model is obtained, a simulator can be adopted to simulate a real driving environment, different collision scenes are set, such as possible conditions of rear-end collision of vehicles, road crossing of pedestrians, side collision and the like, and the reaction of drivers under different conditions is recorded. When the system predicts that a collision is possible at a later moment, the system records the watching coordinate data of the driver in a certain time period before the collision, the driving data (the transverse and longitudinal speed and the acceleration of the vehicle) in the time period and the heart rate data of the driver, and judges the driving state of the driver in the time period, wherein the driving state comprises a watching area and activity degree, self excitation degree, driving speed and the like. Next, the collected data is input into an automatic machine learning (AutoML) module for model design and training.
In a second aspect, an embodiment of the present application provides a vehicle anti-collision early warning device, including:
the data acquisition module is used for acquiring state data of a vehicle driver and driving data of the vehicle;
the collision prediction module is used for inputting the state data and the driving data into a pre-trained collision prevention early warning model, and predicting whether the vehicle is likely to collide or not through the collision prevention early warning model, wherein the collision prevention early warning model is a neural network model obtained by training with corresponding driver state data and vehicle driving data as training sets when the vehicle collides;
and the early warning module is used for determining an early warning mode according to the state data if the vehicle is likely to collide, and early warning the vehicle driver according to the early warning mode.
In a third aspect, an embodiment of the present application provides an in-vehicle terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the vehicle anti-collision warning method as set forth in the first aspect of the embodiment of the present application when executing the computer program.
In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the steps of the vehicle anti-collision warning method as set forth in the first aspect of the present application.
In a fifth aspect, the present application provides a computer program product, which when running on a terminal device, causes the terminal device to execute the steps of the vehicle anti-collision warning method according to the first aspect of the present application.
The advantageous effects achieved by the second aspect to the fifth aspect described above can be referred to the description of the first aspect described above.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a flowchart of a first embodiment of a vehicle anti-collision warning method according to an embodiment of the present application;
fig. 2 is a flowchart of a vehicle anti-collision warning method according to a second embodiment of the present disclosure;
FIG. 3 is a block diagram of an embodiment of a vehicle pre-crash warning apparatus according to an embodiment of the present disclosure;
fig. 4 is a schematic diagram of a vehicle-mounted terminal device provided in an embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail. Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.
The application provides a vehicle anti-collision early warning method and device, vehicle-mounted terminal equipment and a storage medium, which can make a proper early warning mode according to the state of a driver, and further improve the safety of vehicle driving. It should be understood that the main subjects of execution of the vehicle anti-collision warning method proposed in the embodiments of the present application are various types of vehicle-mounted terminal devices.
Referring to fig. 1, a first embodiment of a vehicle anti-collision warning method in the embodiment of the present application includes:
101. acquiring state data of a vehicle driver and driving data of the vehicle;
first, the in-vehicle terminal device acquires state data of the driver of the vehicle and travel data of the vehicle. The state data of the vehicle driver can comprise various data such as head posture data, eye movement track data, hand movement data, human heart rate data and the like; the driving data of the vehicle may include various data such as a driving speed, a driving direction, a vehicle position, lateral acceleration data, steering wheel state data, and in-vehicle infrared distance measurement data.
Specifically, the in-vehicle terminal device may acquire travel data of the vehicle, such as travel speed and acceleration, by docking an Electronic Control Unit (ECU) of the vehicle. In addition, the head posture of the driver can be acquired through a camera or other devices arranged in the carriage, the eye movement track data can be acquired through an eye tracker (a device for recording eye movement track characteristics of a person when the person processes visual information), the human body heart rate data of the driver can be acquired through a wearable device, and the like. Data acquired by different devices such as the camera, the eye tracker and the wearable device are all sent to the vehicle-mounted terminal device to be processed.
102. Inputting the state data and the driving data into a pre-trained anti-collision early warning model, and predicting whether the vehicle is likely to collide or not through the anti-collision early warning model;
after acquiring the driver state data and the driving data of the vehicle, the vehicle-mounted terminal equipment inputs the data into a pre-trained anti-collision early warning model, and outputs the result of whether the vehicle is likely to collide or not through the model. Specifically, the anti-collision early warning model is a neural network model obtained by training with driver state data and vehicle running data corresponding to the vehicle in collision as a training set, and the model can be obtained by training in the following way:
(1) acquiring sample data, wherein the sample data comprises corresponding driver state data and vehicle running data when a vehicle collides;
(2) and inputting the sample data into an automatic machine learning module to design and train a model, so as to obtain the anti-collision early warning model.
When sample data of the training anti-collision early warning model is obtained, a simulator can be adopted to simulate a real driving environment, different collision scenes are set, such as possible conditions of rear-end collision of vehicles, road crossing of pedestrians, side collision and the like, and the reaction of drivers under different conditions is recorded. When the system predicts that a collision is possible at a later moment, the system records the watching coordinate data of the driver in a certain time period before the collision, the driving data (the transverse and longitudinal speed and the acceleration of the vehicle) in the time period and the heart rate data of the driver, and judges the driving state of the driver in the time period, wherein the driving state comprises a watching area and activity degree, self excitation degree, driving speed and the like. Specifically, a multi-camera in the vehicle can be used for acquiring a head or face gesture image of the driver, an eye tracker is used for acquiring eye movement track data or gaze mode data of the driver, wrist wearable equipment is used for acquiring heart rate data of the driver, and various driving data of the vehicle are acquired by butting an ECU of the vehicle.
Next, the collected data is input into an automatic machine learning (AutoML) module for model design and training. The traditional anti-collision early warning signal is released mostly by means of certain established formulas, a limit braking distance is calculated by means of the speed of driving, distance data of infrared detection and the fixation coordinate data of a driver, and the judgment of warning release is completed by comparing a set braking distance threshold. The present application differs from this: a variety of different data may be added, including facial pose of the driver (to aid in determining the gaze area), gaze point coordinates (to determine the gaze area), dwell time of each gaze coordinate (to determine the degree of attention), heart rate (to confirm driver excitement, roughly proportional to the degree of attention), and driving speed, vehicle distance, etc. Therefore, the method and the device introduce an automatic continuous learning (AutoML) technology to carry out model training, and therefore the problem that a single machine learning algorithm is difficult to adapt to multi-source heterogeneous data is solved. The sample data is subjected to standardization processing in the AutoML layer, model training is conducted by using whether collision is avoided or not as result guidance, and a training set and a test set are divided by collecting simulation training data of large-scale and multi-type drivers, so that training of the anti-collision early warning model is completed.
103. And if the vehicle is likely to collide, determining an early warning mode according to the state data, and early warning the vehicle driver according to the early warning mode.
Through the anti-collision early warning model, whether the vehicle is likely to collide within a certain time can be predicted. If the vehicle does not have the collision danger, the vehicle-mounted terminal equipment does not need to perform early warning. If the vehicle is predicted to have collision danger, the vehicle-mounted terminal equipment determines a corresponding early warning mode according to the state data of the driver and carries out early warning on the vehicle driver according to the determined early warning mode. For example, if it is detected that the driver is looking at the location of the imminent collision and the heart rate of the driver is high, indicating that the driver is aware of the danger, a lower degree of warning mode may be used (e.g., a danger indicator light is on, or warning information is output); if the driver is detected not to observe the position where the collision is about to occur, the heart rate of the driver is mild, or the driver is in a state of getting trapped, a higher-degree early warning mode is adopted (for example, a high-decibel loudspeaker plays early warning information).
According to the anti-collision early warning method and the anti-collision early warning device, the state data of the vehicle driver and the running data of the vehicle are input into the designed anti-collision early warning model, whether the vehicle is likely to collide can be predicted, and when the vehicle is predicted to be likely to collide, the corresponding early warning mode can be determined according to the state of the driver, and then the vehicle driver is early warned according to the determined early warning mode, so that the driving safety of the vehicle is further improved.
Referring to fig. 2, a second embodiment of a vehicle anti-collision warning method in the embodiment of the present application includes:
201. acquiring state data of a vehicle driver and driving data of the vehicle;
in an embodiment of the present application, the acquiring the status data of the vehicle driver includes:
(1) acquiring a head image of the vehicle driver through a camera;
(2) inputting the head image into a pre-trained head posture estimation model, and obtaining head posture data of the vehicle driver through the head posture estimation model, wherein the head posture estimation model determines the head posture data by detecting key points of human faces in the head image;
(3) and acquiring the eye movement track data of the vehicle driver through an eye tracker.
The head image of the vehicle driver can be shot through a camera arranged in the carriage, and then the obtained head image is input into a head posture estimation model which is trained in advance, and the head posture estimation model can determine the head posture data of the driver by detecting key points of the human face in the head image. The head gestures can be divided into relative stillness and movement from the motion state, and the movement can be further divided into multiple action modes such as head raising, head turning, head shaking and the like. The use of the system can be divided into two aspects, including detection of fatigue state of the driver (e.g., determination of fatigue driving in accordance with a long-time head still state and blinking frequency) and assistance in performing higher-precision gaze tracking (for determining a gaze region of the driver).
The head posture estimation model is mainly constructed by detecting a plurality of 2D key points (including key points such as eye corners, nose tips, mouth corners and chin, the number of the key points is different according to algorithms, the multiple detection points can bring higher precision, but the calculated amount is increased), matching and fitting the 3D face model with the highest degree of fitting based on the face image (a plurality of face models can be used for matching and fitting in the existing algorithm model), taking the 3D face model as the head posture judgment model of the driver, and then solving the conversion relation between the 3D points and the corresponding acquired 2D points, so that 3 different Euler angles of a pitch angle, a yaw angle and a roll angle are calculated, and the two points respectively correspond to head raising, head turning and head shaking actions. For example, if the model finds that the yaw angle has a large variation within a certain period and exceeds a set variation threshold, it is determined that the driver has a large turning motion, and the corresponding head posture is "in turning motion".
The eye tracker is a device for recording eye movement track characteristics of a person when processing visual information, and can be used for collecting eye movement track data of a vehicle driver so as to determine a watching area of the vehicle driver more accurately. In practical application, the eye tracker may be disposed at a designated position inside a vehicle cabin, and after the eye tracker collects the eye movement trajectory data of the vehicle driver, the data is sent to the vehicle-mounted terminal device for subsequent processing.
202. Inputting the state data and the driving data into a pre-trained anti-collision early warning model, and predicting whether the vehicle is likely to collide or not through the anti-collision early warning model;
and inputting state data containing head posture data and eye movement track data and driving data of the vehicle into a pre-collision warning model, and predicting whether the vehicle is likely to collide or not through the model. For the training process and the working principle of the pre-crash warning model, reference may be made to the related description of the previous embodiment.
203. If the vehicle is likely to collide, determining a gazing area of the vehicle driver according to the head posture data and the eye movement track data;
if the vehicle is predicted to be possibly collided, the vehicle-mounted terminal equipment determines the current watching area of the vehicle driver according to the head posture data and the eye movement track data of the vehicle driver. Studies have shown that a person's prediction of gaze comes from a combination of head pose and eye direction. Because the eye fixation point is collected by the eye tracker, the fixation area judgment with higher precision can be completed by combining the eye fixation coordinate data after the judgment of the head posture is completed.
Further, determining a gaze area of the vehicle driver from the head pose data and the eye movement trajectory data may include:
(1) calculating a maximum view range area of the vehicle driver in front of the vehicle according to the head posture data;
(2) and positioning the gaze region from the maximum visual field range region in combination with the eye movement trajectory data.
Specifically, a calculation model of the vehicle front view range data corresponding to different head postures can be completed according to the human eye view region data in the real scene collected in advance, and then based on each deflection angle calculated in the head postures, namely a pitch angle, a yaw angle and a roll angle, the calculation model is used for calculating to obtain the maximum view range region or the central watching region of the driver in front of the vehicle (including important attention regions such as the front of the vehicle and a rearview mirror) in the current head posture. The eye watching approximate range of the 3D face model can be obtained based on the head posture and the position information of the key points of the eyes, so that the maximum visual field range area of the driver is obtained, and the watching area of the driver at the moment can be obtained by combining the eye movement track data for relocation.
204. Determining a corresponding early warning mode according to the watching area and the vehicle collision position predicted by the anti-collision early warning model;
through the anti-collision early warning model, whether the vehicle is likely to collide or not can be predicted, and when the vehicle is predicted to be likely to collide, the possible collision position of the vehicle can be further predicted. After the gaze area of the driver is determined, the corresponding early warning mode can be determined according to the relative relationship between the vehicle collision position predicted by the model and the gaze area.
Specifically, step 204 may include:
(1) if the watching area continuously covers the collision position within a preset time length, determining that the early warning mode is a first mode;
(2) if the watching area is switched back and forth between the collision position and other positions within a preset time, determining that the early warning mode is a second mode;
(3) and if the gazing area does not cover the collision position, determining that the early warning mode is a third mode.
For the step (1), if the gaze region continuously covers the collision position predicted by the model within the preset time length, it indicates that the eye gaze point of the driver is concentrated on the position where the collision is likely to occur for a long time before the early warning, at this time, it can be determined that the driver is aware of the danger of the collision, and knows the possible collision position, and at this time, the driver enters the first early warning mode. Since the driver is aware of the danger and should not be given a heavy warning at this time to avoid distracting the driver, the first warning mode may be a light warning mode. In addition, when the early warning mode is judged, data such as the heart rate of the driver and the vehicle speed can be combined, for example, when the condition that the watching area continuously covers the collision position is detected, the condition that the heart rate of the driver is higher and the vehicle speed is reduced is further detected, and then the driver is judged to enter the first early warning mode.
For step (2), if the gaze region is switched back and forth between the collision position and other positions within the preset time period, indicating that the driver may be aware of the danger but does not know the possible collision position, the second warning mode is entered. The second early warning mode can be a moderate early warning mode, and when the early warning mode is judged, data such as the heart rate of a driver and the vehicle speed can be combined, for example, if the head posture of the driver is detected to be frequently changed, the heart rate of the driver is high, and the vehicle speed is reduced, the driver is judged to enter the second early warning mode.
For the step (3), if the gazing area does not cover the collision position, the driver is not aware of the danger, and then a third early warning mode is entered. Since the driver is unaware of the danger and needs to perform a high warning at this time, the third warning mode may be a heavy warning mode. In addition, when the early warning mode is judged, data such as the heart rate of the driver and the vehicle speed can be combined, for example, if the head posture of the driver is detected to be basically unchanged, the heart rate of the driver is stable, and the vehicle speed is not changed, the driver is judged to enter the third early warning mode.
205. And early warning is carried out on the vehicle driver according to the early warning mode.
And after the corresponding early warning mode is determined, early warning can be performed on the vehicle driver according to the corresponding early warning mode.
Specifically, step 205 may include:
(1) if the early warning mode is the first mode, no early warning prompt in any form is output;
(2) if the early warning mode is the second mode, projecting preset early warning information to the front glass of the vehicle through a projector;
(3) and if the early warning mode is the third mode, projecting preset early warning information to the front glass of the vehicle through a projector, and controlling a buzzer of the vehicle to play warning sound.
The first mode is a mild early warning mode, and at the moment, simple early warning information can be displayed on a display screen of the vehicle-mounted terminal only, or early warning prompts in any form are not output. The second mode is a moderate early warning mode, and preset early warning information can be projected to the front glass of the vehicle through a projector arranged in the vehicle cabin at the moment and is used for indicating the time and the collision position of possible collision. The third mode is the severe early warning mode, can be through the preset early warning information of projecting to the glass before the carriage projection that sets up this moment to control appointed buzzer broadcast warning sound, in order to remind the driver to be about to the danger of collision.
The embodiment of the application reforms transform traditional vehicle anticollision early warning system, collects driver's eye activity data, collects facial gesture through camera equipment, collects physiological data through wearing equipment etc. through adding the eye detector to carry out the model training to all kinds of data of collecting through the automatic machine learning method, realize more intelligent vehicle early warning system. The system can correspondingly send out appropriate early warning information according to different watching conditions of drivers, and effectively avoids traffic accidents caused by distraction due to inappropriate early warning.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Corresponding to the vehicle pre-warning method in the foregoing embodiment, fig. 3 shows a structural block diagram of a vehicle pre-warning device provided in the embodiment of the present application, and for convenience of description, only the relevant portions of the embodiment of the present application are shown.
Referring to fig. 3, the apparatus includes:
a data acquisition module 301 for acquiring status data of a driver of a vehicle and driving data of the vehicle;
a collision prediction module 302, configured to input the state data and the driving data into a pre-trained anti-collision early warning model, and predict whether the vehicle is likely to collide through the anti-collision early warning model, where the anti-collision early warning model is a neural network model trained by using driver state data and vehicle driving data corresponding to the vehicle when the vehicle collides as a training set;
and the early warning module 303 is configured to determine an early warning mode according to the state data if the vehicle is likely to collide, and perform early warning on the vehicle driver according to the early warning mode.
Further, the data acquisition module may include:
the head image acquisition unit is used for acquiring a head image of the vehicle driver through a camera;
the head pose estimation unit is used for inputting the head image into a head pose estimation model which is trained in advance, and obtaining head pose data of the vehicle driver through the head pose estimation model, wherein the head pose estimation model determines the head pose data by detecting key points of a human face in the head image;
and the eye movement track acquisition unit is used for acquiring the eye movement track data of the vehicle driver through an eye tracker.
Further, the early warning module may include:
a gaze region determination unit for determining a gaze region of the vehicle driver from the head pose data and the eye movement trajectory data;
and the early warning mode determining unit is used for determining a corresponding early warning mode according to the watching region and the vehicle collision position predicted by the anti-collision early warning model.
Still further, the gaze area determination unit may include:
a maximum visual field range area determining subunit, configured to calculate, according to the head posture data, a maximum visual field range area of the vehicle driver in front of the vehicle;
and the gazing area determining subunit is used for combining the eye movement track data to obtain the gazing area from the maximum visual field range area in a positioning mode.
Further, the early warning mode determination unit may include:
a first mode determining subunit, configured to determine that the early warning mode is a first mode if the gaze area continuously covers the collision location within a preset duration;
the second mode determining subunit is configured to determine that the early warning mode is the second mode if the gaze region is switched back and forth between the collision position and another position within a preset duration;
and the third mode determining subunit is configured to determine that the early warning mode is the third mode if the gaze area does not cover the collision location.
Further, the early warning module may include:
the first early warning unit is used for not outputting any form of early warning prompt if the early warning mode is the first mode;
the second early warning unit is used for projecting preset early warning information to the front glass of the vehicle through a projector if the early warning mode is the second mode;
and the third early warning unit is used for projecting preset early warning information to the front glass of the vehicle through a projector if the early warning mode is the third mode, and controlling a buzzer of the vehicle to play warning sound.
Further, the vehicle anti-collision warning device may further include:
the system comprises a sample data acquisition module, a data processing module and a data processing module, wherein the sample data acquisition module is used for acquiring sample data, and the sample data comprises corresponding driver state data and vehicle running data when a vehicle collides;
and the early warning model training module is used for inputting the sample data into an automatic machine learning module to design and train a model so as to obtain the anti-collision early warning model.
Embodiments of the present application further provide a computer-readable storage medium, which stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of any one of the vehicle anti-collision warning methods shown in fig. 1 or fig. 2 are implemented.
The embodiment of the present application further provides a computer program product, which when running on a server, causes the server to execute the steps of implementing any one of the vehicle anti-collision warning methods as shown in fig. 1 or fig. 2.
The embodiment of the application also provides a vehicle-mounted terminal device, which comprises a memory, a processor and computer readable instructions stored in the memory and executable on the processor, wherein the processor executes the computer readable instructions to implement the steps of any one of the vehicle anti-collision warning methods shown in fig. 1 or fig. 2.
Fig. 4 is a schematic diagram of a vehicle-mounted terminal device according to an embodiment of the present application. As shown in fig. 4, the in-vehicle terminal apparatus 4 of the embodiment includes: a processor 40, a memory 41, and computer readable instructions 42 stored in the memory 41 and executable on the processor 40. The processor 40, when executing the computer readable instructions 42, implements the steps in the various vehicle pre-crash warning method embodiments described above, such as steps 101-103 shown in fig. 1. Alternatively, the processor 40, when executing the computer readable instructions 42, implements the functions of the modules/units in the above device embodiments, such as the functions of the modules 301 to 303 shown in fig. 3.
Illustratively, the computer readable instructions 42 may be partitioned into one or more modules/units that are stored in the memory 41 and executed by the processor 40 to accomplish the present application. The one or more modules/units may be a series of computer-readable instruction segments capable of performing specific functions, which are used for describing the execution process of the computer-readable instructions 42 in the in-vehicle terminal device 4.
The vehicle-mounted terminal device 4 may be a computing device such as a smart phone, a notebook, a palm computer, a cloud vehicle-mounted terminal device, and the like. The vehicle-mounted terminal device 4 may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will appreciate that fig. 4 is only an example of the in-vehicle terminal device 4, and does not constitute a limitation to the in-vehicle terminal device 4, and may include more or less components than those shown, or combine some components, or different components, for example, the in-vehicle terminal device 4 may further include an input-output device, a network access device, a bus, and the like.
The Processor 40 may be a CentraL Processing Unit (CPU), other general purpose Processor, a DigitaL SignaL Processor (DSP), an AppLication Specific Integrated Circuit (ASIC), an off-the-shelf ProgrammabLe Gate Array (FPGA) or other ProgrammabLe logic device, discrete Gate or transistor logic device, discrete hardware component, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 41 may be an internal storage unit of the in-vehicle terminal apparatus 4, such as a hard disk or a memory of the in-vehicle terminal apparatus 4. The memory 41 may also be an external storage device of the in-vehicle terminal device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure DigitaL (SD) Card, a FLash memory Card (FLash Card), or the like provided on the in-vehicle terminal device 4. Further, the memory 41 may also include both an internal storage unit and an external storage device of the in-vehicle terminal device 4. The memory 41 is used for storing the computer-readable instructions and other programs and data required by the in-vehicle terminal device. The memory 41 may also be used to temporarily store data that has been output or is to be output.
It should be noted that, for the information interaction, execution process, and other contents between the above-mentioned devices/units, the specific functions and technical effects thereof are based on the same concept as those of the embodiment of the method of the present application, and specific reference may be made to the part of the embodiment of the method, which is not described herein again.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium and can implement the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing apparatus/terminal apparatus, a recording medium, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a usb-disk, a removable hard disk, a magnetic or optical disk, etc.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A vehicle anti-collision early warning method is characterized by comprising the following steps:
acquiring state data of a vehicle driver and driving data of the vehicle;
inputting the state data and the driving data into a pre-trained anti-collision early warning model, and predicting whether the vehicle is likely to collide or not through the anti-collision early warning model, wherein the anti-collision early warning model is a neural network model obtained by training with driver state data and vehicle driving data corresponding to the vehicle when the vehicle collides as a training set;
and if the vehicle is likely to collide, determining an early warning mode according to the state data, and early warning the vehicle driver according to the early warning mode.
2. The vehicle pre-crash warning method as set forth in claim 1, wherein the acquiring the status data of the vehicle driver comprises:
acquiring a head image of the vehicle driver through a camera;
inputting the head image into a pre-trained head posture estimation model, and obtaining head posture data of the vehicle driver through the head posture estimation model, wherein the head posture estimation model determines the head posture data by detecting key points of human faces in the head image;
and acquiring the eye movement track data of the vehicle driver through an eye tracker.
3. A vehicle collision avoidance early warning method as claimed in claim 2 wherein said determining an early warning mode from said status data comprises:
determining a gaze area of the vehicle driver from the head pose data and the eye movement trajectory data;
and determining a corresponding early warning mode according to the gazing area and the vehicle collision position predicted by the anti-collision early warning model.
4. The vehicle pre-crash warning method as recited in claim 3, wherein determining the gaze area of the vehicle driver from the head pose data and the eye movement trajectory data comprises:
calculating a maximum view range area of the vehicle driver in front of the vehicle according to the head posture data;
and positioning the gaze region from the maximum visual field range region in combination with the eye movement trajectory data.
5. The vehicle anti-collision early warning method according to claim 3, wherein determining the corresponding early warning mode according to the gazing area and the collision position predicted by the anti-collision early warning model comprises:
if the watching area continuously covers the collision position within a preset time length, determining that the early warning mode is a first mode;
if the gaze area is switched back and forth between the collision position and other positions within a preset time period, determining that the early warning mode is a second mode;
and if the gazing area does not cover the collision position, determining that the early warning mode is a third mode.
6. A vehicle anti-collision warning method according to claim 5, wherein warning the vehicle driver in the warning mode comprises:
if the early warning mode is the first mode, no early warning prompt in any form is output;
if the early warning mode is the second mode, projecting preset early warning information to the front glass of the vehicle through a projector;
and if the early warning mode is the third mode, projecting preset early warning information to the front glass of the vehicle through a projector, and controlling a buzzer of the vehicle to play warning sound.
7. A vehicle anti-collision warning method according to any one of claims 1 to 6, characterized in that the anti-collision warning model is trained by:
acquiring sample data, wherein the sample data comprises corresponding driver state data and vehicle running data when a vehicle collides;
and inputting the sample data into an automatic machine learning module to design and train a model, so as to obtain the anti-collision early warning model.
8. A vehicle anti-collision early warning device is characterized by comprising:
the data acquisition module is used for acquiring state data of a vehicle driver and driving data of the vehicle;
the collision prediction module is used for inputting the state data and the driving data into a pre-trained collision prevention early warning model, and predicting whether the vehicle is likely to collide or not through the collision prevention early warning model, wherein the collision prevention early warning model is a neural network model obtained by training with corresponding driver state data and vehicle driving data as training sets when the vehicle collides;
and the early warning module is used for determining an early warning mode according to the state data if the vehicle is likely to collide, and early warning the vehicle driver according to the early warning mode.
9. An in-vehicle terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the vehicle pre-crash warning method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the vehicle pre-crash warning method according to any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011337072.7A CN112489425A (en) | 2020-11-25 | 2020-11-25 | Vehicle anti-collision early warning method and device, vehicle-mounted terminal equipment and storage medium |
| PCT/CN2021/097281 WO2022110737A1 (en) | 2020-11-25 | 2021-05-31 | Vehicle anticollision early-warning method and apparatus, vehicle-mounted terminal device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011337072.7A CN112489425A (en) | 2020-11-25 | 2020-11-25 | Vehicle anti-collision early warning method and device, vehicle-mounted terminal equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112489425A true CN112489425A (en) | 2021-03-12 |
Family
ID=74934545
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011337072.7A Pending CN112489425A (en) | 2020-11-25 | 2020-11-25 | Vehicle anti-collision early warning method and device, vehicle-mounted terminal equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112489425A (en) |
| WO (1) | WO2022110737A1 (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113408431A (en) * | 2021-06-22 | 2021-09-17 | 青岛小鸟看看科技有限公司 | Intelligent driving evaluation training method and system based on eyeball tracking |
| CN113431126A (en) * | 2021-06-25 | 2021-09-24 | 铜陵有色金属集团股份有限公司 | Remote control system and method for underground carry scraper |
| CN113792484A (en) * | 2021-09-07 | 2021-12-14 | 珠海格力电器股份有限公司 | Cargo impact attitude model construction method and platform |
| WO2022110737A1 (en) * | 2020-11-25 | 2022-06-02 | 平安科技（深圳）有限公司 | Vehicle anticollision early-warning method and apparatus, vehicle-mounted terminal device, and storage medium |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115883605B (en) * | 2022-11-30 | 2024-05-10 | 重庆赛力斯凤凰智创科技有限公司 | Visitor-based vehicle-mounted function control method and device |
| CN115909813A (en) * | 2022-12-06 | 2023-04-04 | 中国第一汽车股份有限公司 | Vehicle collision early warning method, device, equipment and storage medium |
| CN117601859B (en) * | 2024-01-22 | 2024-04-23 | 北京云驰未来科技有限公司 | Early warning system and method for preventing vehicle from collision |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100007479A1 (en) * | 2008-07-08 | 2010-01-14 | Smith Matthew R | Adaptive driver warning methodology |
| US20110169625A1 (en) * | 2010-01-14 | 2011-07-14 | Toyota Motor Engineering & Manufacturing North America, Inc. | Combining driver and environment sensing for vehicular safety systems |
| CN102656613A (en) * | 2009-12-18 | 2012-09-05 | 本田技研工业株式会社 | A predictive human-machine interface using eye gaze technology, blind spot indicators and driver experience |
| CN104943695A (en) * | 2014-03-24 | 2015-09-30 | 沃尔沃汽车公司 | Driver intention estimation arrangement |
| CN105523043A (en) * | 2014-10-15 | 2016-04-27 | 大陆汽车系统公司 | Collision avoidance system using driver eye monitoring during gear change |
| CN109460780A (en) * | 2018-10-17 | 2019-03-12 | 深兰科技（上海）有限公司 | Safe driving of vehicle detection method, device and the storage medium of artificial neural network |
| CN109572550A (en) * | 2018-12-28 | 2019-04-05 | 西安航空学院 | A kind of wheelpath prediction technique, system, computer equipment and storage medium |
| CN110271561A (en) * | 2019-06-06 | 2019-09-24 | 浙江吉利控股集团有限公司 | A kind of automatic Pilot alarming method for power, device and vehicle |
| CN110794963A (en) * | 2019-10-18 | 2020-02-14 | 天津理工大学 | Depth camera-based eye control auxiliary input method |
| CN111539333A (en) * | 2020-04-24 | 2020-08-14 | 湖北亿咖通科技有限公司 | Method for identifying gazing area and detecting distraction of driver |
| CN111731284A (en) * | 2020-07-21 | 2020-10-02 | 平安国际智慧城市科技股份有限公司 | Driving assistance method and device, vehicle-mounted terminal equipment and storage medium |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102390320B (en) * | 2011-08-22 | 2013-06-12 | 武汉理工大学 | Vehicle anti-collision early warning system based on vehicle-mounted sensing network |
| CN103693040B (en) * | 2013-12-10 | 2016-04-13 | 金陵科技学院 | A kind of vehicle collision avoidance system based on double-mode cooperation |
| CN107346612B (en) * | 2016-05-06 | 2021-02-12 | 中国科学院微电子研究所 | Vehicle anti-collision method and system based on Internet of vehicles |
| CN106297340A (en) * | 2016-08-17 | 2017-01-04 | 上海电机学院 | A kind of driving vehicle pre-warning system for monitoring and method |
| CN111081061B (en) * | 2018-10-22 | 2021-09-21 | 杭州海康威视数字技术股份有限公司 | Collision early warning method and device |
| CN109808685B (en) * | 2019-01-07 | 2020-08-18 | 南京航空航天大学 | Automobile early warning automatic collision avoidance control method based on danger assessment |
| CN110807930B (en) * | 2019-11-07 | 2021-08-17 | 中国联合网络通信集团有限公司 | Dangerous vehicle early warning method and device |
| CN112489425A (en) * | 2020-11-25 | 2021-03-12 | 平安科技（深圳）有限公司 | Vehicle anti-collision early warning method and device, vehicle-mounted terminal equipment and storage medium |
-
2020
- 2020-11-25 CN CN202011337072.7A patent/CN112489425A/en active Pending
-
2021
- 2021-05-31 WO PCT/CN2021/097281 patent/WO2022110737A1/en active Application Filing
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100007479A1 (en) * | 2008-07-08 | 2010-01-14 | Smith Matthew R | Adaptive driver warning methodology |
| CN102656613A (en) * | 2009-12-18 | 2012-09-05 | 本田技研工业株式会社 | A predictive human-machine interface using eye gaze technology, blind spot indicators and driver experience |
| US20110169625A1 (en) * | 2010-01-14 | 2011-07-14 | Toyota Motor Engineering & Manufacturing North America, Inc. | Combining driver and environment sensing for vehicular safety systems |
| CN104943695A (en) * | 2014-03-24 | 2015-09-30 | 沃尔沃汽车公司 | Driver intention estimation arrangement |
| CN105523043A (en) * | 2014-10-15 | 2016-04-27 | 大陆汽车系统公司 | Collision avoidance system using driver eye monitoring during gear change |
| CN109460780A (en) * | 2018-10-17 | 2019-03-12 | 深兰科技（上海）有限公司 | Safe driving of vehicle detection method, device and the storage medium of artificial neural network |
| CN109572550A (en) * | 2018-12-28 | 2019-04-05 | 西安航空学院 | A kind of wheelpath prediction technique, system, computer equipment and storage medium |
| CN110271561A (en) * | 2019-06-06 | 2019-09-24 | 浙江吉利控股集团有限公司 | A kind of automatic Pilot alarming method for power, device and vehicle |
| CN110794963A (en) * | 2019-10-18 | 2020-02-14 | 天津理工大学 | Depth camera-based eye control auxiliary input method |
| CN111539333A (en) * | 2020-04-24 | 2020-08-14 | 湖北亿咖通科技有限公司 | Method for identifying gazing area and detecting distraction of driver |
| CN111731284A (en) * | 2020-07-21 | 2020-10-02 | 平安国际智慧城市科技股份有限公司 | Driving assistance method and device, vehicle-mounted terminal equipment and storage medium |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022110737A1 (en) * | 2020-11-25 | 2022-06-02 | 平安科技（深圳）有限公司 | Vehicle anticollision early-warning method and apparatus, vehicle-mounted terminal device, and storage medium |
| CN113408431A (en) * | 2021-06-22 | 2021-09-17 | 青岛小鸟看看科技有限公司 | Intelligent driving evaluation training method and system based on eyeball tracking |
| CN113408431B (en) * | 2021-06-22 | 2023-03-10 | 青岛小鸟看看科技有限公司 | Intelligent driving evaluation training method and system based on eyeball tracking |
| CN113431126A (en) * | 2021-06-25 | 2021-09-24 | 铜陵有色金属集团股份有限公司 | Remote control system and method for underground carry scraper |
| CN113792484A (en) * | 2021-09-07 | 2021-12-14 | 珠海格力电器股份有限公司 | Cargo impact attitude model construction method and platform |
| CN113792484B (en) * | 2021-09-07 | 2024-02-27 | 珠海格力电器股份有限公司 | Cargo impact attitude model construction method and platform |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022110737A1 (en) | 2022-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112489425A (en) | Vehicle anti-collision early warning method and device, vehicle-mounted terminal equipment and storage medium | |
| EP3588372B1 (en) | Controlling an autonomous vehicle based on passenger behavior | |
| US11945435B2 (en) | Devices and methods for predicting collisions and/or intersection violations | |
| CN108068821A (en) | For determining the device of the focus of driver, there are its system and method | |
| JP2013054737A (en) | Visual input of vehicle operator | |
| Costa et al. | Detecting driver’s fatigue, distraction and activity using a non-intrusive ai-based monitoring system | |
| US20210312193A1 (en) | Devices and methods for predicting intersection violations and/or collisions | |
| US20230166743A1 (en) | Devices and methods for assisting operation of vehicles based on situational assessment fusing expoential risks (safer) | |
| Kondyli et al. | Computer assisted analysis of drivers' body activity using a range camera | |
| WO2023101717A9 (en) | Devices and methods for assisting operation of vehicles based on situational assessment fusing expoential risks (safer) | |
| Tran et al. | Driver assistance for “keeping hands on the wheel and eyes on the road” | |
| Bergasa et al. | Visual monitoring of driver inattention | |
| Solomon et al. | Driver Attention and Behavior Detection with Kinect | |
| Sharara et al. | A real-time automotive safety system based on advanced ai facial detection algorithms | |
| Louie et al. | Towards a driver monitoring system for estimating driver situational awareness | |
| Matviienko et al. | QuantiBike: Quantifying Perceived Cyclists' Safety via Head Movements in Virtual Reality and Outdoors | |
| US11046247B1 (en) | System and method for predicting effects of forward glance durations on latent hazard detection | |
| Murphey et al. | Driver lane change prediction using physiological measures | |
| Mihai et al. | Video-based evaluation of driver's visual attention using smartphones | |
| US20210309221A1 (en) | Devices and methods for determining region of interest for object detection in camera images | |
| Gaidar et al. | Real-time fatigue features detection | |
| Kübler et al. | 55 rides: attention annotated head and gaze data during naturalistic driving | |
| Dixit et al. | Face detection for drivers’ drowsiness using computer vision | |
| Hernández et al. | Vision-Based distraction analysis tested on a realistic driving simulator | |
| Hijaz et al. | Driver Visual Focus of Attention Estimation in Autonomous Vehicles |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210312 |
|
| RJ01 | Rejection of invention patent application after publication |