CN111627131A - Driving recording device and method with driver auxiliary monitoring warning function - Google Patents
Driving recording device and method with driver auxiliary monitoring warning function Download PDFInfo
- Publication number
- CN111627131A CN111627131A CN202010350879.8A CN202010350879A CN111627131A CN 111627131 A CN111627131 A CN 111627131A CN 202010350879 A CN202010350879 A CN 202010350879A CN 111627131 A CN111627131 A CN 111627131A
- Authority
- CN
- China
- Prior art keywords
- driver
- video data
- unit
- fatigue
- driving
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/08—Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
- G07C5/0841—Registering performance data
- G07C5/0875—Registering performance data using magnetic data carriers
- G07C5/0891—Video recorder in combination with video camera
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
Abstract
The invention discloses a driving recorder with a driver auxiliary monitoring warning function and a method thereof, wherein the driving recorder comprises an image acquisition unit, a data storage unit, a data processing unit, an alarm unit, a communication unit and a power supply unit, wherein: the image acquisition unit is used for acquiring the video data of the face posture of the driver and the video data of the road environment in front of the vehicle; the data storage unit is used for storing the acquired video data; the data processing unit is used for judging whether the driver is in a fatigue driving state or not according to the video data; the alarm unit responds to the fatigue state of the driver and sends out an alarm signal; the communication unit issues a search for nearby parking position information to the navigation in response to the driver being in a tired state. The invention monitors and warns the driving state of the driver in real time, automatically navigates to the parking area when the driver is in an exhausted state, and can prevent traffic accidents caused by fatigue driving of the driver in time.
Description
Technical Field
The invention belongs to the field of safe auxiliary driving, and particularly relates to a driving recording device and method with a driver auxiliary monitoring warning function.
Background
With the development of the traffic industry in China, vehicles running on roads are increasing, the occurrence probability of traffic accidents is also increasing, and a vehicle data recorder serving as a safety auxiliary device is widely applied to various types of motor vehicles. The existing automobile data recorder only has an automobile exterior image recording function generally, can record the surrounding conditions of an automobile in the running process in real time, is convenient for dividing responsibility when an accident occurs, can assist a driver to back a car and the like when the automobile normally runs, but cannot play an obvious role in preventing the accident.
Disclosure of Invention
The invention aims to provide a driving recording device with a driver auxiliary monitoring and warning function.
The technical scheme for realizing the purpose of the invention is as follows: the utility model provides a driving recorder with driver assists control warning function, includes image acquisition unit, data storage unit, data processing unit, alarm unit, communication unit and power supply unit, wherein:
the image acquisition unit is used for acquiring driver face posture video data and road environment video data;
the data storage unit is used for storing the acquired video data;
the data processing unit is used for judging whether the driver is in a fatigue driving state or not according to the video data;
the alarm unit sends out an alarm signal in response to the driver being in a fatigue state;
the communication unit issues a search for nearby parking position information to the navigation in response to the driver being in a tired state.
Preferably, the image acquisition unit comprises 2 cameras for respectively acquiring the driver face posture video data and the road environment video data.
Preferably, the data processing unit adopts a 32-bit embedded processor.
Preferably, the specific method for judging whether the driver is in the fatigue driving state according to the video data by the data processing unit is as follows:
according to the face posture video data of the driver, obtaining the blinking frequency of the driver by adopting a dlib fatigue detection algorithm, and judging whether the driver is in fatigue driving or not through the PERCLOS eye movement characteristics;
or HEVC inter-frame prediction is carried out according to the road environment video data, a vehicle running route is obtained, the vehicle running route is compared with a navigation route, and fatigue driving is judged when the coincidence degree of the running route and the navigation route is smaller than a set threshold value.
Preferably, the warning signal emitted by the warning unit in response to the driver being in fatigue comprises an audible warning signal and/or a light warning signal.
The invention also provides a driving recording method with the driver auxiliary monitoring and warning function, which comprises the following specific steps:
acquiring driver face posture video data and road environment video data;
storing the collected video data;
judging whether the driver is in a fatigue driving state or not according to the video data;
issuing a warning signal in response to the driver being in a fatigue state;
and responding to the fatigue state of the driver and sending information for searching the nearby parking position to the navigation.
Preferably, the specific method for judging whether the driver is in the fatigue driving state according to the video data comprises the following steps:
according to the face posture video data of the driver, obtaining the blinking frequency of the driver by adopting a dlib fatigue detection algorithm, and judging whether the driver is in fatigue driving or not through the PERCLOS eye movement characteristics;
or HEVC inter-frame prediction is carried out according to the road environment video data, a vehicle running route is obtained, the vehicle running route is compared with a navigation route, and when the coincidence degree of the running route and the navigation route is smaller than a set threshold value, fatigue driving is judged
The invention has the beneficial effects that: the invention expands the image acquisition function of the automobile data recorder, increases the image acquisition function and the automatic navigation function in the automobile besides realizing the original video recording function outside the automobile, extracts the blink frequency of the driver and the running characteristics of the automobile by utilizing the image processing technology, monitors and warns the driving state of the driver in real time, automatically navigates to a parking area when the driver is in an exhausted state, and can prevent traffic accidents caused by fatigue driving of the driver in time.
Drawings
Fig. 1 is a schematic structural view of the present invention.
Fig. 2 is a schematic workflow diagram of the present invention.
FIG. 3 is a software flow diagram for implementing the present invention.
FIG. 4 is a flowchart of determining whether fatigue driving is performed according to the video of the facial pose of the driver in the present invention.
Fig. 5 is a flowchart for determining whether fatigue driving is performed or not based on information outside the driver's cab in the present invention.
Detailed Description
The following detailed description of the preferred embodiments of the present invention, taken in conjunction with the accompanying drawings, will make the advantages and features of the invention easier to understand by those skilled in the art, and thus will clearly and clearly define the scope of the invention.
As shown in fig. 1, a vehicle event data recorder with driver assistance monitoring and warning function includes: the device comprises an image acquisition unit, a data storage unit, a data processing unit, an auxiliary lighting unit, an alarm unit, a communication unit and a power supply unit. The auxiliary lighting unit is inlaid in the image acquisition unit, the image acquisition unit is connected with the data storage unit, the alarm unit and the communication unit are all connected with the data processing unit, and the power supply unit and other units are connected to provide electric energy required by normal work.
The image acquisition unit is used for acquiring the video data of the face posture of a driver and the video data of the road environment in front of the cab in the running process of the vehicle;
the data storage unit is used for storing the acquired video data;
the data processing unit is used for judging whether the driver is in a fatigue driving state or not according to the video data;
the alarm unit sends out an alarm signal in response to the driver being in a fatigue state;
the communication unit issues a search for nearby parking position information to the navigation in response to the driver being in a tired state.
In a further embodiment, the image acquisition unit comprises 2 cameras for respectively acquiring the driver face posture video data and the road environment video data. In some embodiments, the cameras are 1080P high-definition cameras, which respectively collect the indoor and outdoor environments of the cab, and transmit the collected video data to the data storage unit for storage through a USB data line.
In some embodiments, the data storage unit is a 32G memory, can store video data within 16h, and is partitioned in advance to store video data from the inside and the outside of the cab respectively.
In a further embodiment, the data processing unit employs a 32-bit embedded processor, including an OpenCV library.
The specific method for judging whether the driver is in the fatigue driving state or not by the data processing unit according to the video data is as follows:
as shown in fig. 4, for the driver face pose video data, due to the influence of factors such as non-uniform illumination, limitation of image acquisition equipment, and the like, the driver face pose needs to be preprocessed by graying, geometric transformation, image enhancement, and the like;
carrying out target detection on the preprocessed face posture of the driver by utilizing a face detector carried by the OpenCV;
according to the characteristics of small imaging area, lack of texture information and the like of the human eye target, positioning of the human eye target is realized by adopting a top-hat transformation based method, and target segmentation is realized by utilizing an SPCNN model, namely, the eye segmentation is realized;
calculating the blink frequency by carrying out dlib fatigue detection algorithm on the segmented target, specifically: each eye uses 6 (x, y) coordinates to represent, starting from the left side of the eye, and counting clockwise along the circumference of the eye, continuously checking 50 frames of images, if the aspect ratio of the eye is less than 0.25, judging the eye to be closed, and according to the PERCLOS eye movement characteristic, judging the fatigue driving when the number of closed frames is more than 80% of the total number of frames.
Or as shown in fig. 5, HEVC inter-frame prediction is performed according to road environment video data, and a vehicle driving route is obtained: 1000 frames of history images (20 frames per second) are continuously collected and a prediction is made for 100 frames of images in the future. And if the running track obtained through inter-frame prediction is different from the time of the running route recommended by navigation by more than 40%, judging the driving is fatigue.
And Step1, reducing the influence of random noise on the gray value of the random noise by adopting median filtering according to the influence of factors such as the dynamic range contained in the external scene of the cab during the driving of the vehicle, and completing image preprocessing.
Step2, HEVC inter-frame prediction is carried out on the preprocessed image data, and a vehicle driving route is obtained, wherein the method mainly comprises the following steps: the method comprises the processes of scene segmentation, data collection, feature training, decision tree judgment, scene merging and outputting and the like.
And Step3, comparing the obtained predicted scene with the navigation recommended running route data for 100 continuous frames, and if the probability of the running tracks being different is more than 40%, judging the driver is in fatigue driving.
And when the driver is judged to be in fatigue driving according to the video data of the face posture of the driver or the video data of the road environment, judging that the driver handles the fatigue driving state.
In a further embodiment, the alarm unit is used for giving out sound and light warning when judging that the driver is fatigue driving, and the sound and light warning are respectively in the forms of speaker sounding and 8.8Hz LED flickering.
In a further embodiment, the communication unit is connected with the in-vehicle navigation unit by adopting Bluetooth, feeds back a driving route recommended by navigation in time and informs the navigation unit to search a nearby parking position when a driver drives fatigue.
As shown in fig. 2 and 3, the working process of the invention is as follows:
the image acquisition unit acquires driver face posture video data and road environment video data; the data storage unit stores the acquired video data,
carrying out dlib fatigue detection on the collected face posture video data of the driver by the data processing unit to obtain the blinking frequency of the driver, and judging whether the driver is in fatigue driving or not by utilizing the PERCLOS eye movement characteristics;
the data processing unit carries out HEVC intra-frame prediction on road environment video data, and if the difference between the acquired real data and the data obtained by prediction is larger than a set threshold value, fatigue driving is judged;
the alarm unit sends out an alarm signal, the loudspeaker sounds, and the LED lamp flickers at the frequency of 8.8 Hz;
the communication unit sends information for searching for a nearby parking position to the navigation.
The invention not only can satisfy the function of the traditional automobile data recorder for recording the image, the sound and other related information during the running of the vehicle, but also adds the functions of real-time monitoring of the fatigue state of the driver, prompt warning and prompt navigation and searching of the parking position when the fatigue state of the driver is monitored.
A driving recording method with driver auxiliary monitoring and warning function comprises the following specific steps:
acquiring driver face posture video data and road environment video data;
storing the collected video data;
judging whether the driver is in a fatigue driving state according to the video data, wherein the specific method comprises the following steps:
preprocessing the face posture of a driver, such as graying, geometric transformation, image enhancement and the like;
carrying out target detection on the preprocessed face posture of the driver by utilizing a face detector carried by the OpenCV;
according to the characteristics of small imaging area, lack of texture information and the like of the human eye target, positioning of the human eye target is realized by adopting a top-hat transformation based method, and target segmentation is realized by utilizing an SPCNN model, namely, the eye segmentation is realized;
calculating the blink frequency by carrying out dlib fatigue detection algorithm on the segmented target, specifically: each eye uses 6 (x, y) coordinates to express, starting from the left side of the eye, and calculating clockwise along the circumference of the eye, continuously checking 20 frames of images, if the aspect ratio of the eye is less than 0.25, judging the eye to be closed, and according to the PERCLOS eye movement characteristic, when the number of closed frames is more than 80% of the total number of frames, judging the fatigue driving.
Or HEVC inter-frame prediction is carried out according to the road environment video data, a vehicle running route is obtained, the vehicle running route is compared with a navigation route, and fatigue driving is judged when the coincidence degree of the running route and the navigation route is smaller than a set threshold value.
Specifically, HEVC inter-frame prediction is carried out on road environment video data, 1000 frames of images (the number of frames per second is 20) are continuously collected, 100 frames of images in the future are predicted, and fatigue driving is judged if the predicted scene is different from the navigation recommended running route in more than 40% of time.
Issuing a warning signal in response to the driver being in a fatigue state;
and responding to the fatigue state of the driver and sending information for searching the nearby parking position to the navigation.
Claims (7)
1. The utility model provides a driving recorder with driver assists control warning function which characterized in that, includes image acquisition unit, data storage unit, data processing unit, alarm unit, communication unit and power supply unit, wherein:
the image acquisition unit is used for acquiring the video data of the face posture of the driver and the video data of the road environment in front of the vehicle;
the data storage unit is used for storing the acquired video data;
the data processing unit is used for judging whether the driver is in a fatigue driving state or not according to the video data;
the alarm unit sends out an alarm signal in response to the driver being in a fatigue state;
the communication unit issues a search for nearby parking position information to the navigation in response to the driver being in a tired state.
2. The driving recording device with driver auxiliary monitoring and warning function as claimed in claim 1, wherein the image capturing unit comprises 2 cameras for capturing video data of the face posture of the driver and video data of the road environment respectively.
3. The driving recording device with driver auxiliary monitoring and warning function as claimed in claim 1, wherein the data processing unit adopts a 32-bit embedded processor.
4. A driving recording device with driver auxiliary monitoring and warning function as claimed in claim 1 or 3, wherein the specific method for the data processing unit to judge whether the driver is in fatigue driving state according to the video data is as follows:
according to the face posture video data of the driver, obtaining the blinking frequency of the driver by adopting a dlib fatigue detection algorithm, and judging whether the driver is in fatigue driving or not through the PERCLOS eye movement characteristics;
or HEVC inter-frame prediction is carried out according to the road environment video data, a vehicle running route is obtained, the vehicle running route is compared with a navigation route, and fatigue driving is judged when the coincidence degree of the running route and the navigation route is smaller than a set threshold value.
5. A vehicle recording unit with driver auxiliary monitoring and warning function according to claim 1, wherein the warning signal generated by the warning unit in response to the driver being in fatigue state comprises an audio warning signal and/or a light warning signal.
6. A driving recording method with a driver auxiliary monitoring and warning function is characterized by comprising the following specific steps:
acquiring driver face posture video data and road environment video data;
storing the collected video data;
judging whether the driver is in a fatigue driving state or not according to the video data;
issuing a warning signal in response to the driver being in a fatigue state;
and responding to the fatigue state of the driver and sending information for searching the nearby parking position to the navigation.
7. The driving recording method with the driver auxiliary monitoring and warning function according to claim 6, wherein the specific method for judging whether the driver is in the fatigue driving state according to the video data is as follows:
according to the face posture video data of the driver, obtaining the blinking frequency of the driver by adopting a dlib fatigue detection algorithm, and judging whether the driver is in fatigue driving or not through the PERCLOS eye movement characteristics;
or HEVC inter-frame prediction is carried out according to the road environment video data, a vehicle running route is obtained, the vehicle running route is compared with a navigation route, and fatigue driving is judged when the coincidence degree of the running route and the navigation route is smaller than a set threshold value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010350879.8A CN111627131A (en) | 2020-04-28 | 2020-04-28 | Driving recording device and method with driver auxiliary monitoring warning function |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010350879.8A CN111627131A (en) | 2020-04-28 | 2020-04-28 | Driving recording device and method with driver auxiliary monitoring warning function |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111627131A true CN111627131A (en) | 2020-09-04 |
Family
ID=72270866
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010350879.8A Pending CN111627131A (en) | 2020-04-28 | 2020-04-28 | Driving recording device and method with driver auxiliary monitoring warning function |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111627131A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113051311A (en) * | 2021-03-16 | 2021-06-29 | 鱼快创领智能科技(南京)有限公司 | Method, system and device for monitoring abnormal change of liquid level of vehicle oil tank |
| CN114347997A (en) * | 2021-12-13 | 2022-04-15 | 华人运通(上海)自动驾驶科技有限公司 | Driving auxiliary control method and system in fatigue state |
| WO2023131278A1 (en) * | 2022-01-07 | 2023-07-13 | 长城汽车股份有限公司 | Vehicle driving assistance control method and apparatus, and vehicle and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104881956A (en) * | 2015-06-17 | 2015-09-02 | 上海大学 | Fatigue driving early warning system |
| CN105575125A (en) * | 2015-12-15 | 2016-05-11 | 上海微桥电子科技有限公司 | Vehicle flow video detection and analysis system |
| CN105788028A (en) * | 2016-03-21 | 2016-07-20 | 上海仰笑信息科技有限公司 | Automobile data recorder with fatigue driving pre-warning function |
| CN107200022A (en) * | 2016-03-15 | 2017-09-26 | 奥迪股份公司 | Drive assist system and method |
| CN110765980A (en) * | 2019-11-05 | 2020-02-07 | 中国人民解放军国防科技大学 | Abnormal driving detection method and device |
-
2020
- 2020-04-28 CN CN202010350879.8A patent/CN111627131A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104881956A (en) * | 2015-06-17 | 2015-09-02 | 上海大学 | Fatigue driving early warning system |
| CN105575125A (en) * | 2015-12-15 | 2016-05-11 | 上海微桥电子科技有限公司 | Vehicle flow video detection and analysis system |
| CN107200022A (en) * | 2016-03-15 | 2017-09-26 | 奥迪股份公司 | Drive assist system and method |
| CN105788028A (en) * | 2016-03-21 | 2016-07-20 | 上海仰笑信息科技有限公司 | Automobile data recorder with fatigue driving pre-warning function |
| CN110765980A (en) * | 2019-11-05 | 2020-02-07 | 中国人民解放军国防科技大学 | Abnormal driving detection method and device |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113051311A (en) * | 2021-03-16 | 2021-06-29 | 鱼快创领智能科技(南京)有限公司 | Method, system and device for monitoring abnormal change of liquid level of vehicle oil tank |
| CN113051311B (en) * | 2021-03-16 | 2023-07-28 | 鱼快创领智能科技(南京)有限公司 | Method, system and device for monitoring abnormal change of liquid level of vehicle oil tank |
| CN114347997A (en) * | 2021-12-13 | 2022-04-15 | 华人运通(上海)自动驾驶科技有限公司 | Driving auxiliary control method and system in fatigue state |
| WO2023131278A1 (en) * | 2022-01-07 | 2023-07-13 | 长城汽车股份有限公司 | Vehicle driving assistance control method and apparatus, and vehicle and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111627131A (en) | Driving recording device and method with driver auxiliary monitoring warning function | |
| US8164463B2 (en) | Driver management apparatus and travel management system | |
| CN112885144B (en) | Early warning method and system for vehicle crash event in construction operation area | |
| CN110532976A (en) | Method for detecting fatigue driving and system based on machine learning and multiple features fusion | |
| CN104599443A (en) | Vehicle-mounted forewarning terminal for driving behaviors based on information fusion and forewarning method thereof | |
| US20220406072A1 (en) | Obstacle detection and notification for motorcycles | |
| CN104875681A (en) | Dynamic vehicle-mounted camera control method based on application scenarios | |
| CN103802728B (en) | Automatically the drive assist system selecting scape to show based on Helmet Mounted Display and method | |
| CN103987577A (en) | Method for monitoring and signaling a traffic situation in the surroundings of a vehicle | |
| CN103871200A (en) | Safety warning system and method used for automobile driving | |
| CN102145688A (en) | Vehicle anti-collision monitoring system and method | |
| CN108961678A (en) | One kind being based on Face datection Study in Driver Fatigue State Surveillance System and its detection method | |
| CN102592475A (en) | Crossing traffic early warning system | |
| CN204332007U (en) | A kind of driving behavior early warning car-mounted terminal based on information fusion | |
| CN105185160A (en) | Pavement detection system and detection method | |
| CN113808418B (en) | Road condition information display system, method, vehicle, computer device and storage medium | |
| CN109435844A (en) | A kind of driving auxiliary and drivers ' behavior detection method | |
| CN108312964A (en) | A kind of fatigue driving state identification early warning system and method | |
| CN112896159A (en) | Driving safety early warning method and system | |
| CN104299363A (en) | Fatigue driving pre-warning system based on multi-feature fusion | |
| CN114360210A (en) | Vehicle fatigue driving early warning system | |
| CN113562005A (en) | Intelligent monitoring and obstacle intrusion limit detecting system for driver behaviors of rail train | |
| CN108932503A (en) | The recognition methods of Chinese herbaceous peony obstacle and device, storage medium, terminal under bad weather | |
| CN205210978U (en) | Drive recorder | |
| CN107640111B (en) | Automobile visual image processing system and method based on hundred-core microprocessor control |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200904 |
|
| RJ01 | Rejection of invention patent application after publication |