WO2017193272A1

WO2017193272A1 - Vehicle-mounted fatigue pre-warning system based on human face recognition and pre-warning method

Info

Publication number: WO2017193272A1
Application number: PCT/CN2016/081511
Authority: WO
Inventors: 高远; 陈志远; 罗辉; 苏明珠; 陈亮亮; 钟志威; 刘波
Original assignee: 深圳市赛亿科技开发有限公司
Priority date: 2016-05-10
Filing date: 2016-05-10
Publication date: 2017-11-16
Also published as: CN108369766A

Abstract

Provided are a vehicle-mounted fatigue pre-warning system based on human face recognition and a pre-warning method. The system comprises: a main-control module (1), a human face image acquisition module (2) for providing recognition information for the main-control module (1) and an alarm module (5). Both the human face image acquisition module (2) and the alarm module (5) are electrically connected to the main-control module (1), and the main-control module (1) controls, according to a human face image provided by the human face image acquisition module (2), the alarm module (5) so that the alarm module gives an alarm warning when it is detected that a user is in a fatigue driving state. According to the vehicle-mounted fatigue pre-warning system based on human face recognition and the pre-warning method therefor, a database is used, and learning and recording are performed in real time, thus not simply relying on a data model, and improving the accuracy of fatigue pre-warning.

Description

Vehicle fatigue early warning system based on face recognition and early warning method

Technical field

The invention relates to the field of early warning safety technology, in particular to an on-board fatigue driving early warning system and an early warning method based on face recognition.

Background technique

At present, the main body of the vehicle fatigue driving warning device on the market is divided into: 1. The ear-ear fatigue early warning device, the basic principle is to identify the driver's bow signal, thereby judging and alarming; 2, watch type or glasses type, which passes the pulse Beat to determine whether the person is fatigued; 3, the steering wheel touch type, by determining whether the driver's palm leaves the steering wheel as a fatigue driving signal. The above three methods of fatigue driving warning have the following disadvantages:

First of all, it is difficult to summarize the change law of fatigue state. Each driver has different ages, constitution, mood, diet state, and multiple factors of illness, it is difficult to accurately judge the fatigue state.

Second, through image acquisition and judgment by human motion, the image processing algorithm is greatly affected by the environment and its detection conditions.

Third, the on-board early warning devices on the market need to directly contact the driver, causing many changes to the driver, or the driver's status cannot be accurately identified, and the early warning effect is poor and cannot be commercialized.

Summary of the invention

The object of the present invention is to provide a fatigue early warning system based on face recognition and an early warning method thereof for solving the technical problem of inaccurate fatigue driving warning in the prior art.

In order to achieve the above object, the technical solution proposed by the present invention is:

A vehicle face fatigue warning system based on face recognition, comprising: a main control module, a face image acquisition module for providing identification information for the main control module, and an alarm module, wherein the person The face image obtaining module and the alarm module are all electrically connected to the main control module, and the main control module controls the alarm module to perform an alarm reminder when detecting that the user is in a fatigue driving state according to the face image provided by the face image acquiring module. .

The Bluetooth module is electrically connected to the main control module, and the Bluetooth module and the smart terminal device perform user fatigue data transmission.

The GPRS module is further connected to the main control module, and when detecting that the user is in a fatigue state, the GPRS module sends the fatigue information and the location information to the bound mobile intelligent terminal.

The face image acquisition module is a CCD camera device.

An early warning method for vehicle fatigue early warning system based on face recognition, which comprises the following steps:

The first step is to obtain a user face image and transmit the acquired face image to the main control module;

In the second step, it is determined according to the face image whether to enter the training mode, and if so, the third step is entered, otherwise the fourth step is entered;

The third step is to preprocess the face image, save it to the face database, and then return to the first step;

The fourth step is to extract the facial features and calculate whether the user is currently in a fatigue state, and if so, enter the fifth step, and if not, return to the first step, wherein the facial features include a blink cycle and a yawn duration;

In the fifth step, an alarm is issued to alert the user to fatigue.

Wherein, after the fifth step, the step of determining whether the current alarm reminder has a false positive is further included, and if there is a misjudgment, the current face feature is recorded and learned. If not, return to the first step.

The image preprocessing in the third step includes a grayscale transformation on the face image, a histogram equalization, a median filtering processing step, and a database when the user's eyes and mouth are in a state of fatigue.

Wherein, in the fourth step, the face feature is extracted by using a PCA algorithm.

Wherein, in the fourth step, the parameter for calculating whether the user is currently in a fatigue state is: when the user blinks for more than 0.25s in 2s, the user is determined to be in a fatigue state.

Wherein, in the fourth step, the parameter for calculating whether the user is currently in a fatigue state is: by the user's mouth type judgment, when the yawning time exceeds 3.5s, and the adjacent yawning interval is less than 10 minutes, the judgment is Fatigue state.

Compared with the prior art, the face recognition-based fatigue early warning system and the early warning method thereof adopt the establishment of a database and perform learning records in real time, which does not simply depend on the data model, and improves the accuracy of the fatigue warning.

DRAWINGS

FIG. 1 is a functional block diagram of a fatigue warning system based on face recognition according to the present invention.

2 is a flow chart of an early warning method for fatigue warning based on face recognition according to the present invention.

detailed description

The present invention will be further elaborated below with reference to the accompanying drawings. The face recognition-based fatigue warning system of the present invention is used to monitor whether a user is in a fatigue state. In the present embodiment, the driver is used as a user, and the vehicle is used in detail. System structure and early warning process during fatigue warning.

Referring to FIG. 1, in the embodiment, a face recognition-based vehicle fatigue early warning system of the present invention at least includes: a main control module 1 for providing a face image of the identification information for the main control module 1 Obtain module 2, and alarm module 5. The face image obtaining module 2 and the alarm module 5 are all electrically connected to the main control module 1. The main control module 1 according to the face image provided by the face image acquiring module 2, when detecting that the user is in a fatigue driving state, The alarm module 5 is controlled to perform an alarm reminder.

More specifically, the main control module 1 has the following processing capabilities: according to the face image acquired by the face image acquisition module, the face features of the face image are acquired, and the face features include the blink cycle, the duration of the yawn, and other possible representations. The characteristics of fatigue. After the main control module 1 acquires the facial features, it automatically determines whether it is in a fatigue state, and if so, immediately alerts the alarm. Because each person's face features a different fatigue state, the main control module 1 has a strong self-learning ability. That is, for the first-time user, the initial data is used as the data learned by the main control module, and if the system determines the fatigue state. If it is correct, the current judgment standard is maintained. If the error is determined, the false alarm is triggered by manual manual method or the like, and the false alarm data is sequentially accumulated as the data basis of the later determination, thereby forming an intelligent and accurate determination for the user.

In order to complete the face recognition-based fatigue warning system, the main control module 1 is also electrically connected with a Bluetooth module 3, and the Bluetooth module 3 and the intelligent terminal device perform user fatigue data transmission. The smart terminal includes a mobile phone, a tablet, a computer, etc. If a mobile phone is used as the smart terminal, an APP can be installed on the mobile phone end, and the APP can read the data transmitted by the Bluetooth module and form a data report, such as a fatigue timing diagram, thereby reflecting The user's fatigue distribution map during the day.

At the same time, the GPRS module 4 is also connected to the main control module. When detecting that the user is in a fatigue state, the GPRS module 4 transmits the fatigue information and the location information to the bound mobile intelligent terminal. Or automatically make a call to the bound terminal.

In this embodiment, the above-described face image acquisition module 2 is a CCD camera. In other embodiments, the face image acquiring module 2 may be another image acquiring device, or the master module may cooperate to capture the face features.

Referring to FIG. 2, the embodiment further discloses an early warning method for an on-board fatigue early warning system based on the above face recognition, which includes the following steps:

In the first step, the user face image is obtained, and the acquired face image is transmitted to the main control module; first, the camera is turned on, the early warning system is turned on, and the face image is acquired by the camera, wherein the camera sensor is installed in the middle position of the steering wheel, Will block the user's view. Among them, the early warning system can be self-provided with power supply, or can be directly connected to the vehicle power supply for direct power supply.

In the second step, it is determined according to the face image whether to enter the training mode, and if so, the third step is entered, otherwise the fourth step is entered. Since the database for the face fatigue feature needs to be established for the new user, the recognition is performed according to the facial fatigue characteristics of the individual, thereby improving the accuracy, and thus it is necessary to enter the training mode.

In the third step, the face image is preprocessed, saved to the face database, and then returned to the first step; if it is required to enter the training mode, the acquired face image is first preprocessed, and the preprocessed image is loaded into the database. , thus providing a data foundation for later intelligent judgment. After the training mode is over, return to the first step and repeat the next test.

In the fifth step, an alarm is issued to alert the user to fatigue.

Wherein, after the fifth step, the step of determining whether the current alarm reminder has a false positive is further included, and if there is a misjudgment, the current face feature is recorded and learned.

In the fourth step, the parameters for calculating whether the user is currently in a fatigue state are classified into two types. One is: when the user blinks for more than 0.25s in 2s, the user is determined to be in a fatigue state. The other is: through the user's mouth type judgment, when the yawning time exceeds 3.5s, and the interval between adjacent yawns is less than 10 minutes, it is judged to be in a fatigue state.

The fatigue determination is described below with a specific set of data:

The image collected by the face image acquisition module can be preprocessed by gradation transformation, histogram equalization, median filtering, etc., and the improved PCA algorithm is used to extract facial features and local features of the eye. Because different people blink, yawning, bowing and other characteristics are very different. Initialization is required before the user such as the driver uses the product. Establish corresponding eye, mouth and face database for different drivers, get the driver's data in different situations such as eyes, mouth open, half open, slightly open, closed, etc., and constantly improve themselves in the future use process. Face data.

The PERCLOS (percentage of eye closure time per unit time) is calculated after obtaining the hot face feature. PERCLOS is a parameter proposed by the US Federal Highway Administration to determine the fatigue status of drivers. There are PERLCOS and a continuous cycle of blinking, as well as the change of the degree of eye opening, and the comparison of the characteristic coefficients of the data in the face database. These three parameters are used to calculate a set of data based on the three data. Finally, through this set of data, it is judged whether the driver is in a state of fatigue.

The calculation method of PERCLOS is f=(t3-t2)/(t4-t1).

f is PERCLOS, t1 is the time when the eye is closed to the maximum of 80%. T2 is the time when the eye is 80% closed to 20%. T3 is the time that the eye is increased to 20% after the eye is closed to 20%. T4 is the time when the eyes are 20% open to 80%.

According to the data, under normal circumstances, the human eye blinks 10-20 times per minute, each cycle 0.2~0.4s. Usually, when people are tired, the blinking time will continue to last for more than 0.25s. In particular, when the fatigue is difficult, the blinking will reach more than one second. When the blinking lasts for 2 seconds, the blinking time will continue for 0.25s, or the blinking curve will be high, and the blinking will be difficult. Remind when the phenomenon occurs. Among them, more than 80% of closed eyes are considered to be fully closed.

Then calculate the PERCLOS curve and obtain the accurate parameters of the determined human eye fatigue for judgment.

When t>1, or the driver's eye information cannot be detected, if some drivers have the habit of wearing sunglasses during driving, in this case we need to extract other facial features of the driver, such as yawning time. .

It is set to S0 when the mouth is opened, S1 is the mouth staying at the maximum time, and the closing time is S2. The longer the S1 time is, the more fatigue is proved. The more frequent S0, S1, S2 occurs, the more fatigue is proved. In order to prevent misjudgment, when S1 is too small, it is considered to be loud. According to physiology, it is usually about 6 seconds for a person to snoring, and the driver will consciously control himself to try not to snoring, and the time will be shortened to 4 seconds. Therefore, we judged that when the yawn time exceeded 3.5 seconds, it was judged to be fatigue. When the driver bows and the bow time is greater than the set value, the driver is considered fatigued. Give a corresponding reminder.

And record and generate the driver fatigue curve and save, the fatigue information from one day to one week, get a fatigue curve through weighted average, and correct the curve every day to remind the driver of the most fatigue time period every day.

The above is only a preferred embodiment of the present invention, and is not intended to limit the embodiments of the present invention. Those skilled in the art can make various modifications or modifications in accordance with the main idea and spirit of the present invention. The scope of protection of the invention shall be determined by the scope of protection claimed in the claims.

Claims

An on-board fatigue warning system based on face recognition, comprising: a main control module, a face image acquisition module for providing identification information for the main control module, and an alarm module, wherein the person The face image obtaining module and the alarm module are all electrically connected to the main control module, and the main control module controls the alarm module to perform an alarm reminder when detecting that the user is in a fatigue driving state according to the face image provided by the face image acquiring module. .
The in-vehicle fatigue warning system based on face recognition according to claim 1, wherein the main control module is further electrically connected with a Bluetooth module, and the Bluetooth module and the intelligent terminal device perform user fatigue data transmission.
The vehicle face early warning system based on face recognition according to claim 1 or 2, wherein the main control module is further connected with a GPRS module, and when the user is in a fatigue state, the GPRS module is detected. The fatigue information and the location information are sent to the bound mobile smart terminal.
The vehicle face early warning system based on face recognition according to claim 1, wherein the face image acquisition module is a CCD camera device.
An early warning method for a vehicle fatigue early warning system based on face recognition according to any one of claims 1 to 4, characterized in that it comprises the following steps:

The first step is to obtain a user face image and transmit the acquired face image to the main control module;

In the second step, it is determined according to the face image whether to enter the training mode, and if so, the third step is entered, otherwise the fourth step is entered;

The third step is to preprocess the face image, save it to the face database, and then return to the first step;

The fourth step is to extract the facial features and calculate whether the user is currently in a fatigue state, and if so, enter the fifth step, and if not, return to the first step, wherein the facial features include a blink cycle and a yawn duration;

In the fifth step, an alarm is issued to alert the user to fatigue.
The early warning method according to claim 5, wherein the step 5 further comprises a step of determining whether the current alarm reminder has a false positive, and if there is a misjudgment, the current face is Sign the record to learn, if not return to the first step.
The early warning method according to claim 5, wherein the image preprocessing in the third step comprises grayscale transformation of a face image, histogram equalization, median filtering processing steps, and establishing a user's eyes And the database when the mouth is in a state of fatigue.
The early warning method according to claim 5, wherein the extracting the facial features in the fourth step adopts a PCA algorithm.
The early warning method according to claim 5, wherein in the fourth step, the parameter for calculating whether the user is currently in a fatigue state is: when the user blinks for more than 0.25s in 2s, the user is determined to be fatigued. status.
The early warning method according to claim 5, wherein in the fourth step, the parameter for calculating whether the user is currently in a fatigue state is: by the mouth type judgment of the user, when the yawning time exceeds 3.5s, and the neighboring When the yawn interval is less than 10 minutes, it is judged to be in a state of fatigue.