CN111553190A - Image-based driver attention detection method - Google Patents
Image-based driver attention detection method Download PDFInfo
- Publication number
- CN111553190A CN111553190A CN202010237283.7A CN202010237283A CN111553190A CN 111553190 A CN111553190 A CN 111553190A CN 202010237283 A CN202010237283 A CN 202010237283A CN 111553190 A CN111553190 A CN 111553190A
- Authority
- CN
- China
- Prior art keywords
- driver
- attention
- model
- cnn model
- condition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 18
- 238000012360 testing method Methods 0.000 claims abstract description 20
- 238000012549 training Methods 0.000 claims abstract description 8
- 238000000034 method Methods 0.000 claims description 14
- 240000007651 Rubus glaucus Species 0.000 claims description 6
- 235000011034 Rubus glaucus Nutrition 0.000 claims description 6
- 235000009122 Rubus idaeus Nutrition 0.000 claims description 6
- 238000013135 deep learning Methods 0.000 claims description 2
- 238000010415 tidying Methods 0.000 claims 1
- 238000012545 processing Methods 0.000 description 6
- 238000003909 pattern recognition Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000004424 eye movement Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 208000001491 myopia Diseases 0.000 description 1
- 230000004379 myopia Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
Abstract
The invention relates to a driver attention detection method based on images, which comprises the following steps: step 1: training data in the discrete-Driver-Detection data set by using a lightweight CNN model to extract characteristics corresponding to different situations of Driver attention; step 2: testing the attention focusing condition of the Driver by using the CNN model to the data in the separated-Driver-Detection data set; and step 3: adjusting the weight and the step length in the CNN model according to the accuracy of the test result in the step 2, and testing again until the accuracy of the test result meets the condition; and 4, step 4: deploying the CNN model after parameter adjustment to an embedded system for operation; and 5: the attention focusing state of the driver is obtained in real time through the camera, and the result is output. The invention utilizes the lightweight CNN model to acquire the attention focusing condition of the driver in real time and detect the attention focusing state of the driver.
Description
Technical Field
The invention relates to a method for detecting the attention of a driver, which is suitable for the condition that the state of the driver is unknown but the upper body action of the driver can be clearly obtained through a camera.
Background
The camera can shoot images and convert the images into a format which can be processed by a computer; the image processing technology can analyze the obtained picture of the person to obtain the action of the driver; the pattern recognition technology is to process and interpret data through a computer to realize classification for different states; the embedded system has simple structure and convenient use and can process images.
Although the physiological signal acquisition and other methods used in the existing driver state monitoring technology are accurate in determining the state of a driver by acquiring the physiological signal, various devices need to be worn, and the devices are generally high in price and heavy, difficult to carry on a vehicle in real time and not beneficial to popularization; some signal collection can also influence the normal driving of driver, for example, the collection of eye movement signal can influence the driver and wear myopia glasses, because everyone's eyes size and position have the difference, also can have the error.
Disclosure of Invention
The present invention is directed to overcoming the above-mentioned disadvantages of the prior art and providing a method for detecting driver attention based on images.
The invention can detect the attention focusing state of the driver under the condition that the attention focusing state of the driver is unknown but the action of the driver can be acquired through the camera and the embedded system is used for image processing, provides a detection method of the embedded system, an image processing technology and a mode recognition technology, and provides a thought for the detection method of the attention focusing state of the driver.
The technical scheme adopted by the invention for solving the technical problems is as follows:
an image-based driver attention detection method comprises the following steps:
step 1: training data in the discrete-Driver-Detection data set by using a lightweight CNN model (such as a SqueezeNet model) to extract characteristics corresponding to different situations of Driver attention;
step 2: testing the attention focusing condition of the Driver by using the CNN model to the data in the separated-Driver-Detection data set;
and step 3: adjusting the weight and the step length in the CNN model according to the accuracy of the test result in the step 2, and testing again until the accuracy of the test result meets the condition;
and 4, step 4: deploying the CNN model after parameter adjustment to an embedded system for operation;
and 5: the attention focusing state of the driver is obtained in real time through the camera, and the result is output.
The invention provides a method for detecting the attention focusing condition of a driver under the condition that the attention focusing state of the driver is unknown but the action of the driver can be acquired through a camera and an embedded system is used for image processing, provides a detection method based on the embedded system, an image processing technology and a mode recognition technology, and provides a thought for the detection method of the attention focusing state of the driver. The camera can shoot images and convert the images into a format which can be processed by a computer; the image processing technology can analyze the obtained picture of the person to obtain the action of the driver; the pattern recognition technology is to process and interpret data through a computer to realize classification for different states; the embedded system has simple structure and convenient use and can process images.
Compared with the prior art, the technical scheme of the invention has the advantages that:
(1) acquiring the attention focusing condition of the driver in real time by using a lightweight CNN model, and detecting the attention focusing state of the driver;
(2) a lightweight CNN model is deployed in an embedded system, so that the cost is lower, the practicability is higher, and the popularization is facilitated.
Drawings
FIG. 1: flow chart of the method of the invention.
Detailed Description
In order to facilitate the understanding and implementation of the present invention for those of ordinary skill in the art, the present invention is further described in detail below with reference to the accompanying drawings and examples.
A method for detecting fatigue degree of a programmer comprises the following steps:
step 1: training data of a training set part in a discrete-Driver-Detection data set by using a light-weight SqueezeNet model, wherein the model can determine characteristics corresponding to pictures with different attention of a Driver according to the relation of distances between pixel points of the head and the hand of the Driver in each picture, such as actions of using a mobile phone (including receiving and calling, receiving and sending short messages and the like), speaking to passengers, adjusting a radio, getting back to a back seat to take things, eating, settling appearance and the like;
step 2: the SqueezeNet model is used for testing the attention concentration condition of the Driver on the data of the test set part in the separated-Driver-Detection data set, and the model judges the state of the Driver in a new picture given in the test set according to the characteristics of different states of the attention of the Driver in the picture extracted in the training process in the step 1;
and step 3: adjusting the weights and step sizes corresponding to different elements in the Squeezenet model according to the accuracy sum of the test result in the step 2, and testing again until the accuracy of the test result meets the condition;
and 4, step 4: deploying a deep learning environment in a raspberry pi 4B system, installing opencv, numpy and other libraries, and then deploying the parameter-adjusted Squeezenet model into the raspberry pi 4B system for operation;
and 5: and acquiring the attention concentration state of the driver in real time through a camera of the raspberry group and outputting a result.
The CNN model, the SqueezeNet model and the discrete-Driver-Detection data set are disclosed in Chaojie Ou, Qiang ZHao et al, Design of an End-to-End dual mode Driver Detection System, published in ICIAR 2019.
The invention obtains the action of the driver in real time and judges the attention concentration state of the driver by installing the embedded system which is provided with the camera and is configured with the machine learning environment and the image recognition model in the automobile.
The embodiments described in this specification are merely illustrative of implementations of the inventive concept and the scope of the present invention should not be considered limited to the specific forms set forth in the embodiments but rather by the equivalents thereof as may occur to those skilled in the art upon consideration of the present inventive concept.
Claims (2)
1. A method for detecting fatigue degree of a programmer comprises the following steps:
step 1: training data of a training set part in a discrete-Driver-Detection data set by using a lightweight SqueezeNet model, wherein the model can determine characteristics corresponding to pictures with different attention of a Driver according to the relation of distances between pixel points of the head and the hand of the Driver in each picture and the like;
step 2: the SqueezeNet model is used for testing the attention concentration condition of the Driver on the data of the test set part in the separated-Driver-Detection data set, and the model judges the state of the Driver in a new picture given in the test set according to the characteristics of different states of the attention of the Driver in the picture extracted in the training process in the step 1;
and step 3: adjusting the weights and step sizes corresponding to different elements in the Squeezenet model according to the accuracy sum of the test result in the step 2, and testing again until the accuracy of the test result meets the condition;
and 4, step 4: deploying a deep learning environment in a raspberry pi 4B system, installing opencv, numpy and other libraries, and then deploying the parameter-adjusted Squeezenet model into the raspberry pi 4B system for operation;
and 5: and acquiring the attention concentration state of the driver in real time through a camera of the raspberry group and outputting a result.
2. A method for detecting the fatigue level of a programmer as claimed in claim 1, wherein: the characteristics corresponding to the pictures of different situations of the attention of the driver comprise: using the mobile phone, talking with passengers, tuning the radio, returning to take things to the backseat, eating and tidying appearance.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010237283.7A CN111553190A (en) | 2020-03-30 | 2020-03-30 | Image-based driver attention detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010237283.7A CN111553190A (en) | 2020-03-30 | 2020-03-30 | Image-based driver attention detection method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111553190A true CN111553190A (en) | 2020-08-18 |
Family
ID=72007512
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010237283.7A Pending CN111553190A (en) | 2020-03-30 | 2020-03-30 | Image-based driver attention detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111553190A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105260705A (en) * | 2015-09-15 | 2016-01-20 | 西安邦威电子科技有限公司 | Detection method suitable for call receiving and making behavior of driver under multiple postures |
| CN105469073A (en) * | 2015-12-16 | 2016-04-06 | 安徽创世科技有限公司 | Kinect-based call making and answering monitoring method of driver |
| CN107330378A (en) * | 2017-06-09 | 2017-11-07 | 湖北天业云商网络科技有限公司 | A kind of driving behavior detecting system based on embedded image processing |
| CN110020632A (en) * | 2019-04-12 | 2019-07-16 | 李守斌 | A method of the recognition of face based on deep learning is for detecting fatigue driving |
| CN110532878A (en) * | 2019-07-26 | 2019-12-03 | 中山大学 | A kind of driving behavior recognition methods based on lightweight convolutional neural networks |
| US20190370580A1 (en) * | 2017-03-14 | 2019-12-05 | Omron Corporation | Driver monitoring apparatus, driver monitoring method, learning apparatus, and learning method |
| CN110688921A (en) * | 2019-09-17 | 2020-01-14 | 东南大学 | Method for detecting smoking behavior of driver based on human body action recognition technology |
-
2020
- 2020-03-30 CN CN202010237283.7A patent/CN111553190A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105260705A (en) * | 2015-09-15 | 2016-01-20 | 西安邦威电子科技有限公司 | Detection method suitable for call receiving and making behavior of driver under multiple postures |
| CN105469073A (en) * | 2015-12-16 | 2016-04-06 | 安徽创世科技有限公司 | Kinect-based call making and answering monitoring method of driver |
| US20190370580A1 (en) * | 2017-03-14 | 2019-12-05 | Omron Corporation | Driver monitoring apparatus, driver monitoring method, learning apparatus, and learning method |
| CN107330378A (en) * | 2017-06-09 | 2017-11-07 | 湖北天业云商网络科技有限公司 | A kind of driving behavior detecting system based on embedded image processing |
| CN110020632A (en) * | 2019-04-12 | 2019-07-16 | 李守斌 | A method of the recognition of face based on deep learning is for detecting fatigue driving |
| CN110532878A (en) * | 2019-07-26 | 2019-12-03 | 中山大学 | A kind of driving behavior recognition methods based on lightweight convolutional neural networks |
| CN110688921A (en) * | 2019-09-17 | 2020-01-14 | 东南大学 | Method for detecting smoking behavior of driver based on human body action recognition technology |
Non-Patent Citations (2)
| Title |
|---|
| CHAOJIE OU等: "Design of an End-to-End Dual Mode Driver Distraction Detection System" * |
| CHAOJIE OU等: "Design of an End-to-End Dual Mode Driver Distraction Detection System", IMAGE ANALYSIS AND RECOGNITION, 16TH INTERNATIONAL CONFERENCE, ICIAR 2019, vol. 11663, pages 199 - 207 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107851195B (en) | Target detection using neural networks | |
| CN106548182B (en) | Pavement crack detection method and device based on deep learning and main cause analysis | |
| JP5538160B2 (en) | Pupil detection device and pupil detection method | |
| WO2001046707A3 (en) | Method for learning-based object detection in cardiac magnetic resonance images | |
| US7599524B2 (en) | Method and apparatus for providing a robust object finder | |
| CN111476306A (en) | Object detection method, device, equipment and storage medium based on artificial intelligence | |
| EP2596746A1 (en) | Pupil detection device and pupil detection method | |
| CN111027481B (en) | Behavior analysis method and device based on human body key point detection | |
| JP2017215861A (en) | Behavior recognition device, learning device, method and program | |
| CN110276257B (en) | Face recognition method, device, system, server and readable storage medium | |
| US20200085296A1 (en) | Eye state detection system and method of operating the same for utilizing a deep learning model to detect an eye state | |
| CN115699082A (en) | Defect detection method and device, storage medium and electronic equipment | |
| CN110859609A (en) | Multi-feature fusion fatigue driving detection method based on voice analysis | |
| CN110555400A (en) | icing detection method and system based on unmanned aerial vehicle inspection overhead transmission line | |
| CN112201343B (en) | Cognitive state recognition system and method based on facial micro-expressions | |
| CN114639078A (en) | Vehicle type recognition method, device and system | |
| CN111553190A (en) | Image-based driver attention detection method | |
| CN111275126A (en) | Sample data set generation method, device, equipment and storage medium | |
| JP6478221B2 (en) | Person identification device and program | |
| CN111856445A (en) | Target detection method, device, equipment and system | |
| CN111695535A (en) | Multi-granularity feature and middle-layer feature-based distraction driving detection method | |
| US20230259782A1 (en) | Artificial neural network | |
| CN110879988A (en) | Information compliance detection method, client, device, server and computer readable medium | |
| Saha et al. | Visual, navigation and communication aid for visually impaired person | |
| CN111695510A (en) | Image-based fatigue detection method for computer operator |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |