CN112699807A - Driver state information monitoring method and device - Google Patents
Driver state information monitoring method and device Download PDFInfo
- Publication number
- CN112699807A CN112699807A CN202011633557.0A CN202011633557A CN112699807A CN 112699807 A CN112699807 A CN 112699807A CN 202011633557 A CN202011633557 A CN 202011633557A CN 112699807 A CN112699807 A CN 112699807A
- Authority
- CN
- China
- Prior art keywords
- driver
- information
- state
- duty
- eye
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 65
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000001514 detection method Methods 0.000 claims abstract description 29
- 230000002159 abnormal effect Effects 0.000 claims abstract description 15
- 230000006399 behavior Effects 0.000 claims description 50
- 230000001815 facial effect Effects 0.000 claims description 45
- 238000012545 processing Methods 0.000 claims description 30
- 230000009471 action Effects 0.000 claims description 20
- 239000000779 smoke Substances 0.000 claims description 13
- 230000004424 eye movement Effects 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 9
- 238000003860 storage Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 8
- 238000005070 sampling Methods 0.000 claims description 6
- 238000012806 monitoring device Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000004397 blinking Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012790 confirmation Methods 0.000 description 2
- 235000019800 disodium phosphate Nutrition 0.000 description 2
- 239000002360 explosive Substances 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 239000013074 reference sample Substances 0.000 description 2
- 238000013068 supply chain management Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 235000012054 meals Nutrition 0.000 description 1
- 230000006996 mental state Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 239000002994 raw material Substances 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000011265 semifinished product Substances 0.000 description 1
- 230000000391 smoking effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000000638 stimulation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B19/00—Alarms responsive to two or more different undesired or abnormal conditions, e.g. burglary and fire, abnormal temperature and abnormal rate of flow
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Landscapes
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Traffic Control Systems (AREA)
- Emergency Alarm Devices (AREA)
Abstract
The embodiment of the invention relates to a method for monitoring driver state information, which comprises the following steps: collecting monitoring video image data of a driver in real time; the monitoring video image data comprises first time information; carrying out face image detection on the monitoring video image data to obtain a driver face image and detection result information of the driver face image; determining whether the on-duty state of the driver is a normal state or an abnormal state according to the detection result information and the first time information; if the on-duty state is an abnormal state, generating alarm information of the driver leaving the duty; if the on-duty state is a normal state, analyzing the face image of the driver to generate the working state information of the driver; the driver operating state information includes: blink behavior information and/or closed-eye behavior information; generating a fatigue index of the driver according to the blink behavior information and/or the eye closing behavior information; and when the fatigue index exceeds a preset fatigue index threshold value, generating driver fatigue alarm information.
Description
Technical Field
The invention relates to the technical field of supply chain management, in particular to a method and a device for monitoring driver state information.
Background
Logistics belongs to a very important link of supply chain management, and refers to the whole process of planning, implementing and managing raw materials, semi-finished products, finished products or related information from a commodity production place to a commodity consumption place in a mode of transportation, storage, distribution and the like at the lowest cost in order to meet the requirements of customers.
In modern logistics, an effective monitoring and reminding device for the working state of a driver is lacked, the driver sits on a fixed seat for a long time, the action is limited to a certain extent, the driver is busy judging stimulation information inside and outside the vehicle, the mental state is highly nervous, and fatigue is easily caused. When a vehicle loaded with flammable and explosive energy sources is driven, immeasurable loss of life and property is brought to the driver through illegal operation or fatigue driving.
Disclosure of Invention
The invention aims to provide a method for monitoring the state information of a driver, aiming at the defects in the prior art, the method can effectively monitor the working state, particularly the fatigue state, of the driver in the driving process, and can give an alarm in time when the driver is in the fatigue state, so that the driving safety is improved.
In order to achieve the above object, a first aspect of an embodiment of the present invention provides a method for monitoring driver state information, including:
collecting monitoring video image data of a driver in real time; the monitoring video image data comprises first time information;
carrying out face image detection on the monitoring video image data to obtain a driver face image and detection result information of the driver face image;
determining the on duty state of the driver to be a normal state or an abnormal state according to the detection result information and the first time information;
if the on-duty state is an abnormal state, generating alarm information of the driver leaving the duty;
if the on duty state is a normal state, analyzing the face image of the driver to generate working state information of the driver; the driver working state information includes: blink behavior information and/or closed-eye behavior information;
generating a fatigue index of the driver according to the blink behavior information and/or the eye closing behavior information;
and when the fatigue index exceeds a preset fatigue index threshold value, generating driver fatigue alarm information.
Preferably, the detecting the face image of the monitored video image data to obtain the detection result information of the face image of the driver specifically includes:
carrying out gray level processing on the monitoring video image data;
processing the preprocessed image based on a characteristic face method (Eigenface), and extracting characteristic points of a face area of the driver;
and positioning the face area according to the feature points of the face area of the driver to obtain the detection result information of the face image of the driver.
Preferably, before the real-time collecting of the monitoring video image data of the driver, the method further comprises:
establishing a facial image information database of a driver; the facial image information database comprises a driver facial reference sampling image and corresponding identity information.
Further preferably, before analyzing the facial image of the driver and generating the information of the working state of the driver, the method further includes:
performing first feature extraction on the facial image of the driver based on a Haar-like feature extraction algorithm to obtain a first facial feature set, and searching a matched facial reference sampling image of the driver in a facial image information database according to the first facial feature set to obtain identity information;
acquiring the scheduling information of the driver, determining the identity information of the driver who should be on duty, and generating the alarm information of the identity error of the driver if the identity information of the driver who should be on duty is not consistent with the acquired identity information.
Preferably, the analyzing the facial image of the driver to generate the working state information of the driver specifically includes:
identifying the eye action of the driver according to the monitoring video image data; the eye movement is specifically a process from an eye opening state to an eye closing state to an eye opening state;
counting the times of eye actions of a driver within a preset time to generate blink behavior information of the driver;
and calculating the duration of each eye action, recording one eye closing action when the duration exceeds the preset duration, and counting the eye closing action within the preset time to generate the information of the eye closing action of the driver.
Preferably, the method further comprises: and acquiring an alarm signal sent by the smoke alarm, and analyzing the alarm signal to generate smoke alarm information.
A second aspect of an embodiment of the present invention provides a driver state information monitoring apparatus, including:
the processing module is used for acquiring monitoring video image data of a driver in real time; the monitoring video image data comprises first time information; and the number of the first and second groups,
carrying out face image detection on the monitoring video image data to obtain a driver face image and detection result information of the driver face image;
determining the on duty state of the driver to be a normal state or an abnormal state according to the detection result information and the first time information;
if the on-duty state is an abnormal state, generating alarm information of the driver leaving the duty;
if the on duty state is a normal state, analyzing the face image of the driver to generate working state information of the driver; the driver working state information includes: blink behavior information and/or closed-eye behavior information;
generating a fatigue index of the driver according to the blink behavior information and/or the eye closing behavior information;
and when the fatigue index exceeds a preset fatigue index threshold value, generating driver fatigue alarm information.
A third aspect of an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;
the processor is configured to be coupled to the memory, read and execute instructions in the memory, so as to implement the method steps of the first aspect;
the transceiver is coupled to the processor, and the processor controls the transceiver to transmit and receive messages.
A fourth aspect of embodiments of the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a computer, cause the computer to perform the method of the first aspect.
The embodiment of the invention provides a driver state information monitoring method, a driver state information monitoring device, electronic equipment and a computer readable storage medium, which are used for detecting face images of the collected monitoring video image data of a driver, determining the on-duty state of the driver, analyzing the face images of the driver in the monitoring video image data to obtain whether the driver is in fatigue driving, and giving an alarm in time when the driver is in the fatigue state, so that the driving safety is improved.
Drawings
Fig. 1 is a flowchart of a method for monitoring driver status information according to an embodiment of the present invention;
fig. 2 is a block diagram of a driver status information monitoring apparatus according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
According to the method for monitoring the state information of the driver, provided by the embodiment of the invention, the behavior of the driver for energy transportation is monitored and recorded, and when dangerous conditions are possibly generated, alarm processing is timely carried out, so that the safety in the process of energy transportation is improved.
As shown in fig. 1, which is a flowchart of a method for monitoring driver status information according to an embodiment of the present invention, an execution subject of the method may be a processor in an energy transport vehicle or a cloud server, and the method mainly includes the following steps:
specifically, the video monitoring equipment is installed at the driving position of the driver, and the installation direction and the installation angle of the video monitoring equipment can be ensured to acquire the facial state of the driver in real time, such as facial feature points, facial orientation and the like. The video monitoring device may be a webcam with real-time transmission of video image data. The monitoring video image data is composed of a plurality of frames of monitoring images, and the time corresponding to each frame of monitoring image is the first time information.
Before step 101, the method further comprises, establishing a database of facial image information of the driver; the facial image information database includes a driver's facial reference sample image and corresponding identity information.
Specifically, facial image information of the driver is collected and recorded, and a facial image information database of the driver is established. The facial image information database comprises a facial reference sampling image of the driver and corresponding identity information, so that the comparison of facial features of the driver and the confirmation of the identity are facilitated. Wherein the identity information comprises a driver ID. In this step, the network camera in step 101 may be used to collect the facial image information of the driver, or a camera may be used, which is not limited herein.
102, detecting a face image of the monitoring video image data to obtain a driver face image and detection result information of the driver face image;
specifically, the face image detection means that a certain strategy is adopted to search the monitoring video image data to determine whether the monitoring video image data contains a face, so that a face image of a driver in the extracted monitoring video image is obtained. The detection result information of the face image of the driver includes the face image in which the driver is detected and the face image in which the driver is not detected.
Further specifically, gray level processing is carried out on the monitoring video image data; the influence of ambient light of the collection environment is avoided, and the contrast of the image is enhanced.
Processing the preprocessed image based on a characteristic face method (Eigenface), and extracting characteristic points of a face area of the driver;
and positioning the face area according to the feature points of the face area of the driver to obtain the detection result information of the face image of the driver.
103, determining the on duty state of the driver to be a normal state or an abnormal state according to the detection result information and the first time information;
104, if the on-duty state is an abnormal state, generating alarm information of the driver leaving the duty;
specifically, when the face image of the driver is not detected, whether the driver is on duty or not is further judged by combining the first time information. For example, when the first time information is just the meal time of the driver, the face image of the driver is not detected with certainty, but it cannot be determined that the driver is in the off-shift state. When the first time information is working time, the driver is off duty, namely, when the on duty state is abnormal, alarm information is generated and output to a corresponding alarm device, and an alarm signal is sent out, so that the driver can be reminded.
105, if the on duty state is a normal state, analyzing the face image of the driver to generate the working state information of the driver; the driver operating state information includes: blink behavior information and/or closed-eye behavior information;
specifically, the facial image of the driver is analyzed only when the driver is in the normal state on duty. The driver working state information can be understood as face action information of the driver during working, and the working state of the driver is reflected by acquiring the face action information.
Further specifically, according to the monitoring video image data, the eye action of the driver is identified; the eye movement is specifically the process from the eye opening state to the eye closing state and then to the eye opening state;
and counting the times of eye movements of the driver within a preset time to generate the blinking behavior information of the driver.
In a specific embodiment, the preset time is 1min, and the number of eye movements of the driver is counted to be 20 times, so that the blinking behavior information of the driver is 20 times/min.
And calculating the duration of each eye action, recording the eye closing behavior once when the duration exceeds the preset duration, and counting the eye closing behavior in the preset duration to generate the information of the eye closing behavior of the driver.
Generally, the duration of an eye movement of a driver is relatively fixed, so that the duration of the eye movement of the driver can be counted, and then an average value is calculated, that is, the preset duration is the average duration of the eye movement of the driver.
In another specific embodiment, the preset time period is 3S, and when the time period of the driver' S eye movement is 6S, a closed-eye behavior is determined. And if the preset time is 1min and the eye closing behavior is 10 times, the information of the eye closing behavior of the driver is 10 times/min.
Prior to step 105, the method further comprises:
performing first feature extraction on the facial image of the driver based on a Haar-like feature extraction algorithm to obtain a first facial feature set, and searching a matched facial reference sampling image of the driver in a facial image information database according to the first facial feature set to obtain identity information;
the method comprises the steps of obtaining scheduling information of a driver, determining identity information of the driver who is supposed to be on duty, generating driver identity error alarm information according to whether the identity information of the driver who is supposed to be on duty is consistent with the obtained identity information or not, and sending the alarm information to a management terminal so that management personnel can know the situation in time, take corresponding measures and improve safety.
Specifically, after the on duty state of the driver is determined through step 103, it is only determined whether the driver is on duty during the working hours, and it is not determined whether the driver on duty is the driver who should be on duty. Confirmation of the driver status information is also performed. The on duty driver identity information comprises the on duty driver ID, the on duty driver ID is compared with the on duty driver ID, and if the on duty driver ID and the on duty driver ID are not consistent, the driver identity error alarm information is generated.
106, generating a fatigue index of the driver according to the blink behavior information and/or the eye closing behavior information;
and step 107, when the fatigue index exceeds a preset fatigue index threshold, generating fatigue alarm information of the driver.
Specifically, the fatigue index is derived from the blink behavior information, the close-eye behavior information, or a combination of the blink behavior information and the close-eye behavior information. The fatigue index is equal to the ratio of the actual blink behavior information of the driver to the blink behavior information of the driver; or the ratio of the actual eye-closing behavior information of the driver to the eye-closing behavior information of the driver. The preset fatigue index threshold is data obtained by combining the physiological characteristics of the human body. Assuming that the preset fatigue index threshold is 1, when the fatigue index is greater than 1, the driver is in a fatigue state.
In a specific example, the actual blinking behavior information of the driver is 25 times/min, the fatigue index is 1.25, and the driver is determined to be in a fatigue state.
In another specific example, the actual eye-closing behavior information of the driver is 15 times/min, the fatigue index is 1.5, and the driver is determined to be in a fatigue state.
It is to be noted that, as long as the fatigue index is larger than 1, it is determined that the driver is in a fatigue state.
Preferably, the method further comprises: and acquiring an alarm signal sent by the smoke alarm, and analyzing the alarm signal to generate smoke alarm information.
Specifically, when a vehicle loaded with flammable and explosive energy resources is driven, the smoking behavior of a driver may bring about a devastating disaster, and therefore, it is very necessary to arrange a smoke alarm in a cab, when the smoke concentration in the cab reaches a preset smoke concentration alarm threshold, the smoke alarm sends out an alarm signal to remind the driver to pay attention, and when the alarm signal is acquired by a processor or a cloud server, the alarm signal is analyzed to generate smoke alarm information to be stored.
Fig. 2 is a block diagram of a driver status information monitoring apparatus according to a second embodiment of the present invention, where the apparatus may be a video monitoring device (such as a web camera) described in the foregoing embodiment, or may be an apparatus capable of enabling the video monitoring device to implement the method according to the second embodiment of the present application, and for example, the apparatus may be an apparatus in the video monitoring device or a chip system. As shown in fig. 2, the apparatus includes:
the processing module 201 is used for collecting monitoring video image data of a driver in real time; the monitoring video image data comprises first time information; and the number of the first and second groups,
carrying out face image detection on the monitoring video image data to obtain a driver face image and detection result information of the driver face image;
determining whether the on-duty state of the driver is a normal state or an abnormal state according to the detection result information and the first time information;
if the on-duty state is an abnormal state, generating alarm information of the driver leaving the duty;
if the on-duty state is a normal state, analyzing the face image of the driver to generate the working state information of the driver; the driver operating state information includes: blink behavior information and/or closed-eye behavior information;
generating a fatigue index of the driver according to the blink behavior information and/or the eye closing behavior information;
and when the fatigue index exceeds a preset fatigue index threshold value, generating driver fatigue alarm information.
In a specific implementation manner provided in this embodiment, the processing module 201 is specifically configured to:
carrying out gray level processing on the monitoring video image data;
processing the preprocessed image based on a characteristic face method (Eigenface), and extracting characteristic points of a face area of the driver;
and positioning the face area according to the feature points of the face area of the driver to obtain the detection result information of the face image of the driver.
In another specific implementation manner provided by the present embodiment, the processing module 201 is further configured to establish a facial image information database of the driver; the facial image information database includes a driver's facial reference sample image and corresponding identity information.
In another specific implementation manner provided in this embodiment, the processing module 201 is further configured to perform first feature extraction on the facial image of the driver based on a Haar-like feature extraction algorithm to obtain a first facial feature set, and search a matching reference facial sample image of the driver in the facial image information database according to the first facial feature set, so as to obtain the identity information;
acquiring the scheduling information of the driver, determining the identity information of the driver who should be on duty, and generating the alarm information of the identity error of the driver if the identity information of the driver who should be on duty is not consistent with the acquired identity information.
In another specific implementation manner provided in this embodiment, the processing module 201 is specifically configured to:
identifying the eye action of the driver according to the monitoring video image data; the eye movement is specifically the process from the eye opening state to the eye closing state and then to the eye opening state;
counting the times of eye actions of a driver within a preset time to generate blink behavior information of the driver;
and calculating the duration of each eye action, recording the eye closing behavior once when the duration exceeds the preset duration, and counting the eye closing behavior in the preset duration to generate the information of the eye closing behavior of the driver.
In another specific implementation manner provided in this embodiment, the processing module 201 is further configured to obtain an alarm signal sent by the smoke alarm, analyze the alarm signal, and generate smoke alarm information.
The driver state information monitoring device provided by the embodiment of the invention can execute the method steps in the method embodiment, the implementation principle and the technical effect are similar, and the details are not repeated herein.
It should be noted that the division of the modules of the above apparatus is only a logical division, and the actual implementation may be wholly or partially integrated into one physical entity, or may be physically separated. And these modules can be realized in the form of software called by processing element; or may be implemented entirely in hardware; and part of the modules can be realized in the form of calling software by the processing element, and part of the modules can be realized in the form of hardware. For example, the determining module may be a processing element separately set up, or may be implemented by being integrated in a chip of the apparatus, or may be stored in a memory of the apparatus in the form of program code, and the function of the determining module is called and executed by a processing element of the apparatus. Other modules are implemented similarly. In addition, all or part of the modules can be integrated together or can be independently realized. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.
For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), etc. For another example, when some of the above modules are implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor that can invoke the program code. As another example, these modules may be integrated together and implemented in the form of a System-on-a-chip (SOC).
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optics, Digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, bluetooth, microwave, etc.). DVD), or semiconductor media (e.g., Solid State Disk (SSD)), etc.
Fig. 3 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention. The electronic device may be the aforementioned video monitoring device (e.g., a webcam). As shown in fig. 3, the electronic device 300 may include: a processor 31 (e.g., CPU), a memory 32, a transceiver 33; the transceiver 33 is coupled to the processor 31, and the processor 31 controls the transceiving operation of the transceiver 33. Various instructions may be stored in memory 32 for performing various processing functions and implementing method steps performed by the electronic device of embodiments of the present invention. Preferably, the electronic device according to an embodiment of the present invention may further include: a power supply 34, a system bus 35, and a communication port 36. The system bus 35 is used to implement communication connections between the elements. The communication port 36 is used for connection communication between the electronic device and other peripherals.
The system bus mentioned in fig. 3 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The system bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus. The communication interface is used for realizing communication between the database access device and other equipment (such as a client, a read-write library and a read-only library). The Memory may include a Random Access Memory (RAM) and may also include a Non-Volatile Memory (Non-Volatile Memory), such as at least one disk Memory.
The Processor may be a general-purpose Processor, including a central processing unit CPU, a Network Processor (NP), and the like; but also a digital signal processor DSP, an application specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components.
It should be noted that the embodiment of the present invention also provides a computer-readable storage medium, which stores instructions that, when executed on a computer, cause the computer to execute the method and the processing procedure provided in the above-mentioned embodiment.
The embodiment of the invention also provides a chip for running the instructions, and the chip is used for executing the method and the processing process provided by the embodiment.
Embodiments of the present invention also provide a program product, which includes a computer program stored in a storage medium, from which the computer program can be read by at least one processor, and the at least one processor executes the methods and processes provided in the embodiments.
According to the method for monitoring the state information of the driver, provided by the embodiment of the invention, the face image detection is carried out on the collected monitoring video image data of the driver, the on-duty state of the driver is determined, then the face image of the driver in the monitoring video image data is analyzed, whether the driver is in fatigue driving or not is obtained, and when the driver is in the fatigue state, an alarm can be given in time, so that the driving safety is improved.
Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied in hardware, a software module executed by a processor, or a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM powertrain control method, or any other form of storage medium known in the art.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (9)
1. A driver state information monitoring method, characterized by comprising:
collecting monitoring video image data of a driver in real time; the monitoring video image data comprises first time information;
carrying out face image detection on the monitoring video image data to obtain a driver face image and detection result information of the driver face image;
determining the on duty state of the driver to be a normal state or an abnormal state according to the detection result information and the first time information;
if the on-duty state is an abnormal state, generating alarm information of the driver leaving the duty;
if the on duty state is a normal state, analyzing the face image of the driver to generate working state information of the driver; the driver working state information includes: blink behavior information and/or closed-eye behavior information;
generating a fatigue index of the driver according to the blink behavior information and/or the eye closing behavior information;
and when the fatigue index exceeds a preset fatigue index threshold value, generating driver fatigue alarm information.
2. The method for monitoring driver state information according to claim 1, wherein the detecting the face image of the monitored video image data to obtain the detection result information of the face image of the driver specifically comprises:
carrying out gray level processing on the monitoring video image data;
processing the preprocessed image based on a characteristic face method (Eigenface), and extracting characteristic points of a face area of the driver;
and positioning the face area according to the feature points of the face area of the driver to obtain the detection result information of the face image of the driver.
3. The driver state information monitoring method according to claim 1, wherein before the collecting the monitoring video image data of the driver in real time, the method further comprises:
establishing a facial image information database of a driver; the facial image information database comprises a driver facial reference sampling image and corresponding identity information.
4. The driver state information monitoring method according to claim 3, wherein before the analyzing the driver face image to generate driver working state information, the method further comprises:
performing first feature extraction on the facial image of the driver based on a Haar-like feature extraction algorithm to obtain a first facial feature set, and searching a matched facial reference sampling image of the driver in a facial image information database according to the first facial feature set to obtain identity information;
acquiring the scheduling information of the driver, determining the identity information of the driver who should be on duty, and generating the alarm information of the identity error of the driver if the identity information of the driver who should be on duty is not consistent with the acquired identity information.
5. The method for monitoring driver state information according to claim 1, wherein the analyzing the facial image of the driver to generate the driver working state information specifically comprises:
identifying the eye action of the driver according to the monitoring video image data; the eye movement is specifically a process from an eye opening state to an eye closing state to an eye opening state;
counting the times of eye actions of a driver within a preset time to generate blink behavior information of the driver;
and calculating the duration of each eye action, recording one eye closing action when the duration exceeds the preset duration, and counting the eye closing action within the preset time to generate the information of the eye closing action of the driver.
6. The driver state information monitoring method according to claim 1, characterized by further comprising:
and acquiring an alarm signal sent by the smoke alarm, and analyzing the alarm signal to generate smoke alarm information.
7. A driver state information monitoring apparatus, characterized by comprising:
the processing module is used for acquiring monitoring video image data of a driver in real time; the monitoring video image data comprises first time information; and the number of the first and second groups,
carrying out face image detection on the monitoring video image data to obtain a driver face image and detection result information of the driver face image;
determining the on duty state of the driver to be a normal state or an abnormal state according to the detection result information and the first time information;
if the on-duty state is an abnormal state, generating alarm information of the driver leaving the duty;
if the on duty state is a normal state, analyzing the face image of the driver to generate working state information of the driver; the driver working state information includes: blink behavior information and/or closed-eye behavior information;
generating a fatigue index of the driver according to the blink behavior information and/or the eye closing behavior information;
and when the fatigue index exceeds a preset fatigue index threshold value, generating driver fatigue alarm information.
8. An electronic device, comprising: a memory, a processor, and a transceiver;
the processor is used for being coupled with the memory, reading and executing the instructions in the memory to realize the method steps of any one of claims 1-6;
the transceiver is coupled to the processor, and the processor controls the transceiver to transmit and receive messages.
9. A computer-readable storage medium having stored thereon computer instructions which, when executed by a computer, cause the computer to perform the method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011633557.0A CN112699807A (en) | 2020-12-31 | 2020-12-31 | Driver state information monitoring method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011633557.0A CN112699807A (en) | 2020-12-31 | 2020-12-31 | Driver state information monitoring method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112699807A true CN112699807A (en) | 2021-04-23 |
Family
ID=75513721
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011633557.0A Pending CN112699807A (en) | 2020-12-31 | 2020-12-31 | Driver state information monitoring method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112699807A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113408466A (en) * | 2021-06-30 | 2021-09-17 | 东风越野车有限公司 | Method and device for detecting bad driving behavior of vehicle driver |
| CN113537135A (en) * | 2021-07-30 | 2021-10-22 | 三一重机有限公司 | Driving monitoring method, device and system and readable storage medium |
| CN113591533A (en) * | 2021-04-27 | 2021-11-02 | 浙江工业大学之江学院 | Anti-fatigue driving method, device, equipment and storage medium based on road monitoring |
| CN113705427A (en) * | 2021-08-26 | 2021-11-26 | 成都上富智感科技有限公司 | Fatigue driving monitoring and early warning method and system based on vehicle gauge chip SoC |
| CN113780108A (en) * | 2021-08-24 | 2021-12-10 | 中联重科建筑起重机械有限责任公司 | Method, processor and device for identifying driver behavior of tower crane and tower crane |
| CN113903140A (en) * | 2021-09-10 | 2022-01-07 | 武汉中极氢能源发展有限公司 | Method, device and system for prompting driving information, storage medium and processor |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108446600A (en) * | 2018-02-27 | 2018-08-24 | 上海汽车集团股份有限公司 | A kind of vehicle driver's fatigue monitoring early warning system and method |
| CN108846308A (en) * | 2018-04-24 | 2018-11-20 | 浙江吉利控股集团有限公司 | A kind of method for detecting fatigue driving and device |
| WO2019232972A1 (en) * | 2018-06-04 | 2019-12-12 | 上海商汤智能科技有限公司 | Driving management method and system, vehicle-mounted intelligent system, electronic device and medium |
| CN110879973A (en) * | 2019-10-31 | 2020-03-13 | 安徽普华灵动机器人科技有限公司 | Driver fatigue state facial feature recognition and detection method |
-
2020
- 2020-12-31 CN CN202011633557.0A patent/CN112699807A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108446600A (en) * | 2018-02-27 | 2018-08-24 | 上海汽车集团股份有限公司 | A kind of vehicle driver's fatigue monitoring early warning system and method |
| CN108846308A (en) * | 2018-04-24 | 2018-11-20 | 浙江吉利控股集团有限公司 | A kind of method for detecting fatigue driving and device |
| WO2019232972A1 (en) * | 2018-06-04 | 2019-12-12 | 上海商汤智能科技有限公司 | Driving management method and system, vehicle-mounted intelligent system, electronic device and medium |
| CN110879973A (en) * | 2019-10-31 | 2020-03-13 | 安徽普华灵动机器人科技有限公司 | Driver fatigue state facial feature recognition and detection method |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113591533A (en) * | 2021-04-27 | 2021-11-02 | 浙江工业大学之江学院 | Anti-fatigue driving method, device, equipment and storage medium based on road monitoring |
| CN113408466A (en) * | 2021-06-30 | 2021-09-17 | 东风越野车有限公司 | Method and device for detecting bad driving behavior of vehicle driver |
| CN113537135A (en) * | 2021-07-30 | 2021-10-22 | 三一重机有限公司 | Driving monitoring method, device and system and readable storage medium |
| CN113780108A (en) * | 2021-08-24 | 2021-12-10 | 中联重科建筑起重机械有限责任公司 | Method, processor and device for identifying driver behavior of tower crane and tower crane |
| CN113705427A (en) * | 2021-08-26 | 2021-11-26 | 成都上富智感科技有限公司 | Fatigue driving monitoring and early warning method and system based on vehicle gauge chip SoC |
| CN113705427B (en) * | 2021-08-26 | 2024-06-11 | 成都上富智感科技有限公司 | Fatigue driving monitoring and early warning method and system based on vehicle-mounted chip SoC |
| CN113903140A (en) * | 2021-09-10 | 2022-01-07 | 武汉中极氢能源发展有限公司 | Method, device and system for prompting driving information, storage medium and processor |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112699807A (en) | Driver state information monitoring method and device | |
| CN108320256A (en) | Social security events recognition methods, equipment and storage medium based on big data | |
| CN111814775B (en) | Target object abnormal behavior identification method, device, terminal and storage medium | |
| CN111177714A (en) | Abnormal behavior detection method and device, computer equipment and storage medium | |
| CN111613335A (en) | Health early warning system and method | |
| CN108053627A (en) | A kind of fire alarm method, device and equipment | |
| CN115865649B (en) | Intelligent operation and maintenance management control method, system and storage medium | |
| CN107832721B (en) | Method and apparatus for outputting information | |
| CN112992325B (en) | Detection data processing method, system and device for monitored object | |
| CN108449571A (en) | A kind of car monitoring method and equipment | |
| CN111754241A (en) | User behavior perception method, device, equipment and medium | |
| CN115358155A (en) | Power big data abnormity early warning method, device, equipment and readable storage medium | |
| CN113468530A (en) | Real-time risk management safety monitoring method based on cloud computing | |
| CN117079211B (en) | Safety monitoring system and method for network machine room | |
| CN114898261A (en) | Sleep quality assessment method and system based on fusion of video and physiological data | |
| CN112910733A (en) | Full link monitoring system and method based on big data | |
| CN116308366A (en) | Payment security monitoring processing method, system and storage medium | |
| CN113312958B (en) | Method and device for adjusting dispatch priority based on driver state | |
| CN111723767B (en) | Image processing method, device and computer storage medium | |
| CN111126167B (en) | Method and system for quickly identifying serial activities of multiple specific personnel | |
| CN112801783A (en) | Entity identification method and device based on digital currency transaction characteristics | |
| CN113920590A (en) | Living body detection method, living body detection device, living body detection equipment and readable storage medium | |
| CN111339142A (en) | Data monitoring response method, computer readable storage medium and data driving platform | |
| CN111814649B (en) | Customer service expression exercise method based on artificial intelligence and related equipment | |
| CN116311080B (en) | Monitoring image detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210708 Address after: 255000 south-02, 4th floor, building 15, Instrument Industrial Park, 9009 Qinglongshan Road, high tech Zone, Zibo City, Shandong Province Applicant after: Energy chain logistics technology Co.,Ltd. Address before: 100025 building 7, Huitong Times Square, 1 yaojiayuan South Road, Chaoyang District, Beijing Applicant before: CHEZHUBANG (BEIJING) TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right |