CN117533341A - Fatigue driving monitoring method and system based on face recognition and vehicle - Google Patents

Fatigue driving monitoring method and system based on face recognition and vehicle Download PDF

Info

Publication number
CN117533341A
CN117533341A CN202311508290.6A CN202311508290A CN117533341A CN 117533341 A CN117533341 A CN 117533341A CN 202311508290 A CN202311508290 A CN 202311508290A CN 117533341 A CN117533341 A CN 117533341A
Authority
CN
China
Prior art keywords
driver
fatigue driving
face
alarm
face recognition
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311508290.6A
Other languages
Chinese (zh)
Inventor
陈鹏
刘海艳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Deli New Energy Vehicle Co ltd
Original Assignee
Deli New Energy Vehicle Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Deli New Energy Vehicle Co ltd filed Critical Deli New Energy Vehicle Co ltd
Priority to CN202311508290.6A priority Critical patent/CN117533341A/en
Publication of CN117533341A publication Critical patent/CN117533341A/en
Pending legal-status Critical Current

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W50/16—Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/143—Alarm means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
    • B60W2420/403—Image sensing, e.g. optical camera
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/22—Psychological state; Stress level or workload
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/225—Direction of gaze
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/229—Attention level, e.g. attentive to driving, reading or sleeping

Landscapes

  • Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Human Computer Interaction (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention discloses a fatigue driving monitoring method, a system and a vehicle based on face recognition, which comprise the steps of acquiring the face state of a driver, identifying, judging whether the driver is in fatigue driving according to the identified face state, and controlling an alarm to give an alarm and/or controlling a steering wheel to vibrate if the driver is in fatigue driving. According to the invention, face recognition is carried out through the sensor, the face state of the driver is obtained, whether the driver is in fatigue driving is judged according to the face state, and if the driver is in fatigue driving, the fused data is transmitted to the alarm and the steering wheel through the CAN bus to remind the driver.

Description

Fatigue driving monitoring method and system based on face recognition and vehicle
Technical Field
The invention relates to the technical field of driver fatigue driving monitoring, in particular to a fatigue driving monitoring method and system based on face recognition and a vehicle.
Background
The gradual popularization of vehicles brings convenience to the life of people and more traffic accidents are caused, so that the problem of safe driving is an important problem to be considered by drivers and society at present.
The problem of safe driving of the driver is one of the main reasons for traffic accidents caused by fatigue driving of the driver. Fatigue driving refers to a phenomenon that after a driver continuously drives for a long time, physiological functions and psychological functions are disturbed, and driving skills are objectively reduced, and the driver has poor or insufficient sleep quality and is easy to fatigue after driving the vehicle for a long time. The situation is extremely easy to cause road traffic accidents, and more importantly, the driver does not know that the driver is in dangerous fatigue driving state before the accidents happen, or knows that the driver is in fatigue driving but still has to continue driving for various reasons.
Therefore, how to avoid traffic accidents caused by fatigue driving of drivers is a urgent problem to be solved at present.
Disclosure of Invention
The object of the present invention is to solve at least to some extent one of the above-mentioned technical problems.
Therefore, a first object of the present invention is to provide a fatigue driving monitoring method based on face recognition, which uses a sensor to perform face recognition to obtain a face state of a driver, and determines whether the driver is in fatigue driving according to the face state, if so, the driver is reminded by transmitting the fused data to an alarm and a steering wheel through a CAN bus.
The second aim of the invention is to provide a fatigue driving monitoring system based on face recognition.
A third object of the present invention is to propose a vehicle.
A fourth object of the present invention is to propose an electronic device.
A fifth object of the invention is to propose a non-transitory computer readable storage medium.
In order to achieve the above object, a fatigue driving monitoring method based on face recognition according to an embodiment of the first aspect of the present invention includes:
acquiring and identifying the face state of a driver;
judging whether the driver is in fatigue driving or not according to the recognized face state;
and if the driver is in fatigue driving, controlling an alarm to alarm and/or controlling the steering wheel to vibrate.
According to one embodiment of the present invention, the method for acquiring and identifying the face state of the driver includes:
acquiring a face state of the driver during running of the vehicle through an image pickup device;
identifying the face state of the driver based on an Adaboost algorithm;
the camera equipment comprises a camera and/or a thermal infrared imager.
According to an embodiment of the present invention, the determining whether the driver is fatigue driving according to the identified face state includes:
determining that the driver is driving fatigue based on the identified face states including one or more of a first set of face states including: eye closing behavior, nodding behavior, yawning behavior and behavior of deviating the sight from the road surface.
According to one embodiment of the invention, the eye-closing behavior comprises: one or more of eye closure, eye closure time, blink frequency, and pupil variation;
the nodding behavior includes: one or more of the number of nodding, nodding frequency, and whether the head is leaning forward;
the yawing behavior includes: one or more of the number of yawing times, the yawing frequency and the yawing duration;
the line-of-sight off-road behavior includes: and detecting pupil points of the driver in real time by utilizing human eye imaging to calculate a sight angle, measuring according to a preset normal angle range, and determining whether the sight of the driver deviates from a road surface.
According to an embodiment of the present invention, the determining whether the driver is fatigue driving according to the identified face state further includes:
judging whether the driver is in fatigue driving or not in an eye positioning and mouth positioning mode;
wherein, the eye location and mouth location include:
positioning the positions of eyes and a mouth based on an integral projection curve mode, and extracting images corresponding to the eyes and the mouth;
and judging whether the driver is in fatigue driving or not based on the comparison between the extracted images corresponding to the eyes and the mouth and the images in an ORL face standard image library.
According to one embodiment of the present invention, if the driver is tired driving, the controlling the alarm to alarm and/or controlling the steering wheel to vibrate includes:
if the driver is judged to be in fatigue driving, the face state of the identified driver is fused and transmitted to a CAN bus through a controller;
and transmitting the identified face state to an alarm through the CAN bus to alarm and/or transmitting the face state to a steering wheel to vibrate, so as to early warn the fatigue driving behavior of the driver.
To achieve the above object, a second aspect of the present invention provides a fatigue driving monitoring system based on face recognition, including:
the acquisition and identification module is used for acquiring the face state of the driver and identifying the face state;
the judging module is used for judging whether the driver is in fatigue driving or not according to the recognized face state;
and the control module is used for controlling the alarm to alarm and/or controlling the steering wheel to vibrate if the driver is in fatigue driving.
To achieve the above object, an embodiment of a third aspect of the present invention provides a vehicle, where the vehicle includes any one of the embodiments of the fatigue driving monitoring system based on face recognition in the second aspect.
To achieve the above object, an electronic device according to a fourth aspect of the present invention includes:
a memory for storing computer-executable instructions; and
a processor for executing the computer executable instructions to perform any embodiment of the face recognition based fatigue driving monitoring method in the first aspect.
To achieve the above object, a fifth aspect of the present invention provides a non-transitory computer-readable storage medium, on which computer-executable instructions are stored, which when executed by a computer, cause the computer to perform any one of the embodiments of the fatigue driving monitoring method based on face recognition in the first aspect.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Compared with the prior art, the beneficial effects of the embodiment of the application are as follows:
the invention provides a fatigue driving monitoring method based on face recognition, which is characterized in that face recognition is carried out through a sensor, the face state of a driver is obtained, whether the driver is in fatigue driving is judged according to the face state, and if the driver is in fatigue driving, fused data are transmitted to an alarm and a steering wheel through a CAN bus to remind the driver.
The invention provides a fatigue driving monitoring system based on face recognition, which can detect the fatigue state of a driver in advance before sleepiness, and the controller continuously alarms and reminds before entering the sleepy driving state, so that the driver is timely reminded by utilizing sound alarm or steering wheel vibration, accidents are prevented, the safety reminding effect is achieved, the occurrence rate of traffic accidents caused by fatigue driving in the driving process is reduced, and brand-new driving experience is brought to a user.
In order to make the technical means of the present invention more clearly understood, the present invention can be implemented according to the content of the specification, and in order to make the above and other objects, features and advantages of the present invention more comprehensible, preferred embodiments accompanied with the accompanying drawings are described in detail below. Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
In order to more clearly illustrate the technical solutions of the present invention, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a fatigue driving monitoring method based on face recognition according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a fatigue driving monitoring system based on face recognition according to an embodiment of the present invention;
FIG. 3 is another schematic structural diagram of a fatigue driving monitoring system based on face recognition according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present invention and should not be construed as limiting the invention. Therefore, the invention provides a fatigue driving monitoring method and system based on face recognition and a vehicle.
Specifically, the fatigue driving monitoring method, the system and the vehicle based on face recognition in the embodiment of the invention are described below with reference to the accompanying drawings.
Fig. 1 is a flowchart of a fatigue driving monitoring method based on face recognition according to an embodiment of the present invention, and it should be noted that the fatigue driving monitoring method based on face recognition according to the embodiment of the present invention may be applied to a fatigue driving monitoring system based on face recognition according to the embodiment of the present invention, where the system may be configured on an electronic device or may be configured in a server. The electronic device may be a PC or a mobile terminal (e.g., a smart phone, a tablet computer, etc.). The embodiment of the present invention is not limited thereto.
Referring to fig. 1, the embodiment provides a fatigue driving monitoring method based on face recognition, wherein the method includes:
s110, acquiring and identifying the face state of the driver;
in this embodiment, acquiring and identifying the face state of the driver includes:
acquiring a face state of a driver during running of the vehicle through an image pickup device;
identifying the face state of the driver based on an Adaboost algorithm;
wherein the camera device comprises a camera and/or a thermal infrared imager.
It is understood that any imaging device that may be used to obtain the facial status of a driver, including but not limited to a camera or a thermal infrared imager, is within the scope of the present invention.
S120, judging whether the driver is in fatigue driving or not according to the identified face state;
in this embodiment, determining whether the driver is driving fatigue according to the recognized face state includes:
determining that the driver is driving fatigue based on the identified facial states including one or more of a first set of facial states comprising: eye closing behavior, nodding behavior, yawning behavior and behavior of deviating the sight from the road surface.
As one embodiment, the eye-closing behavior includes: one or more of eye closure, eye closure time, blink frequency, and pupil variation;
the nodding behavior includes: one or more of the number of nodding, nodding frequency, and whether the head is leaning forward;
the yawning action includes: one or more of the number of yawing times, the yawing frequency and the yawing duration;
the line of sight off road behavior includes: and detecting pupil points of the driver in real time by utilizing human eye imaging to calculate a sight angle, measuring according to a preset normal angle range, and determining whether the sight of the driver deviates from a road surface.
In one embodiment of the present invention, determining whether the driver is driving fatigue according to the identified face state further includes:
judging whether the driver is in fatigue driving or not in a mode of eye positioning and mouth positioning;
wherein, the eyes location and mouth location include:
positioning the positions of eyes and a mouth based on an integral projection curve mode, and extracting images corresponding to the eyes and the mouth;
and judging whether the driver is in fatigue driving or not based on the comparison between the extracted images corresponding to the eyes and the mouth and the images in the ORL face standard image library.
And S130, if the driver is in fatigue driving, controlling an alarm to alarm and/or controlling the steering wheel to vibrate.
In this embodiment, if the driver is fatigue driving, the control alarm alarms and/or controls the steering wheel to vibrate, including:
if the driver is judged to be in fatigue driving, the face state of the identified driver is fused and transmitted to the CAN bus through the controller;
the identified face state is transmitted to an alarm through a CAN bus to alarm and/or transmitted to a steering wheel to vibrate, so that fatigue driving behaviors of a driver are pre-warned.
According to the fatigue driving monitoring method based on face recognition, face recognition is carried out through the sensor, the face state of the driver is obtained, whether the driver is in fatigue driving is judged according to the face state, and if the driver is in fatigue driving, fused data are transmitted to the alarm and the steering wheel through the CAN bus to remind the driver.
According to the fatigue driving monitoring method based on face recognition, the fatigue state of the driver can be detected in advance before the driver sleeps, the controller continuously carries out alarming reminding before the driver enters the sleeps state, and the driver is timely reminded by utilizing sound alarming or steering wheel vibration so as to prevent accidents, play a role in safety reminding, reduce the occurrence rate of traffic accidents caused by fatigue driving in the driving process and bring brand-new driving experience to the user.
Corresponding to the fatigue driving monitoring method based on face recognition provided by the above embodiments, an embodiment of the present invention further provides a fatigue driving monitoring system based on face recognition, and since the fatigue driving monitoring system based on face recognition provided by the embodiment of the present invention corresponds to the fatigue driving monitoring method based on face recognition provided by the above embodiments, implementation of the fatigue driving monitoring method based on face recognition is also applicable to the fatigue driving monitoring system based on face recognition provided by the present embodiment, and will not be described in detail in the present embodiment.
Fig. 2 is a schematic structural diagram of a fatigue driving monitoring system based on face recognition according to an embodiment of the present invention; referring to fig. 2, the fatigue driving monitoring system 200 based on face recognition provided by the present invention includes: an acquisition identification module 210, a judgment module 220, and a control module 230, wherein:
an acquisition and recognition module 210, configured to acquire and recognize a face state of a driver;
a judging module 220, configured to judge whether the driver is driving fatigue according to the identified face state;
the control module 230 is used for controlling the alarm to alarm and/or controlling the steering wheel to vibrate if the driver is in fatigue driving.
According to the fatigue driving monitoring system based on face recognition, face recognition is carried out through the sensor, the face state of the driver is obtained, whether the driver is in fatigue driving is judged according to the face state, and if the driver is in fatigue driving, the fused data are transmitted to the alarm and the steering wheel through the CAN bus to remind the driver.
The fatigue driving monitoring system based on face recognition provided by the embodiment of the invention can detect the fatigue state of a driver in advance before the driver sleeps, and the controller continuously carries out alarming reminding before the driver enters the sleeps driving state, and timely reminds the driver by utilizing sound alarming or steering wheel vibration so as to prevent accidents, play a role in safety reminding, reduce the occurrence rate of traffic accidents caused by fatigue driving in the driving process and bring brand-new driving experience to users.
In one embodiment of the present invention, acquiring and identifying a face state of a driver includes:
acquiring a face state of a driver during running of the vehicle through an image pickup device;
identifying the face state of the driver based on an Adaboost algorithm;
wherein the camera device comprises a camera and/or a thermal infrared imager.
In one embodiment of the present invention, determining whether the driver is driving fatigue based on the recognized face state includes:
determining that the driver is driving fatigue based on the identified facial states including one or more of a first set of facial states comprising: eye closing behavior, nodding behavior, yawning behavior and behavior of deviating the sight from the road surface.
In one embodiment of the invention, the eye-closing behavior comprises: one or more of eye closure, eye closure time, blink frequency, and pupil variation;
the nodding behavior includes: one or more of the number of nodding, nodding frequency, and whether the head is leaning forward;
the yawning action includes: one or more of the number of yawing times, the yawing frequency and the yawing duration;
the line of sight off road behavior includes: and detecting pupil points of the driver in real time by utilizing human eye imaging to calculate a sight angle, measuring according to a preset normal angle range, and determining whether the sight of the driver deviates from a road surface.
In one embodiment of the present invention, determining whether the driver is driving fatigue according to the recognized face state further includes:
judging whether the driver is in fatigue driving or not in a mode of eye positioning and mouth positioning;
wherein, the eyes location and mouth location include:
positioning the positions of eyes and a mouth based on an integral projection curve mode, and extracting images corresponding to the eyes and the mouth;
and judging whether the driver is in fatigue driving or not based on the comparison between the extracted images corresponding to the eyes and the mouth and the images in the ORL face standard image library.
In one embodiment of the present invention, if the driver is tired driving, controlling the alarm to alarm and/or controlling the steering wheel to vibrate comprises:
if the driver is judged to be in fatigue driving, the face state of the identified driver is fused and transmitted to the CAN bus through the controller;
the identified face state is transmitted to an alarm through a CAN bus to alarm and/or transmitted to a steering wheel to vibrate, so that fatigue driving behaviors of a driver are pre-warned.
FIG. 3 is another schematic structural diagram of a fatigue driving monitoring system based on face recognition according to an embodiment of the present invention; referring to fig. 3, first, the face state of the driver during the running of the vehicle is acquired by an image pickup apparatus, and the face state of the driver is recognized based on an Adaboost algorithm; secondly, determining that the driver is in fatigue driving based on one or more of the recognized face states including a first face state set including eye closing behavior, nodding behavior, yawing behavior and line-of-sight deviation behavior; and finally, if the driver is judged to be in fatigue driving, the face state of the identified driver is fused and transmitted to a CAN bus through a controller, and the face state of the identified driver is transmitted to an alarm through the CAN bus to give an alarm and/or is transmitted to a steering wheel to vibrate, so that the fatigue driving behavior of the driver is pre-warned.
In another embodiment of the present invention, a vehicle is further provided, where the vehicle includes any one of the embodiments of the fatigue driving monitoring system based on face recognition.
In another embodiment of the present invention, there is also provided an electronic apparatus including:
a memory for storing computer-executable instructions; and
a processor for executing computer-executable instructions to perform the face recognition based fatigue driving monitoring method as discussed in any of the above embodiments. Wherein the electronic device may include one or more processors and memory. The memory has stored therein computer executable instructions that, when executed by the processor, cause the electronic device to perform any of the embodiments of the face recognition based fatigue driving monitoring method described above. The electronic device may also include a communication interface.
The processor may be any suitable processing device, such as a microprocessor, microcontroller, integrated circuit, or other suitable processing device. The memory may include any suitable computing system or medium including, but not limited to, non-transitory computer-readable media, random Access Memory (RAM), read-only memory (ROM), hard disk, flash memory, or other memory devices. The memory may store computer executable instructions that are executable by the processor to cause the electronic device to perform any of the embodiments of the face recognition based fatigue driving monitoring method described above. The memory may also store data.
In the embodiment of the invention, the processor can execute various modules included in the instruction to realize the embodiment of the fatigue driving monitoring method based on the face recognition in the fatigue driving monitoring system based on the face recognition. For example, the electronic device may implement each module in the fatigue driving monitoring system based on face recognition to execute the methods S110, S120, and S130 shown in fig. 1.
Referring now to fig. 4, a block diagram of an electronic device 400 suitable for use in implementing embodiments of the present invention is shown. The terminal device in the embodiment of the present invention may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), car terminals (e.g., car navigation terminals), and the like, and stationary terminals such as digital TVs, desktop computers, and the like.
The electronic device shown in fig. 4 is only an example and should not be construed as limiting the functionality and scope of use of the embodiments of the invention.
As shown in fig. 4, the electronic device 400 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 401, which may perform various suitable actions and processes according to a program stored in a Read Only Memory (ROM) 402 or a program loaded from a storage means 408 into a Random Access Memory (RAM) 403. In the RAM 403, various programs and data necessary for the operation of the electronic device 400 are also stored. The processing device 401, the ROM402, and the RAM 403 are connected to each other by a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.
In general, the following devices may be connected to the I/O interface 405: input devices 406 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 407 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 408 including, for example, magnetic tape, hard disk, etc.; and a communication device 409. The communication means 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. While fig. 4 shows an electronic device 400 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead.
In particular, according to embodiments of the present invention, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program embodied on a non-transitory computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via communications device 409, or from storage 408, or from ROM 402. The above-described functions defined in the method of the embodiment of the present invention are performed when the computer program is executed by the processing means 401.
The computer readable medium of the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: and acquiring and identifying the face state of the driver, judging whether the driver is in fatigue driving according to the identified face state, and controlling an alarm to alarm and/or controlling the steering wheel to vibrate if the driver is in fatigue driving.
Computer program code for carrying out operations of the present invention may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present invention may be implemented in software or in hardware. The name of the unit does not in any way constitute a limitation of the unit itself, for example the first acquisition unit may also be described as "unit acquiring at least two internet protocol addresses".
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
In the context of the present invention, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The above description is only illustrative of the preferred embodiments of the present invention and of the principles of the technology employed. It will be appreciated by persons skilled in the art that the scope of the disclosure referred to in the present invention is not limited to the specific combinations of technical features described above, but also covers other technical features formed by any combination of the technical features described above or their equivalents without departing from the spirit of the disclosure. Such as the above-mentioned features and the technical features disclosed in the present invention (but not limited to) having similar functions are replaced with each other.
Moreover, although operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the invention. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are example forms of implementing the claims.
Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention, and that various modifications and variations of the present invention will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the scope of the claims of the present invention as filed.

Claims (10)

1. The fatigue driving monitoring method based on face recognition is characterized by comprising the following steps of:
acquiring and identifying the face state of a driver;
judging whether the driver is in fatigue driving or not according to the recognized face state;
and if the driver is in fatigue driving, controlling an alarm to alarm and/or controlling the steering wheel to vibrate.
2. The fatigue driving monitoring method based on face recognition according to claim 1, wherein the step of acquiring and recognizing the face state of the driver comprises:
acquiring a face state of the driver during running of the vehicle through an image pickup device;
identifying the face state of the driver based on an Adaboost algorithm;
the camera equipment comprises a camera and/or a thermal infrared imager.
3. The fatigue driving monitoring method based on face recognition according to claim 2, wherein the determining whether the driver is fatigue driving according to the recognized face state includes:
determining that the driver is driving fatigue based on the identified face states including one or more of a first set of face states including: eye closing behavior, nodding behavior, yawning behavior and behavior of deviating the sight from the road surface.
4. The fatigue driving monitoring method based on face recognition according to claim 3, wherein the eye closing behavior comprises: one or more of eye closure, eye closure time, blink frequency, and pupil variation;
the nodding behavior includes: one or more of the number of nodding, nodding frequency, and whether the head is leaning forward;
the yawing behavior includes: one or more of the number of yawing times, the yawing frequency and the yawing duration;
the line-of-sight off-road behavior includes: and detecting pupil points of the driver in real time by utilizing human eye imaging to calculate a sight angle, measuring according to a preset normal angle range, and determining whether the sight of the driver deviates from a road surface.
5. The face recognition-based fatigue driving monitoring method according to claim 3, wherein the determining whether the driver is a fatigue driving according to the recognized face state further comprises:
judging whether the driver is in fatigue driving or not in an eye positioning and mouth positioning mode;
wherein, the eye location and mouth location include:
positioning the positions of eyes and a mouth based on an integral projection curve mode, and extracting images corresponding to the eyes and the mouth;
and judging whether the driver is in fatigue driving or not based on the comparison between the extracted images corresponding to the eyes and the mouth and the images in an ORL face standard image library.
6. The fatigue driving monitoring method based on face recognition according to claim 5, wherein if the driver is fatigue driving, controlling an alarm to alarm and/or controlling a steering wheel to vibrate comprises:
if the driver is judged to be in fatigue driving, the face state of the identified driver is fused and transmitted to a CAN bus through a controller;
and transmitting the identified face state to an alarm through the CAN bus to alarm and/or transmitting the face state to a steering wheel to vibrate, so as to early warn the fatigue driving behavior of the driver.
7. Fatigue driving monitoring system based on face identification, which is characterized by comprising:
the acquisition and identification module is used for acquiring the face state of the driver and identifying the face state;
the judging module is used for judging whether the driver is in fatigue driving or not according to the recognized face state;
and the control module is used for controlling the alarm to alarm and/or controlling the steering wheel to vibrate if the driver is in fatigue driving.
8. A vehicle comprising the face recognition-based fatigue driving monitoring system of claim 7.
9. An electronic device, comprising:
a memory for storing computer-executable instructions; and
a processor for executing the computer-executable instructions to perform the face recognition-based fatigue driving monitoring method of any one of claims 1 to 6.
10. A non-transitory computer-readable storage medium, characterized in that the storage medium has stored thereon computer-executable instructions, which when executed by a computer, cause the computer to perform the face recognition-based fatigue driving monitoring method of any one of claims 1 to 6.
CN202311508290.6A 2023-11-13 2023-11-13 Fatigue driving monitoring method and system based on face recognition and vehicle Pending CN117533341A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311508290.6A CN117533341A (en) 2023-11-13 2023-11-13 Fatigue driving monitoring method and system based on face recognition and vehicle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311508290.6A CN117533341A (en) 2023-11-13 2023-11-13 Fatigue driving monitoring method and system based on face recognition and vehicle

Publications (1)

Publication Number Publication Date
CN117533341A true CN117533341A (en) 2024-02-09

Family

ID=89787575

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311508290.6A Pending CN117533341A (en) 2023-11-13 2023-11-13 Fatigue driving monitoring method and system based on face recognition and vehicle

Country Status (1)

Country Link
CN (1) CN117533341A (en)

Similar Documents

Publication Publication Date Title
Zhang et al. SOVCAN: Safety-oriented vehicular controller area network
Chang et al. Design and implementation of a drowsiness-fatigue-detection system based on wearable smart glasses to increase road safety
CN108961681B (en) Fatigue driving reminding method and device and storage medium
US10640123B2 (en) Driver monitoring system
US20210133468A1 (en) Action Recognition Method, Electronic Device, and Storage Medium
EP3825981A1 (en) Warning apparatus, driving tendency analysis device, driving tendency analysis method, and program
US20200247422A1 (en) Inattentive driving suppression system
US10829130B2 (en) Automated driver assistance system
CN111027506B (en) Method and device for determining sight direction, electronic equipment and storage medium
US20200209850A1 (en) Methods and systems to facilitate monitoring center for ride share and safe testing method based for selfdriving cars to reduce the false call by deuddaction systems based on deep learning machine
CN111899545B (en) Driving reminding method and device, storage medium and mobile terminal
CN111532281A (en) Driving behavior monitoring method and device, terminal and storage medium
CN110254442B (en) Method and apparatus for controlling vehicle display
CN113093980B (en) Terminal control method and device, terminal and storage medium
CN112896387B (en) Control method and device for riding vehicle and electronic equipment
CN112102584B (en) Automatic driving alarm method and device for vehicle, vehicle and storage medium
CN117533341A (en) Fatigue driving monitoring method and system based on face recognition and vehicle
CN111292493A (en) Vibration reminding method, device, electronic equipment and system
CN116923372A (en) Driving control method, device, equipment and medium
CN116030447A (en) Perception method, system and vehicle supporting multi-camera dynamic input
CN115831077A (en) Vehicle-mounted screen brightness control method and device and related equipment
CN113183971B (en) Driver state monitoring method and device, vehicle-mounted terminal and readable storage medium
CN113734173B (en) Intelligent vehicle monitoring method, device and storage medium
Sinha Android-Based Driver Alert System for Accident Avoidance
CN116353612A (en) Vehicle driving safety monitoring and guaranteeing method, device and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination