CN115457523A - Method and device for judging fatigue driving of driver based on mobile terminal - Google Patents

Abstract

The invention discloses a method and a device for judging fatigue driving of a driver based on a mobile terminal, wherein the method comprises the following steps: setting an abnormal value for a driver and assigning a value of 0; monitoring lane deviation times, and adding 1 to an abnormal value every time; monitoring abnormal driving rate, and adding 1 to each abnormal value; in a preset time period, if the abnormal value is larger than a preset critical value, observing a driver; continuously monitoring the behavior of a driver through a camera of the mobile terminal, and capturing a preset specific behavior; if the driver has a specific behavior, carrying out face recognition on the driver to judge whether the driver has fatigue driving; and if fatigue driving exists based on the judgment result, stopping the driver from driving and forcibly resting. According to the invention, the fatigue driving of the driver can be accurately judged in a multi-dimensional mode in a multi-layer progressive mode by considering from multiple dimensions through the functional characteristics of the mobile terminal.

Description

Method and device for judging fatigue driving of driver based on mobile terminal

Technical Field

The invention relates to the technical field of driver fatigue driving judging methods, in particular to a method and a device for judging driver fatigue driving based on a mobile terminal.

Background

Fatigue driving refers to the phenomenon that after a driver drives a vehicle continuously for a long time, the physiological function and the psychological function are imbalanced, and the driving skill is objectively reduced, so that unsafe factors such as delayed or early actions, operation pause or improper correction time and the like occur to the driver, and road traffic safety accidents are very easy to occur.

The existing network car appointment travel platform cannot directly acquire driving data from automobile equipment, and the existing fatigue driving judging mode is generally directly monitored by passenger reporting or an automobile system, the passenger may not pay attention to the platform, and the monitoring of the automobile equipment can be bypassed by a mode of counterfeiting videos, so that the fatigue driving judging technology of the existing network car appointment travel platform is difficult to execute, the misjudgment or the misjudgment is easy, the normal driving of a driver is easily influenced during the misjudgment, and the accidental risk is easily caused during the misjudgment.

Disclosure of Invention

In order to overcome the technical defect that the existing fatigue driving identification is easy to misjudge and miss judge, the invention provides a method and a device for judging driver fatigue driving based on a mobile terminal.

In order to solve the problems, the invention is realized according to the following technical scheme:

in a first aspect, the invention discloses a method for judging fatigue driving of a driver based on a mobile terminal, which comprises the following steps:

step S1: setting an abnormal value for a driver and assigning a value of 0;

step S2: monitoring lane shift times, and adding 1 to each abnormal value;

and step S3: monitoring abnormal driving rate, and adding 1 to each abnormal value;

and step S4: in a preset time period, if the abnormal value is larger than a preset critical value, observing a driver;

step S5: continuously monitoring the behavior of a driver through a camera of the mobile terminal, and capturing preset specific behavior;

step S6: if the driver has a specific behavior, carrying out face recognition on the driver to judge whether the driver has fatigue driving;

step S7: and if fatigue driving exists based on the judgment result, stopping the driver from driving and forcibly resting.

As a preferred implementation of the first aspect, the monitoring the number of lane shifts and adding 1 to each occurrence of an abnormal value specifically includes the following sub-steps:

acquiring a running track of a vehicle based on a motion sensor in the mobile terminal;

judging whether the lane deviation occurs in the running track of the vehicle;

judging the time for correcting after lane deviation;

counting the abnormal times of lane deviation and correction exceeding a preset time;

and adding 1 to the abnormal value corresponding to the driver every time the lane abnormal deviation occurs and the lane abnormal deviation is corrected.

As a preferred implementation of the first aspect, the monitoring of the driving rate abnormality, and adding 1 to the abnormal value every time the abnormal value occurs specifically includes the following sub-steps:

acquiring historical driving speed data of a driver;

acquiring the average driving speed of vehicles passing through the current road section;

acquiring the habitual speed of a driver based on the historical driving speed data of the driver;

dividing the habitual driving speed of the driver by the average driving speed to obtain the habitual driving speed of the driver;

dividing the current driving speed of the driver by the average driving speed to obtain the current driving speed of the driver;

comparing the current driving rate with the habitual driving rate;

and if the speed gap exceeds a preset range based on the comparison result, adding 1 to the abnormal value corresponding to the driver.

As a preferred implementation of the first aspect, the continuously monitoring the behavior of the driver through the camera of the mobile terminal, and capturing the preset specific behavior specifically includes the following sub-steps:

when the abnormal value of the driver is larger than a preset critical value, sending a starting instruction to a camera of the mobile terminal;

acquiring a preset specific behavior image, wherein the specific behavior image comprises behavior images of rubbing eyes, yawning, swinging heads, nipping thighs and hamstrings;

continuously receiving driver behaviors shot by a camera from the mobile terminal;

when the driver behavior in the video is compared with the pattern of the specific behavior continuously;

when the driver has a specific behavior, the next stage of fatigue driving recognition is entered.

As a preferred implementation of the first aspect, if the driver has a specific behavior, performing face recognition on the driver to determine whether the driver has fatigue driving, specifically includes the following sub-steps:

when the driver behavior hits one specific behavior, informing the driver to search a safe position for parking so as to perform further face recognition;

adopting a multi-frame detection algorithm to shoot a facial close-up video for a driver, and dividing the facial close-up video into a plurality of frames of face images;

acquiring a face image, and carrying out image preprocessing;

removing the noise of the face image to emphasize useful information, enhancing the image contrast, highlighting image details and improving the quality of the face image;

normalizing the face image, and adjusting the characteristic attribute values of the image to the same dimension so as to improve the model precision;

the face image and the characteristic attribute value are transmitted to a fatigue driving recognition model;

extracting a mouth region and an eye region of a driver's face by a mouth classifier and an eye classifier;

carrying out characteristic discrimination on the mouth region, and discriminating the abnormality of the shape, the mouth opening duration and the breathing rate of the mouth region;

carrying out feature discrimination on the eye region, and discriminating the abnormality of the eye shape, the eye closing time length and the blink rate;

and acquiring a fatigue driving judgment result output by the fatigue driving identification model.

The second invention also discloses a device for judging driver fatigue driving based on the mobile terminal, which comprises:

the abnormal assignment module M1 is used for setting an abnormal value for the driver and assigning the value to be 0;

the offset monitoring module M2 is used for monitoring the lane offset times, and 1 is added to each abnormal value;

a speed monitoring module M3 for monitoring abnormal driving speed, wherein 1 is added to each abnormal value;

a preliminary observation module M4 for observing the driver if the abnormal value is greater than a preset critical value within a preset time period;

the behavior monitoring module M5 is used for continuously monitoring the behavior of the driver through a camera of the mobile terminal and capturing preset specific behavior;

the fatigue judging module M6 is used for carrying out face recognition on the driver if the driver has a specific behavior so as to judge whether the driver has fatigue driving;

and the forced execution module M7 is used for suspending the driver to go out of the vehicle and forcibly resting if fatigue driving exists on the basis of the judgment result.

As a preferred implementation of the second aspect, when the offset monitoring module M2 is running, the following steps are specifically executed:

acquiring a running track of a vehicle based on a motion sensor in the mobile terminal;

judging whether the running track of the vehicle has the condition of lane deviation or not;

judging the time for correcting the lane deviation;

counting the abnormal times of lane deviation and correction exceeding the preset time;

and adding 1 to the abnormal value corresponding to the driver every time the lane abnormal deviation occurs and the lane abnormal deviation is corrected.

As a preferred implementation of the second aspect, when the rate monitoring module M3 is running, the following steps are specifically executed:

acquiring historical driving speed data of a driver;

acquiring the average driving speed of vehicles passing through the current road section;

acquiring the habitual speed of a driver based on the historical driving speed data of the driver;

dividing the habitual speed of the driver by the average driving speed to obtain the habitual driving speed of the driver;

dividing the current driving speed of the driver by the average driving speed based on the current driving speed of the driver to obtain the current driving speed of the driver;

comparing the current driving rate with the habitual driving rate;

and on the basis of the comparison result, if the speed difference exceeds a preset range, adding 1 to an abnormal value corresponding to the driver.

As a preferred implementation of the second aspect, the behavior monitoring module M5, when running, specifically performs the following steps:

when the abnormal value of the driver is larger than a preset critical value, sending a starting instruction to a camera of the mobile terminal;

acquiring preset specific behavior images, wherein the specific behavior images comprise behavior images of rubbing eyes, yawning, throwing heads, nipping thighs and palms;

continuously receiving driver behaviors shot by a camera from the mobile terminal;

when the driver behavior in the video is compared with the pattern of the specific behavior continuously;

when the driver has a specific behavior, the next stage of fatigue driving recognition is entered.

As a preferred implementation of the second aspect, the fatigue judging module M6 specifically executes the following steps when in operation:

when the driver behavior hits one specific behavior, informing the driver to search a safe position for parking so as to perform further face recognition;

adopting a multi-frame detection algorithm to shoot a facial close-up video for a driver, and dividing the facial close-up video into a plurality of frames of face images;

acquiring a face image, and carrying out image preprocessing;

removing the noise of the face image to emphasize useful information, enhancing the image contrast, highlighting image details and improving the quality of the face image;

normalizing the face image, and adjusting the characteristic attribute values of the image to the same dimension so as to improve the model precision;

the face image and the characteristic attribute value are transmitted to a fatigue driving recognition model;

extracting a mouth region and an eye region of a driver's face by a mouth classifier and an eye classifier;

carrying out characteristic discrimination on the mouth region, and discriminating the abnormality of the shape, the mouth opening duration and the breathing rate of the mouth region;

carrying out feature discrimination on the eye region, and discriminating the abnormality of the eye shape, the eye closing time length and the blink rate;

and acquiring a fatigue driving judgment result output by the fatigue driving identification model.

Compared with the prior art, the invention has the beneficial effects that:

according to the invention, through the functional characteristics of the mobile terminal, the technical difficulty that the network car booking travel platform cannot directly acquire the driving data of the automobile system is solved, and the accurate and timely monitoring of the network car booking travel platform on the fatigue driving of the driver is realized in a multi-dimensional consideration and multi-layer progressive mode. The method comprises the steps of firstly realizing data acquisition of correction after lane deviation through a motion sensor of a mobile terminal, then realizing data acquisition of vehicle speed through the motion sensor, locking a driver who may have fatigue driving through the abnormality of the lane deviation and the abnormality of the vehicle speed, realizing image acquisition through a camera of the mobile terminal, capturing specific behaviors of the driver, and further carrying out face recognition on the driver when the specific behaviors occur so as to judge whether the fatigue driving exists or not, so that the accurate judgment of the fatigue driving of the driver is realized, the fault tolerance rate is high, and the normal driving of the driver is not easily influenced by misjudgment.

Drawings

Embodiments of the invention are described in further detail below with reference to the attached drawing figures, wherein:

FIG. 1 is a flow chart illustrating a method for determining driver fatigue driving based on a mobile terminal according to the present invention;

fig. 2 is a schematic structural diagram of the device for judging driver fatigue driving based on the mobile terminal.

Detailed Description

Preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While the preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

The term "include" and variations thereof as used herein is meant to be inclusive in an open-ended manner, i.e., "including but not limited to". The term "or" means "and/or" unless specifically stated otherwise. The term "based on" means "based at least in part on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first," "second," and the like may refer to different or the same objects. Other explicit and implicit definitions are also possible below.

The access device and the server may be directly or indirectly connected through wired or wireless communication. The access device may be a terminal or a server. The access device has a target application running thereon. The target application is an application program capable of initiating a data request to the server, such as a social application, a payment application, a game application, and the like. The server may be an application server that the target application provides a service, or may be a proxy server that distinguishes the application server corresponding to the target application. The server is used for identifying whether each access device belongs to the malicious device and intercepting the data message from the malicious device. When the server is a proxy server, the proxy server forwards the data message which does not belong to the malicious equipment to the application server. The terminal may specifically be a desktop terminal or a mobile terminal, and the mobile terminal may specifically be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart sound box, a smart watch, and the like, but is not limited thereto. The server and the server can be independent physical servers respectively, can also be a server cluster or distributed system formed by a plurality of physical servers, and can also be cloud servers for providing basic cloud computing services such as cloud service, a cloud database, cloud computing, cloud functions, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN (content delivery network) and big data and artificial intelligence platforms.

Example 1

In consideration of the fact that the network car booking travel platform cannot directly acquire data of an automobile and cannot accurately and timely monitor the fatigue driving condition of a network car booking driver through the automobile data, the mobile terminal is adopted, specifically comprises a mobile phone, a tablet personal computer, a notebook computer, an intelligent telephone and other equipment which can be carried on the automobile and has the functions of network connection, a motion sensor, a GPS positioning function, a camera and the like, and is connected with a server through installing a corresponding application program to achieve data interaction. The invention realizes the monitoring of the fatigue driving of the network car booking driver only by collecting the data of the mobile terminal so as to ensure the safety of passengers and the driver.

As shown in fig. 1, in a first aspect, the present invention discloses a method for determining fatigue driving of a driver based on a mobile terminal, comprising the following steps:

step S1: setting an abnormal value for a driver and assigning a value of 0;

step S2: monitoring lane deviation times, and adding 1 to an abnormal value every time;

specifically, the present step specifically includes the following substeps:

acquiring a running track of a vehicle based on a motion sensor in the mobile terminal;

judging whether the lane deviation occurs in the running track of the vehicle;

judging the time for correcting after lane deviation;

counting the abnormal times of lane deviation and correction exceeding the preset time;

and adding 1 to the abnormal value corresponding to the driver every time the lane abnormal deviation occurs and the lane abnormal deviation is corrected.

Specifically, based on the motion sensor in the mobile terminal, the vehicle normal running motion sensor is stationary at the center position due to inertia, and if the vehicle is normally accelerated, a positive value is generated in the X-ray direction, and if the vehicle is decelerated, a negative value is generated in the X-ray direction. The left turn can generate a positive value in Y drawing, the right turn can generate a negative value in Y drawing, the positive and negative values generated in a certain time through Y drawing indicate that the driver has suspicion of quickly returning to the route, and 1 is added to the abnormal value corresponding to the driver every time.

And step S3: monitoring abnormal driving rate, and adding 1 to each abnormal value;

specifically, the present step specifically includes the following substeps:

acquiring historical driving speed data of a driver;

acquiring the average driving speed of vehicles passing through the current road section;

acquiring the habitual speed of a driver based on the historical driving speed data of the driver;

dividing the habitual speed of the driver by the average driving speed to obtain the habitual driving speed of the driver;

dividing the current driving speed of the driver by the average driving speed to obtain the current driving speed of the driver;

comparing the current driving rate with the habitual driving rate;

and on the basis of the comparison result, if the speed difference exceeds a preset range, adding 1 to an abnormal value corresponding to the driver.

And step S4: in a preset time period, if the abnormal value is larger than a preset critical value, observing a driver;

step S5: continuously monitoring the behavior of a driver through a camera of the mobile terminal, and capturing preset specific behavior;

specifically, the present step specifically includes the following substeps:

when the abnormal value of the driver is larger than a preset critical value, sending a starting instruction to a camera of the mobile terminal;

acquiring preset specific behavior images, wherein the specific behavior images comprise behavior images of rubbing eyes, yawning, throwing heads, nipping thighs and palms;

continuously receiving driver behaviors shot by a camera from the mobile terminal;

when the driver behavior in the video is compared with the pattern of the specific behavior continuously;

when the driver has a specific behavior, the next stage of fatigue driving recognition is entered.

Step S6: if the driver has a specific behavior, carrying out face recognition on the driver to judge whether the driver has fatigue driving;

specifically, the present step specifically includes the following substeps:

when the driver behavior hits one specific behavior, informing the driver to search a safe position for parking so as to perform further face recognition;

adopting a multi-frame detection algorithm to shoot a facial close-up video for a driver, and dividing the facial close-up video into a plurality of frames of face images;

acquiring a face image, and performing image preprocessing;

removing the noise of the face image to emphasize useful information, enhancing the image contrast, highlighting image details and improving the quality of the face image;

normalizing the face image, and adjusting the characteristic attribute values of the image to the same dimension so as to improve the model precision;

the face image and the characteristic attribute value are transmitted to a fatigue driving recognition model;

extracting a mouth region and an eye region of the face of the driver through a mouth classifier and an eye classifier;

carrying out characteristic discrimination on the mouth region, and discriminating the abnormality of the shape, the mouth opening duration and the breathing rate of the mouth region;

carrying out feature discrimination on the eye region, and discriminating the abnormality of the eye shape, the eye closing time length and the blink rate;

and acquiring a fatigue driving judgment result output by the fatigue driving identification model.

Step S7: and if fatigue driving exists based on the judgment result, stopping the driver from driving and forcibly resting.

Specifically, based on the judgment result, if the driver is in fatigue driving, the current order of the driver is forcibly stopped, the driver is subjected to a forced rest scheme, the driver is not dispatched within 8 hours, and other nearby vehicles are dispatched to continuously deliver passengers or goods to the destination.

If the driver does not have fatigue driving, the driver is allowed to continue to complete the order.

According to the invention, through the functional characteristics of the mobile terminal, the technical difficulty that the network car booking travel platform cannot directly acquire the driving data of the automobile system is solved, and the accurate and timely monitoring of the network car booking travel platform on the fatigue driving of the driver is realized in a multi-dimensional consideration and multi-layer progressive mode. The method comprises the steps of firstly realizing data acquisition of correction after lane deviation through a motion sensor of a mobile terminal, then realizing data acquisition of vehicle speed through the motion sensor, locking a driver who may have fatigue driving through the abnormality of the lane deviation and the abnormality of the vehicle speed, realizing image acquisition through a camera of the mobile terminal, capturing specific behaviors of the driver, and further carrying out face recognition on the driver when the specific behaviors occur so as to judge whether the fatigue driving exists or not, so that the accurate judgment of the fatigue driving of the driver is realized, the fault tolerance rate is high, and the normal driving of the driver is not easily influenced by misjudgment.

Other steps of the method for judging driver fatigue driving based on the mobile terminal in the embodiment are shown in the prior art.

Example 2

As shown in fig. 2, the second invention also discloses a device for determining fatigue driving of a driver based on a mobile terminal, which comprises:

the abnormal assignment module M1 is used for setting an abnormal value for the driver and assigning the value to be 0;

the offset monitoring module M2 is used for monitoring the lane offset times, and 1 is added to each abnormal value;

a speed monitoring module M3 for monitoring abnormal driving speed, wherein 1 is added to each abnormal value;

a preliminary observation module M4 for observing the driver if the abnormal value is greater than a preset critical value within a preset time period;

the behavior monitoring module M5 is used for continuously monitoring the behavior of the driver through a camera of the mobile terminal and capturing preset specific behavior;

the fatigue judging module M6 is used for carrying out face recognition on the driver if the driver has a specific behavior so as to judge whether the driver has fatigue driving;

and the forced execution module M7 is used for suspending the driver from leaving the vehicle and forcibly resting the vehicle if fatigue driving exists on the basis of the judgment result.

As a preferred implementation of the second aspect, when the offset monitoring module M2 is running, the following steps are specifically executed:

acquiring a running track of a vehicle based on a motion sensor in the mobile terminal;

judging whether the lane deviation occurs in the running track of the vehicle;

judging the time for correcting after lane deviation;

counting the abnormal times of lane deviation and correction exceeding the preset time;

and adding 1 to the abnormal value corresponding to the driver every time the lane abnormal deviation occurs and the lane abnormal deviation is corrected.

As a preferred implementation of the second aspect, when the rate monitoring module M3 is running, the following steps are specifically executed:

acquiring historical driving speed data of a driver;

acquiring the average driving speed of vehicles passing through the current road section;

acquiring the habitual speed of a driver based on the historical driving speed data of the driver;

dividing the habitual speed of the driver by the average driving speed to obtain the habitual driving speed of the driver;

dividing the current driving speed of the driver by the average driving speed based on the current driving speed of the driver to obtain the current driving speed of the driver;

comparing the current driving rate with the habitual driving rate;

and if the speed gap exceeds a preset range based on the comparison result, adding 1 to the abnormal value corresponding to the driver.

As a preferred implementation of the second aspect, when running, the behavior monitoring module M5 specifically performs the following steps:

when the abnormal value of the driver is larger than a preset critical value, sending a starting instruction to a camera of the mobile terminal;

acquiring a preset specific behavior image, wherein the specific behavior image comprises behavior images of rubbing eyes, yawning, swinging heads, nipping thighs and hamstrings;

continuously receiving driver behaviors shot by a camera from the mobile terminal;

when the driver behavior in the video is compared with the pattern of the specific behavior continuously;

when the driver has a specific behavior, the next stage of fatigue driving recognition is entered.

As a preferred implementation of the second aspect, the fatigue judging module M6 specifically executes the following steps when in operation:

when the driver behavior hits one specific behavior, informing the driver to search a safe position for parking so as to perform further face recognition;

adopting a multi-frame detection algorithm to shoot a facial close-up video for a driver, and dividing the facial close-up video into a plurality of frames of face images;

acquiring a face image, and carrying out image preprocessing;

removing the noise of the face image to emphasize useful information, enhancing the image contrast, highlighting image details and improving the quality of the face image;

normalizing the face image, and adjusting the characteristic attribute values of the image to the same dimension so as to improve the model precision;

the face image and the characteristic attribute value are transmitted to a fatigue driving recognition model;

extracting a mouth region and an eye region of the face of the driver through a mouth classifier and an eye classifier;

distinguishing the characteristics of the mouth region, and distinguishing the abnormality of the shape of the mouth, the mouth opening time and the breathing rate;

carrying out feature discrimination on the eye region, and discriminating the abnormality of the eye shape, the eye closing time length and the blink rate;

and acquiring a fatigue driving judgment result output by the fatigue driving identification model.

When the device for judging driver fatigue driving based on the mobile terminal disclosed by the embodiment of the invention runs, all the steps of the method for judging driver fatigue driving based on the mobile terminal disclosed by the embodiment 1 can be executed, so that the technical effect of accurately judging driver fatigue driving based on the mobile terminal is realized.

Other structures of the device for judging driver fatigue driving based on the mobile terminal are disclosed in the prior art.

Example 3

The invention also discloses an electronic device, at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, and when the at least one processor executes the instructions, the following steps are specifically implemented: setting an abnormal value for a driver and assigning a value of 0; monitoring lane deviation times, and adding 1 to an abnormal value every time; monitoring abnormal driving rate, and adding 1 to each abnormal value; in a preset time period, if the abnormal value is larger than a preset critical value, observing a driver; continuously monitoring the behavior of a driver through a camera of the mobile terminal, and capturing preset specific behavior; if the driver has a specific behavior, carrying out face recognition on the driver to judge whether the driver has fatigue driving; and if fatigue driving exists based on the judgment result, stopping the driver from driving and forcibly resting.

Example 4

The invention also discloses a storage medium, which stores a computer program, and when the computer program is executed by a processor, the following steps are concretely realized: setting an abnormal value for a driver and assigning a value of 0; monitoring lane deviation times, and adding 1 to an abnormal value every time; monitoring abnormal driving rate, and adding 1 to each abnormal value; in a preset time period, if the abnormal value is larger than a preset critical value, observing a driver; continuously monitoring the behavior of a driver through a camera of the mobile terminal, and capturing preset specific behavior; if the driver has a specific behavior, carrying out face recognition on the driver to judge whether the driver has fatigue driving; and if fatigue driving exists based on the judgment result, stopping the driver from driving and forcibly resting.

The present disclosure may be methods, apparatus, systems, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for performing various aspects of the present disclosure.

The computer-readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be interpreted as a transitory signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + +, java, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the disclosure are implemented by personalizing an electronic circuit, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA), with state information of computer-readable program instructions, which can execute the computer-readable program instructions.

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer-readable program instructions may be provided to a processing unit of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or improvements to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (10)

1. A method for judging fatigue driving of a driver based on a mobile terminal is characterized by comprising the following steps:

setting an abnormal value for a driver and assigning a value of 0;

monitoring lane deviation times, and adding 1 to an abnormal value every time;

monitoring abnormal driving rate, and adding 1 to each abnormal value;

in a preset time period, if the abnormal value is larger than a preset critical value, observing a driver;

continuously monitoring the behavior of a driver through a camera of the mobile terminal, and capturing preset specific behavior;

if the driver has a specific behavior, carrying out face recognition on the driver to judge whether the driver has fatigue driving;

and if fatigue driving exists based on the judgment result, stopping the driver from driving and forcibly resting.

2. The method for judging driver fatigue driving based on mobile terminal according to claim 1, wherein the step of monitoring the number of lane deviations and adding 1 to each abnormal value is specifically comprised of the following substeps:

acquiring a running track of a vehicle based on a motion sensor in the mobile terminal;

judging whether the lane deviation occurs in the running track of the vehicle;

judging the time for correcting the lane deviation;

counting the abnormal times of lane deviation and correction exceeding the preset time;

and adding 1 to the abnormal value corresponding to the driver every time the lane abnormal deviation occurs and the lane abnormal deviation is corrected.

3. The method for judging driver fatigue driving based on mobile terminal as claimed in claim 2, wherein the monitoring of abnormal driving rate, adding 1 to each abnormal value, comprises the following sub-steps:

acquiring historical driving speed data of a driver;

acquiring the average driving speed of vehicles passing through the current road section;

acquiring the habitual speed of a driver based on the historical driving speed data of the driver;

dividing the habitual driving speed of the driver by the average driving speed to obtain the habitual driving speed of the driver;

dividing the current driving speed of the driver by the average driving speed based on the current driving speed of the driver to obtain the current driving speed of the driver;

comparing the current driving speed with the habitual driving speed;

and on the basis of the comparison result, if the speed difference exceeds a preset range, adding 1 to an abnormal value corresponding to the driver.

4. The method for determining driver fatigue driving based on mobile terminal as claimed in claim 3, wherein the step of continuously monitoring the driver's behavior through the camera of the mobile terminal and capturing the preset specific behavior comprises the following sub-steps:

when the abnormal value of the driver is larger than a preset critical value, sending a starting instruction to a camera of the mobile terminal;

acquiring a preset specific behavior image, wherein the specific behavior image comprises behavior images of rubbing eyes, yawning, swinging heads, nipping thighs and hamstrings;

continuously receiving driver behaviors shot by a camera from the mobile terminal;

when the driver behavior in the video is compared with the pattern of the specific behavior continuously;

when the driver has a specific behavior, the next stage of fatigue driving recognition is entered.

5. The method for determining driver fatigue driving based on mobile terminal of claim 4, wherein if the driver has a specific behavior, the driver is face-recognized to determine whether the driver has fatigue driving, specifically comprising the following sub-steps:

when the driver behavior hits one specific behavior, informing the driver to search a safe position for parking so as to perform further face recognition;

adopting a multi-frame detection algorithm to shoot a facial close-up video for a driver, and dividing the facial close-up video into a plurality of frames of face images;

acquiring a face image, and performing image preprocessing;

removing the noise of the face image to emphasize useful information, enhancing the image contrast, highlighting image details and improving the quality of the face image;

normalizing the face image, and adjusting the characteristic attribute values of the image to the same dimension so as to improve the model precision;

the face image and the characteristic attribute value are transmitted to a fatigue driving recognition model;

extracting a mouth region and an eye region of the face of the driver through a mouth classifier and an eye classifier;

distinguishing the characteristics of the mouth region, and distinguishing the abnormality of the shape of the mouth, the mouth opening time and the breathing rate;

carrying out feature discrimination on the eye region, and discriminating the abnormality of the eye shape, the eye closing time length and the blink rate;

and acquiring a fatigue driving judgment result output by the fatigue driving identification model.

6. An apparatus for discriminating fatigue driving of a driver based on a mobile terminal, comprising:

the abnormal value assignment module is used for setting an abnormal value for the driver and assigning the value to be 0;

the offset monitoring module is used for monitoring the lane offset times, and 1 is added to each abnormal value;

the speed monitoring module is used for monitoring abnormal driving speed, and adding 1 to each abnormal value;

the preliminary observation module is used for observing the driver if the abnormal value is greater than a preset critical value within a preset time period;

the behavior monitoring module is used for continuously monitoring the behavior of a driver through a camera of the mobile terminal and capturing preset specific behavior;

the fatigue judging module is used for carrying out face recognition on the driver if the driver has specific behaviors so as to judge whether the driver has fatigue driving;

and the forced execution module is used for suspending the driver to go out of the vehicle and forcibly resting if fatigue driving exists on the basis of the judgment result.

7. The device for determining driver fatigue driving based on mobile terminal of claim 6, wherein the offset monitoring module, when running, specifically performs the following steps:

acquiring a running track of a vehicle based on a motion sensor in the mobile terminal;

judging whether the lane deviation occurs in the running track of the vehicle;

judging the time for correcting the lane deviation;

counting the abnormal times of lane deviation and correction exceeding the preset time;

and adding 1 to the abnormal value corresponding to the driver every time the lane abnormal deviation occurs and the lane abnormal deviation is corrected.

8. The device for driver fatigue driving determination based on mobile terminal of claim 7, wherein the speed monitoring module, when operating, specifically performs the following steps:

acquiring historical driving speed data of a driver;

acquiring the average driving speed of vehicles passing through the current road section;

acquiring the habitual speed of a driver based on the historical driving speed data of the driver;

dividing the habitual speed of the driver by the average driving speed to obtain the habitual driving speed of the driver;

dividing the current driving speed of the driver by the average driving speed to obtain the current driving speed of the driver;

comparing the current driving speed with the habitual driving speed;

and if the speed gap exceeds a preset range based on the comparison result, adding 1 to the abnormal value corresponding to the driver.

9. The device for determining driver fatigue driving based on mobile terminal of claim 8, wherein the behavior monitoring module, when running, specifically performs the following steps:

when the abnormal value of the driver is larger than a preset critical value, sending a starting instruction to a camera of the mobile terminal;

acquiring a preset specific behavior image, wherein the specific behavior image comprises behavior images of rubbing eyes, yawning, swinging heads, nipping thighs and hamstrings;

continuously receiving driver behaviors shot by a camera from the mobile terminal;

when the driver behavior in the video is compared with the pattern of the specific behavior continuously;

when the driver has a specific behavior, the fatigue driving recognition of the next stage is entered.

10. The device for determining driver fatigue driving based on mobile terminal of claim 9, wherein the fatigue determination module, when running, specifically performs the following steps:

when the driver behavior hits one specific behavior, informing the driver to search a safe position for parking so as to perform further face recognition;

adopting a multi-frame detection algorithm to shoot a facial close-up video for a driver, and dividing the facial close-up video into a plurality of frames of face images;

acquiring a face image, and carrying out image preprocessing;

removing the noise of the face image to emphasize useful information, enhancing the image contrast, highlighting image details and improving the quality of the face image;

normalizing the face image, and adjusting the characteristic attribute values of the image to the same dimension so as to improve the model precision;

the face image and the characteristic attribute value are transmitted to a fatigue driving recognition model;

extracting a mouth region and an eye region of a driver's face by a mouth classifier and an eye classifier;

carrying out characteristic discrimination on the mouth region, and discriminating the abnormality of the shape, the mouth opening duration and the breathing rate of the mouth region;

carrying out feature discrimination on the eye region, and discriminating the abnormality of the eye shape, the eye closing time length and the blink rate;

and acquiring a fatigue driving judgment result output by the fatigue driving identification model.

Images