CN109584507A

CN109584507A - Driver behavior modeling method, apparatus, system, the vehicles and storage medium

Info

Publication number: CN109584507A
Application number: CN201811339289.4A
Authority: CN
Inventors: 刘国清; 杨广
Original assignee: Shenzhen Minieye Innovation Technology Co Ltd
Current assignee: Hangzhou Ruijian Zhixing Technology Co ltd
Priority date: 2018-11-12
Filing date: 2018-11-12
Publication date: 2019-04-05
Anticipated expiration: 2038-11-12
Also published as: CN109584507B

Abstract

This application involves a kind of driver behavior modeling method, apparatus, system, the vehicles and storage mediums.The described method includes: executing detection operation to the video information in the vehicles of acquisition, determine whether the current driving condition of driver is abnormal driving state；Wherein, the detection operation includes diverting one's attention to drive detection and fatigue driving detection；If the current driving condition is abnormal driving state, early warning operation corresponding with the abnormal driving state is executed.Comprehensive monitoring to fatigue driving state, both abnormal driving states of driving condition of diverting one's attention can be realized using this method, and realize targetedly efficient early warning by executing early warning operation corresponding with abnormal driving state, drop ground drives risk.

Description

Driver behavior modeling method, apparatus, system, the vehicles and storage medium

Technical field

This application involves technical field of vehicle, more particularly to a kind of driver behavior modeling method, apparatus, system, traffic Tool and storage medium.

Background technique

With the development of safe driving of vehicle technology, driving behavior detection system (The driver behavior Detection system) it is important as one of driver assistance system (The driver assistance systems) Technology has broad application prospects and the pass by more and more research institutions and scholar in terms of the prevention of traffic accident Note.

Generally, driving behavior detection system is by the driving behavior of real-time monitoring driver and driving condition, can be with Unsafe driver behavior is found ahead of time, is improved the safe driving consciousness of driver, is avoided major traffic accidents.

However, the narrow range of the detection of current driving behavior detection system, only makes further investigation to fatigue driving, And other factors being related to very few, this undoubtedly increases driving risk.

Summary of the invention

Based on this, it is necessary to which in view of the above technical problems, providing one kind can be realized including driving detection and the fatigue of diverting one's attention Driver behavior modeling method, apparatus, system, the vehicles and the storage for driving the comprehensive driving detection including detection are situated between Matter.

In a first aspect, a kind of driver behavior modeling method, which comprises

To in the vehicles of acquisition video information execute detection operation, determine driver current driving condition whether For abnormal driving state；Wherein, the detection operation includes diverting one's attention to drive detection and fatigue driving detection；

If the current driving condition is abnormal driving state, early warning behaviour corresponding with the abnormal driving state is executed Make.

The detection operation in one of the embodiments, further include: abnormal speed change drives detection, and the method is also wrapped It includes:

The abnormal speed change is executed to the running state information of the vehicles of acquisition and drives detection.

The video information execution in the vehicles of described pair of acquisition diverts one's attention to drive detection in one of the embodiments, Include:

Each picture frame of the video information in the first preset time period is obtained, and is calculated by the detection of preset elliptic contour Method handles each picture frame, obtains each pixel of steering wheel region in each picture frame；

According to the pixel value of preset Gauss complexion model and each pixel of the steering wheel region, each figure is calculated As each pixel of steering wheel region in frame belongs to the characteristic probability of area of skin color；

According to the size of each picture frame corresponding characteristic probability and predetermined probabilities threshold value, the current driving shape is identified Whether state is driving condition of diverting one's attention.

In one of the embodiments, according to the big of the corresponding characteristic probability of each picture frame and predetermined probabilities threshold value It is small, identify whether the current driving condition is driving condition of diverting one's attention, comprising:

Calculate the difference of the corresponding characteristic probability of two picture frames of arbitrary continuation in each picture frame；

If the difference of the characteristic probability is greater than predetermined probabilities threshold value, it is determined that the current driving condition is to divert one's attention to drive State.

Each picture frame of the video information in the first preset time period is obtained, and passes through preset face recognition technology pair Each picture frame is handled, and each pixel in mouth region in each picture frame is obtained；

Calculate the character pixel value of each pixel in mouth region in each picture frame；

According to each corresponding character pixel value of picture frame and default character pixel threshold value, the current driving shape is identified Whether state is driving condition of diverting one's attention.

The video information in the vehicles of described pair of acquisition executes fatigue driving detection in one of the embodiments, Include:

Each picture frame of the video information in the first preset time period is obtained, and passes through preset face recognition technology pair Each picture frame is handled, and the closure frequency of human face characteristic point is calculated；

If the closure frequency of the human face characteristic point is greater than predeterminated frequency threshold value, it is determined that the current driving condition is tired Labor driving condition.

If human face characteristic point is eyes in one of the embodiments, the closure frequency for calculating human face characteristic point, Include:

According to the human eye state in each picture frame, the number of winks in first preset time period is counted；It is described Human eye state includes opening state and closed state；

According to first preset time period and the number of winks, the frequency of wink of eyes is calculated.

If human face characteristic point is mouth in one of the embodiments, the closure frequency for calculating human face characteristic point, Include:

According to the mouth states in each picture frame, the mouth counted in first preset time period is closed number； The mouth states include open configuration and closed state；

It is closed number according to first preset time period and the mouth, calculates the closure frequency of mouth.

The running state information of the vehicles of described pair of acquisition executes the abnormal speed change in one of the embodiments, Drive detection, comprising:

Obtain the angle information of the velocity information of vehicle and steering wheel in the second preset time period；

According to the velocity information of the vehicle, the characteristic velocity and feature for obtaining vehicle in second preset time period add Speed；And the angle information according to the steering wheel, obtain the feature angular acceleration of vehicle in second preset time period；

According to the characteristic velocity and default characteristic velocity threshold value, the characteristic acceleration and default characteristic acceleration threshold Value and the feature angular acceleration and default angular acceleration threshold value identify whether the current driving condition is abnormal speed change Driving condition.

Second aspect, a kind of driver behavior modeling device, described device include:

First detection module executes detection operation for the video information in the vehicles to acquisition, determines driver Current driving condition whether be abnormal driving state；Wherein, the detection operation includes diverting one's attention to drive detection and fatigue driving Detection；

Warning module executes and the abnormal driving shape if being abnormal driving state for the current driving condition The corresponding early warning operation of state.

The third aspect, a kind of driver behavior modeling system, comprising: video information acquisition device, speed acquisition device, angle Acquisition device, memory and processor, the memory are stored with computer program, and the processor executes the computer journey It is performed the steps of when sequence

Fourth aspect, a kind of vehicles, including above-mentioned driver behavior modeling system.

5th aspect, a kind of computer readable storage medium are stored thereon with computer program, the computer program quilt Processor performs the steps of when executing

Above-mentioned driver behavior modeling method, apparatus, system, the vehicles and storage medium pass through the traffic work to acquisition Video information in tool executes detection operation, determines whether the current driving condition of driver is abnormal driving state, and working as When preceding driving condition is abnormal driving state, early warning operation corresponding with abnormal driving state is executed, to realize including diverting one's attention to drive The comprehensive driving detection including detection and fatigue driving detection is sailed, therefore may be implemented to fatigue driving state, divert one's attention to drive The comprehensive monitoring of both abnormal driving states of state is sailed, and real by executing early warning operation corresponding with abnormal driving state Now targetedly efficient early warning, drop ground drive risk.

Detailed description of the invention

Fig. 1 is the applied environment figure of driver behavior modeling method in one embodiment；

Fig. 2 is the flow diagram of driver behavior modeling method in one embodiment；

Fig. 3 is the flow diagram of driver behavior modeling method in another embodiment；

Fig. 4 is a kind of flow diagram for diverting one's attention to drive detecting step in one embodiment；

Fig. 5 is that another kind diverts one's attention to drive the flow diagram of detecting step in one embodiment；

Fig. 6 is a kind of flow diagram of fatigue driving detecting step in one embodiment；

Fig. 7 is a kind of flow diagram of abnormal speed change driving detecting step in one embodiment；

Fig. 8 is the schematic diagram of driver behavior modeling method in one embodiment；

Fig. 9 is the structural block diagram of driver behavior modeling device in one embodiment；

Figure 10 is the structural block diagram of driver behavior modeling device in another embodiment；

Figure 11 is the structural block diagram of driver behavior modeling system in one embodiment.

Specific embodiment

It is with reference to the accompanying drawings and embodiments, right in order to which the objects, technical solutions and advantages of the application are more clearly understood The application is further elaborated.It should be appreciated that specific embodiment described herein is only used to explain the application, not For limiting the application.

Driver behavior modeling method provided by the present application, can be applied in terminal device, be suitable for all kinds of vehicles, The including but not limited to vehicles such as vehicle, ship.It is understood that when the vehicles are ship, in the present embodiment Steering wheel can be rudder for ship.By taking vehicle as an example, terminal device can be, but not limited to be various car-mounted terminals or notebook electricity Brain, smart phone, tablet computer and portable wearable device；Car-mounted terminal can be stand-alone terminal or be integrated in vehicle-mounted more In the devices such as media termination, driver assistance system.Shown in referring to Fig.1, by taking vehicle 10 as an example, above-mentioned terminal device can be vehicle Mounted terminal 12, the car-mounted terminal 12 can be connect with the video information acquisition device 14 of car installation, obtain interior video information； The car-mounted terminal 12 can also be connect with speed acquisition device 15, the angle acquisition device 16 of car installation, obtain the fortune of vehicle Row status information；Car-mounted terminal 12 can determine driver 11 according to interior video information and/or the running state information of vehicle Current driving condition whether be abnormal driving state.For example, car-mounted terminal can pass through preset elliptic contour detection algorithm With Gauss complexion model, detection operation is executed to the video information in the vehicles of acquisition, determines that the both hands of driver 11 are No off-direction disk 13 determines whether current driving condition is abnormal driving state.

In addition, car-mounted terminal can also be smart phone, which can believe with the video installed in the vehicles Breath acquisition device, speed acquisition device, angle acquisition device interact；For example, the image collecting device and smart phone are equal Have the short-range communication modules such as bluetooth module, then smart phone can carry out short haul connection with the image collecting device；Again For example, above-mentioned speed acquisition device, angle acquisition device can be interacted with driver assistance system, and smart phone then pass through it is short The modes such as field communication module or internet are interacted with driver assistance system.In short, the present embodiment is not limited to State example.

In one embodiment, as shown in Fig. 2, executing detection behaviour by the video information in the vehicles to acquisition Make, determine whether the current driving condition of driver is abnormal driving state, and is abnormal driving state in current driving condition When, early warning operation corresponding with abnormal driving state is executed, to realize including diverting one's attention to drive detection and fatigue driving detection Comprehensive driving detection and corresponding early warning operation.Above-mentioned driver behavior modeling method, with the vehicle-mounted end being applied in Fig. 1 It is illustrated, may comprise steps of for end:

S201 executes detection operation to the video information in the vehicles of acquisition, determines the current driving shape of driver Whether state is abnormal driving state.

Wherein, detection operation includes diverting one's attention to drive detection and fatigue driving detection；Diverting one's attention to drive may include in drive the cross Cheng Zhong, driver's both hands off-direction disk, at least one the divert one's attention situation such as driver makes a phone call, driver smokes；Fatigue driving It may include that in the process of moving, driver is because of tired situations such as frequently yawning, blinking.Therefore, in the present embodiment, vehicle Mounted terminal can obtain the face of driver by the video information in the image acquisition device vehicle that is installed in the vehicle Image, hand images or steering wheel image etc. simultaneously execute detection operation, to determine whether driver is in driving condition of diverting one's attention Or the abnormal drivings state such as fatigue driving state.

It is understood that S201 may include: when the vehicles are in driving condition, in the vehicles of acquisition Video information execute detection operation, determine whether the current driving condition of driver is abnormal driving state.Wherein, traffic work Tool is greater than preset stop threshold speed in the speed that driving condition may include: the vehicles.For example, for vehicle and Speech, preset stop threshold speed can be 3km/h.Therefore, it when the vehicles are in driving condition, just carries out such as S201 institute The detection operation shown；And when the vehicles are in the non-driving conditions such as resting state, then without detection operation and it is subsequent Early warning operation, can so save detection and operate corresponding resource, can also avoid causing wrong early warning when non-driving condition.

Certainly, in the present embodiment, multiple images acquisition device can be installed in the vehicles, for acquire it is multiple not With the video information in the vehicles in orientation, such as the facial video information and the vehicles of driver can be acquired respectively Steering wheel video information, and fatigue driving detection is executed to the facial video information of the driver of acquisition, and to acquisition The steering wheel video information execution of the vehicles diverts one's attention to drive detection, by obtaining more accurately video information, redundancy is avoided to regard The interference of frequency information improves the accuracy of detection operation.

S202 executes early warning operation corresponding with abnormal driving state if current driving condition is abnormal driving state.

Above-mentioned early warning operation may include the various modes such as video playing form, text importing form, voice broadcast mode Early warning operation.Illustratively, it if current driving condition is the corresponding abnormal driving state of driving condition of diverting one's attention, executes and divides The corresponding early warning operation of heart driving condition, early warning operation can be for reminding driver attentively to drive presetting of not diverting one's attention Voice plays；Further, when this divert one's attention driving condition be driver's both hands off-direction disk the case where when, the early warning operation It can be the default voice for reminding driver's both hands to grasp steering wheel to play.Similarly, if current driving condition is fatigue The corresponding abnormal driving state of driving condition, then execute corresponding with fatigue driving state early warning operation, which operates can be with It is the default voice content broadcasting for reminding driver's parked vehicles to rest.In short, by executing and abnormal driving shape Precisely effective early warning operation may be implemented in the corresponding early warning operation of state, reduces and drives risk.

In above-mentioned driver behavior modeling method, detection operation is executed by the video information in the vehicles to acquisition, Whether the current driving condition for determining driver is abnormal driving state, and when current driving condition is abnormal driving state, Early warning corresponding with abnormal driving state is executed to operate, it is complete including driving of diverting one's attention detects and fatigue driving detects to realize The driving in orientation detects, thus may be implemented to fatigue driving state, driving condition both the abnormal driving states of diverting one's attention it is complete Directional surveillance, and targetedly efficient early warning is realized by executing early warning operation corresponding with abnormal driving state, drop ground drives Risk.

In one embodiment, as shown in figure 3, diverting one's attention to drive by the video information execution in the vehicles to acquisition Detection and fatigue driving detection, and abnormal speed change is executed to the running state information of the vehicles of acquisition and drives detection, come Whether the current driving condition for determining driver is abnormal driving state, and when current driving condition is abnormal driving state, Early warning operation corresponding with abnormal driving state is executed, includes diverting one's attention to drive detection, fatigue driving detection and exception with realization Speed change drives the comprehensive driving detection and corresponding early warning operation including detection.Above-mentioned driver behavior modeling method, to answer For being illustrated for the car-mounted terminal in Fig. 1, may comprise steps of:

S301 diverts one's attention to drive detection and fatigue driving is detected to the video information execution in the vehicles of acquisition, and Abnormal speed change is executed to the running state information of the vehicles of acquisition and drives detection, determines that the current driving condition of driver is No is abnormal driving state.

Wherein, the driving of abnormal speed change may include in the process of moving, because of driver's operational issue or vehicle problem, Vehicle is caused the abnormal speed change situation such as exceed the speed limit, suddenly accelerate, take a sudden turn, bringing to a halt occur.Above-mentioned exception speed change situation can pass through The running state information of vehicle is obtained to determine, such as is equipped in advance when the time that the brake pedal of vehicle is in brake height is greater than When the dynamic time, then the vehicle has been likely to occur the abnormal speed change situation brought to a halt.When the operating status of the vehicles to acquisition Information executes abnormal speed change and drives detection, when detecting that abnormal speed change occur in the vehicles, it is determined that driver's currently drives State is sailed as this abnormal driving state of abnormal speed change driving condition.

It should be noted that divert one's attention to drive detection, fatigue driving detection and abnormal speed change drive detection these three detections can To influence each other: when driver is in fatigue driving state, because fatigue leads to driving demand, power is low, and then diverts one's attention to drive, It may cause not in place to the grasp of throttle and gear lever and speed made to mutate；When driver is in driving condition of diverting one's attention When, it blocks sight since the behaviors such as smoke, make a phone call make to go smoothly and makes driver distraction or since hand leaves operating stick or side To disk, cause to lose hold of to vehicle, the mutation of Yi Yinfa speed；When driver is in abnormal speed change driving condition, generally Be due to fatigue driving and divert one's attention drive and lead to vehicle speed variation.It influences each other due to being connected each other between three, then any inspection Surveying exception can all illustrate that driver is in abnormal driving state；Therefore Through Several Survey Measure is used in the present embodiment, indirectly and The driving behavior for directly detecting driver, is capable of the driving behavior of comprehensive acquisition driver, makes the behavior of driver Feature is richer, rather than simple some feature performance is reduced and driven to improve the comprehensive and accuracy for driving monitoring Risk.

Optionally, divert one's attention to drive detection to the video information execution in the vehicles of acquisition in reference Fig. 4, S301, it can To include:

S401, obtains each picture frame of video information in the first preset time period, and is detected by preset elliptic contour Algorithm handles each picture frame, obtains each pixel of steering wheel region in each picture frame.

Up to working as at the time of above-mentioned first preset time period generally can be for from apart from the first preset time period of current time Period between the preceding moment, time span can be configured according to the actual situation, for example, above-mentioned first preset time period It can be the period within 10 seconds current times or 5 seconds.Illustratively, if the frame per second of the video is 12fps (number of pictures per second), when the first preset time segment length is 5 seconds, the picture frame of the video information of acquisition can be 60, then Direction in the picture frame can be identified using preset elliptic contour detection algorithm to each picture frame in 60 picture frames Disk area obtains each pixel of direction disk area, can specifically obtain the coordinate value of each pixel.Above-mentioned elliptic contour Algorithm can be using in the library opencv (Open Source Computer Vision Library, computer vision library of increasing income) Contour detecting Ellipses Detection, can also use the Ellipses Detection based on Hough transformation, and based on it is random suddenly The Ellipses Detection etc. of husband's transformation.

It is understood that the profile of steering wheel is generally circular in cross section, but in above-mentioned picture frame all directions disk profile because The relationship of shooting angle is generally ellipse, therefore can be identified in image by preset elliptic contour detection algorithm Each outer characteristic point of the outer layer elliptic contour of steering wheel is formed, and forms each interior feature of the internal layer elliptic contour of steering wheel Point, and then the steering wheel region formed by the corresponding oval oval encirclement corresponding with each interior characteristic point of each outer characteristic point is obtained, Further obtain each pixel of direction disk area.

S402 calculates each image according to the pixel value of preset Gauss complexion model and each pixel of steering wheel region Each pixel of steering wheel region belongs to the characteristic probability of area of skin color in frame.

In the present embodiment, the pixel of each pixel of the available above-mentioned steering wheel region identified of car-mounted terminal Value specifically can obtain pixel value corresponding with the coordinate value according to the coordinate value of each pixel of steering wheel region.Example Property, when each picture frame is color image, the pixel value of each pixel can be indicated using YCb-Cr color space, it is bright ignoring It spends under the influence of component Y, (Cb, Cr) can be expressed as^T, wherein Cb is chroma blue component, and Cr is red chrominance component.Phase Ying Di can be in the present embodiment two dimension according to the distribution trend of chromatic component Cb and Cr in the pixel value of skin pixel point Gaussian Profile is built using a large amount of skin pixel points in the hand color image of the driver obtained in advance as training sample Vertical correspondingly Gauss complexion model G=(m, C), in which:

x_iFor the pixel value (Cb of skin pixel point i in training sample_i,Cr_i)^T, n is skin pixel point in training sample Number, m indicate that the Mean Matrix of the pixel value of each skin pixel point in training sample, C indicate each skin pixel in training sample The covariance matrix of the pixel value of point, then when the pixel value of the pixel in above-mentioned each picture frame in steering wheel region is x, the picture Vegetarian refreshments belongs to the probability of area of skin color are as follows:

P (x)=exp (- 0.5 (x-m)^TC^-1(x-m))

It illustratively, can be according to the distribution trend of the gray value of skin pixel point when each picture frame is gray level image For one-dimensional gaussian profile, using a large amount of skin pixel points in the hand gray level image of the driver obtained in advance as training sample This, establishes correspondingly Gauss complexion model.Similarly, in the present embodiment, color space can also be empty using colors such as RGB Between, but other colour of skin spaces are compared, YCbCr color space is carrying out being not easy to be illuminated by the light when face complexion identification and other objects Interference, accuracy are higher.

It can be the image that each pixel of steering wheel region, which belongs to the characteristic probability of area of skin color, in above-mentioned each picture frame Each pixel in frame in steering wheel region belong to the average value of the probability value of area of skin color, maximum value, minimum value or other Statistical value, the present embodiment comparison are not intended to limit.In addition, in the present embodiment, it can be other using oval complexion model etc. Complexion model is more concerned with the colour of skin for the detection in terms of profile compared to oval complexion model, and Gauss complexion model advantage exists It is more specific in terms of color, suitable for detection of the colour of skin in terms of color.

S403 identifies current driving condition according to the size of each picture frame corresponding characteristic probability and predetermined probabilities threshold value It whether is driving condition of diverting one's attention.

Illustratively, in the present embodiment, the corresponding characteristic probability of each picture frame can be preset generally with preset Rate threshold value is compared, and each picture frame which shoots when can be driver's both hands steer direction disk is corresponding The average value of each characteristic probability, then, if there are the corresponding feature of at least one picture frame is general in each picture frame of video information Rate is less than predetermined probabilities threshold value, then at the time of meaning that the picture frame corresponds to, that is, the possible both hands of driver should departing from steering wheel Moment driving condition is driving condition of diverting one's attention, then it is assumed that current driving condition is driving condition of diverting one's attention.

Specifically, S403 may include: the corresponding characteristic probability of two picture frames for calculating arbitrary continuation in each picture frame Difference；If the difference of characteristic probability is greater than predetermined probabilities threshold value, it is determined that current driving condition is driving condition of diverting one's attention.Example Property, in the present embodiment, each picture frame pair which shoots when can be driver's both hands steer direction disk The standard deviation for the characteristic probability value answered, then two picture frames that can calculate arbitrary continuation in each picture frame of video information are corresponding Characteristic probability difference；The difference of the corresponding characteristic probability of at least a pair of continuous two picture frames is default greater than this if it exists Probability threshold value then means within above-mentioned continuous two picture frames corresponding time, the both hands of driver and connecing for steering wheel Touching state has occurred compared with big change, it is meant that the possible both hands of driver are departing from steering wheel, then it is assumed that current driving condition is point Heart driving condition.

Similarly, in the present embodiment, each figure shot when predetermined probabilities threshold value is driver's both hands steer direction disk When as the corresponding maximum characteristic probability value of frame and the difference of minimal characteristic probability value, each picture frame of video information can also be calculated It is pre- to characterize above-mentioned first for the difference of corresponding maximum characteristic probability value and minimal characteristic probability value, the difference characterized by the difference If the maximum changing amplitude of the corresponding characteristic probability value of each picture frame in the period；It is apparent that if this feature difference and default general The difference of rate threshold value is less than preset difference value, then means the both hands of driver and connecing for steering wheel in first preset time period Touching state has occurred compared with big change, it is meant that the possible both hands of driver are departing from steering wheel, then it is assumed that current driving condition is point Heart driving condition.

In addition, in the present embodiment, car-mounted terminal can also obtain the direction information and gear information of vehicle, according to vehicle Direction information and gear information the result for diverting one's attention to drive detection is modified, be detected as point to avoid by normal driving state Heart driving condition.For example, when the steering of vehicle changes, the case where generally driver's steer direction disk just will appear, Then such situation should not be identified as the corresponding driving condition of diverting one's attention of hand off-direction disk；In another example the gear when vehicle occurs When variation, it is meant that the hand of driver may off-direction disk simultaneously in setting gear, then the both hands of driver and side at this time It has occurred to the contact condition of disk compared with big change, such situation should not be identified as driving condition of diverting one's attention.

Optionally, divert one's attention to drive detection to the video information execution in the vehicles of acquisition in reference Fig. 5, S301, it can To include:

S501 obtains each picture frame of video information in the first preset time period, and passes through preset face recognition technology Each picture frame is handled, each pixel in mouth region in each picture frame is obtained.

It is understood that cigarette butt is located at mouth region, because flickering for cigarette butt leads to mouth area when driver smokes The pixel value in domain is frequently mutated, and can be judged to divert one's attention driving condition caused by whether driver is in smoking according to this feature. Therefore, in the present embodiment, the human face region in each picture frame can be identified by pre-set face recognition technology Mouth region, and then obtain each pixel in mouth region.

Specifically, which can know for the face based on multi-task network (multi-task learning network) Other technology can carry out the positioning of face and human face characteristic point simultaneously, have speed fast on same hardware device, effect is good Advantage, can carry out on the mobile apparatus using.By the training sample training largely comprising face and inhuman face image In obtained multi-task network, the image comprising driver's face can be inputted, output obtains one and remembered with collimation mark Good face and the feature point image marked with point.Multi-task Face datection network based on deep learning can be used Three cascade modes of CNN (convolutional neural networks, convolutional neural networks) are by Face datection and face Characteristic point is detected while being carried out, and above three CNN includes: P-Net, R-Net and O-Net, wherein P-Net is a full convolution Network is used for generating candidate frame and frame regression vector (bounding box regression vectors) The method of Bounding box regression calibrates these candidate frames, and non-maxima suppression (NMS) is used to merge overlapping Candidate frame；R-Net will be inputted in R-Net, refusal falls most of false's for improving candidate frame by the candidate frame of P-Net Candidate frame continues to use Bounding box regression and NMS merging, and O-Net is for exporting final candidate frame (people Face frame and characteristic point position).

S502 calculates the character pixel value of each pixel in mouth region in each picture frame.

In the present embodiment, the character pixel value of each pixel in mouth region can be the image in above-mentioned each picture frame The average value of the pixel value of each pixel in mouth region, maximum value, minimum value or other statistical values in frame；Optionally, when When pixel value is rgb value, this feature pixel value is the maximum of R value in the pixel value of each pixel in mouth region in the picture frame Value, because the R value of the corresponding pixel value of cigarette butt is maximum R value in mouth region when cigarette butt vigorous combustion.

S503 identifies current driving condition according to the corresponding character pixel value of each picture frame and default character pixel threshold value It whether is driving condition of diverting one's attention.

Illustratively, in the present embodiment, the corresponding character pixel value of each picture frame can be preset with preset Character pixel threshold value is compared, and each picture frame which shoots when can smoke for driver is corresponding The average value of each character pixel value, then, if there are the corresponding features of at least one picture frame in each picture frame of video information When the difference of pixel value and default character pixel threshold value is less than preset difference value, then at the time of meaning that the picture frame corresponds to, drive Member may smoke, i.e., the moment driving condition is driving condition of diverting one's attention, then it is assumed that current driving condition is driving condition of diverting one's attention.

Similarly, in the present embodiment, it is shot when default character pixel threshold value is driver's both hands steer direction disk When the corresponding maximum character pixel value of each picture frame and the difference of minimal characteristic pixel value, each figure of video information can also be calculated As the difference of frame corresponding maximum character pixel value and minimal characteristic pixel value, the difference characterized by the difference characterizes above-mentioned the The maximum changing amplitude of the corresponding character pixel value of each picture frame in one preset time period；It is apparent that if this feature difference and pre- If the difference of character pixel threshold value is less than preset difference value, then mean that driver is smoking in first preset time period, then Think that current driving condition is driving condition of diverting one's attention.

Optionally, referring to shown in Fig. 6, fatigue driving inspection is executed to the video information in the vehicles of acquisition in S301 It surveys, may include:

S601 obtains each picture frame of video information in the first preset time period, and passes through preset face recognition technology Each picture frame is handled, the closure frequency of human face characteristic point is calculated.

In the present embodiment, can by the above-mentioned face recognition technology based on multi-task network to each picture frame into Row recognition of face and human face characteristic point identification, then identify and count the closed state or opening of human face characteristic point in each picture frame State calculates the closure frequency of human face characteristic point with this.

Specifically, if human face characteristic point is eyes, the closure frequency of human face characteristic point is calculated, comprising: according to each image Human eye state in frame counts the number of winks in the first preset time period；Human eye state includes opening state and closed state； According to the first preset time period and number of winks, the frequency of wink of eyes is calculated.It illustratively, can be by comparing each picture frame The character pixel value of each pixel of middle eye areas and preset skin pixel value determine, such as when eyes in each picture frame When the character pixel value of each pixel in region and smaller preset skin pixel value difference, it is meant that eyes area in the picture frame Each pixel in domain is biased to the colour of skin, then eye state is closed state in the picture frame；Conversely, when eye areas in each picture frame Each pixel character pixel value and preset skin pixel value difference it is larger when, it is meant that eye areas in the picture frame Each pixel deviation colour of skin is larger, then eye state is to open state in the picture frame.It is understood that when in each picture frame Eye state is to open state in a upper picture frame, when eye state is closed state in next image frame, it is meant that eyes warp Primary blink is crossed, then can count to obtain the number of winks in the first preset time period, and then divided by the first preset time period Time span obtains frequency of wink.

Specifically, if human face characteristic point is mouth, the closure frequency of human face characteristic point is calculated, comprising: according to each image Mouth states in frame, the mouth counted in the first preset time period are closed number；Mouth states include open configuration and closure State；It is closed number according to the first preset time period and mouth, calculates the closure frequency of mouth.It is understood that above-mentioned meter The mode of the closure frequency and above-mentioned calculating frequency of wink of calculating mouth is essentially identical, and which is not described herein again.

S602, if the closure frequency of human face characteristic point is greater than predeterminated frequency threshold value, it is determined that current driving condition is fatigue Driving condition.

For example, when frequency of wink is greater than predeterminated frequency threshold value, it is meant that driver's blink is excessively frequent, it may be possible to because Fatigue leads to blink frequently, therefore appropriate preceding driving condition is fatigue driving state；When mouth closure frequency is greater than predeterminated frequency When threshold value, it is meant that driver's mouth closure is excessively frequent, it may be possible to because fatigue causes to yawn, drive shape before appropriate State is fatigue driving state.It should be noted that the corresponding predeterminated frequency threshold value of different human face characteristic points is different, example Such as, the corresponding predeterminated frequency threshold value of mouth is less than the corresponding predeterminated frequency threshold value of eyes.

Optionally, referring to shown in Fig. 7, abnormal speed change is executed to the running state information of the vehicles of acquisition in S301 and is driven Sail detection, comprising:

S701 obtains the velocity information of vehicle and the angle information of steering wheel in the second preset time period.

Above-mentioned second preset time period is similar with above-mentioned first preset time period, but difference is arranged in the time span of the two. Generally, because when the second preset time period is corresponding the angle information of the velocity information and steering wheel of vehicle transient change May be larger, thus the time span of above-mentioned second preset time period can be set it is shorter, such as 2 seconds.Illustratively, above-mentioned Velocity information may include the speed of each moment corresponding vehicle in above-mentioned second preset time period, and the angle information of steering wheel can To include the angle of each moment corresponding steering wheel in above-mentioned second preset time period.

S702, information, the characteristic velocity and feature for obtaining vehicle in the second preset time period accelerate according to the speed of vehicle Degree；And the angle information according to steering wheel, obtain the feature angular acceleration of vehicle in the second preset time period.

Illustratively, characteristic velocity can for moment each in above-mentioned second preset time period corresponding vehicle speed most Big value；Furthermore, it is possible to which it is corresponding to calculate each moment according to the speed of moment each in above-mentioned second preset time period corresponding vehicle The acceleration of vehicle, characteristic acceleration can for moment each in above-mentioned second preset time period corresponding vehicle acceleration most Big value；It is corresponding can also to calculate each moment according to the angle of moment each in above-mentioned second preset time period corresponding steering wheel The angular acceleration of steering wheel, feature angular acceleration can be the angle of moment each in above-mentioned second preset time period corresponding steering wheel The maximum value of acceleration.

S703, according to characteristic velocity and default characteristic velocity threshold value, characteristic acceleration and default characteristic acceleration threshold value, with And feature angular acceleration and default angular acceleration threshold value, identify whether current driving condition is abnormal speed change driving condition.

If characteristic velocity is greater than pre-set velocity threshold value, mean that vehicle may exceed the speed limit；If characteristic acceleration is greater than default Acceleration rate threshold then means that vehicle suddenly may accelerate or take a sudden turn；If feature angular acceleration is greater than default angular acceleration threshold value, Then mean that vehicle may take a sudden turn；Therefore car-mounted terminal can determine current driving condition for abnormal speed change driving condition.

S302 executes early warning operation corresponding with abnormal driving state if current driving condition is abnormal driving state.

Illustratively, if current driving condition is the corresponding abnormal driving state of abnormal speed change driving condition, execute with The corresponding early warning operation of abnormal speed change driving condition, early warning operation can be for reminding driver to control vehicle smooth-ride Default voice play.

Referring to shown in Fig. 8, in the present embodiment, the video image in vehicle can be acquired by camera, can be passed through The methods of gray processing, image filtering, histogram equalization carry out image preprocessing, carry out direction by elliptic contour detection algorithm Disk positioning, and the positioning by multi-task network progress Face detection and eyes and mouth；It may finally be by right Whether each pixel than the steering wheel region between successive frame belongs to the probability of area of skin color and substantially change, and judges driver Whether the corresponding driving condition of diverting one's attention of hand off-direction disk is in；And the mouth region between comparison successive frame can be passed through Whether the pixel value of each pixel substantially change, and judges whether driver is in corresponding driving condition of diverting one's attention of smoking；And Can by comparison successive frame between frequency of wink and/or mouth be closed frequency whether substantially change, judge that driver is It is no to be in fatigue driving state；And comparison car speed, vehicle acceleration, the angular acceleration of steering wheel and each can be passed through Self-corresponding threshold value, judges whether driver is in abnormal speed change driving condition.

In above-mentioned driver behavior modeling method, divert one's attention to drive inspection by the video information execution in the vehicles to acquisition It surveys and fatigue driving detects, and abnormal speed change is executed to the running state information of the vehicles of acquisition and drives detection, it is next true Whether the current driving condition for determining driver is abnormal driving state, and when current driving condition is abnormal driving state, is held Row early warning operation corresponding with abnormal driving state drives detection, fatigue driving detection and abnormal change including diverting one's attention to realize Speed drives the comprehensive driving detection and corresponding early warning operation including detection, thus may be implemented to fatigue driving state, The comprehensive monitoring for driving condition and abnormal these three the abnormal driving states of speed change driving condition of diverting one's attention, and by execute with it is different Targetedly efficient early warning, drop ground driving risk are realized in the often corresponding early warning operation of driving condition.

It should be understood that although each step in the flow chart of Fig. 2-7 is successively shown according to the instruction of arrow, These steps are not that the inevitable sequence according to arrow instruction successively executes.Unless expressly stating otherwise herein, these steps Execution there is no stringent sequences to limit, these steps can execute in other order.Moreover, at least one in Fig. 2-7 Part steps may include that perhaps these sub-steps of multiple stages or stage are not necessarily in synchronization to multiple sub-steps Completion is executed, but can be executed at different times, the execution sequence in these sub-steps or stage is also not necessarily successively It carries out, but can be at least part of the sub-step or stage of other steps or other steps in turn or alternately It executes.

In one embodiment, as shown in figure 9, providing a kind of driver behavior modeling device 90, comprising: the first detection mould Block 91 and warning module 92, in which:

First detection module 91 executes detection operation for the video information in the vehicles to acquisition, determines and drive Whether the current driving condition of member is abnormal driving state；Wherein, detection operation includes diverting one's attention to drive detection and fatigue driving inspection It surveys；

Warning module 92 executes corresponding with abnormal driving state if being abnormal driving state for current driving condition Early warning operation.

Optionally, referring to Fig.1 shown in 0, on the basis of above-mentioned Fig. 9, detection operation further include: abnormal speed change drives inspection It surveys, device further include: the second detection module 93, the running state information for the vehicles to acquisition execute abnormal speed change and drive Sail detection.

Optionally, referring to Fig.1 shown in 0, first detection module 91 may include:

Steering wheel recognition unit 911, for obtaining each picture frame of video information in the first preset time period, and by pre- If elliptic contour detection algorithm each picture frame is handled, obtain each pixel of steering wheel region in each picture frame；

Computation of differential characteristic probability unit 912, for each pixel according to preset Gauss complexion model and steering wheel region Pixel value, each pixel for calculating steering wheel region in each picture frame belongs to the characteristic probability of area of skin color；

First, which diverts one's attention, drives unit 913, for according to the big of the corresponding characteristic probability of each picture frame and predetermined probabilities threshold value It is small, identify whether current driving condition is driving condition of diverting one's attention.

Optionally, first diverts one's attention to drive unit 913 for calculating two picture frames correspondence of arbitrary continuation in each picture frame Characteristic probability difference；If the difference of characteristic probability is greater than predetermined probabilities threshold value, it is determined that current driving condition is to divert one's attention to drive Sail state.

Mouth region identification block 914 obtains each picture frame of video information in the first preset time period, and by default Face recognition technology each picture frame is handled, obtain each pixel in mouth region in each picture frame；

Character pixel value computing unit 915, for calculating the character pixel of each pixel in mouth region in each picture frame Value；

Second, which diverts one's attention, drives unit 916, for according to the corresponding character pixel value of each picture frame and default character pixel threshold Value, identifies whether current driving condition is driving condition of diverting one's attention.

It is closed probability calculation unit 917, for obtaining each picture frame of video information in the first preset time period, and is passed through Preset face recognition technology handles each picture frame, calculates the closure frequency of human face characteristic point；

Fatigue driving unit 918, if the closure frequency for human face characteristic point is greater than predeterminated frequency threshold value, it is determined that current Driving condition is fatigue driving state.

Optionally, if human face characteristic point is eyes, probability calculation unit 917 is closed for according to the people in each picture frame Eye shape state counts the number of winks in the first preset time period；Human eye state includes opening state and closed state；According to first Preset time period and number of winks calculate the frequency of wink of eyes.

Optionally, if human face characteristic point is mouth, probability calculation unit 917 is closed for according to the mouth in each picture frame Bar state, the mouth counted in the first preset time period are closed number；Mouth states include open configuration and closed state；According to First preset time period and mouth are closed number, calculate the closure frequency of mouth.

Optionally, referring to Fig.1 shown in 0, the second detection module 93 may include:

Operating status acquiring unit 931, for obtaining the velocity information and steering wheel of vehicle in the second preset time period Angle information；

Operating status computing unit 932 obtains vehicle in the second preset time period for information according to the speed of vehicle Characteristic velocity and characteristic acceleration；And the angle information according to steering wheel, obtain the feature of vehicle in the second preset time period Angular acceleration；

Abnormal speed change drives unit 933, for according to characteristic velocity and default characteristic velocity threshold value, characteristic acceleration and pre- If characteristic acceleration threshold value and feature angular acceleration and default angular acceleration threshold value, identify whether current driving condition is different Normal speed change driving condition.

In above-mentioned driver behavior modeling device, detection operation is executed by the video information in the vehicles to acquisition, Whether the current driving condition for determining driver is abnormal driving state, and when current driving condition is abnormal driving state, Early warning corresponding with abnormal driving state is executed to operate, it is complete including driving of diverting one's attention detects and fatigue driving detects to realize The driving in orientation detects, thus may be implemented to fatigue driving state, driving condition both the abnormal driving states of diverting one's attention it is complete Directional surveillance, and targetedly efficient early warning is realized by executing early warning operation corresponding with abnormal driving state, drop ground drives Risk.

Specific about driver behavior modeling device limits the limit that may refer to above for driver behavior modeling method Fixed, details are not described herein.Modules in above-mentioned driver behavior modeling device can fully or partially through software, hardware and its Combination is to realize.Above-mentioned each module can be embedded in the form of hardware or independently of in the processor in computer equipment, can also be with It is stored in the memory in computer equipment in a software form, in order to which processor calls the above modules of execution corresponding Operation.

In one embodiment, referring to Fig.1 shown in 1, a kind of driver behavior modeling system is provided, comprising: video information Acquisition device, speed acquisition device, angle acquisition device, memory and processor, memory are stored with computer program, processing Device performs the steps of when executing computer program

To in the vehicles of acquisition video information execute detection operation, determine driver current driving condition whether For abnormal driving state；Wherein, detection operation includes diverting one's attention to drive detection and fatigue driving detection；

If current driving condition is abnormal driving state, early warning operation corresponding with abnormal driving state is executed.

Wherein, video information acquisition device can be vehicle-mounted camera, and camera may be mounted at vehicle right front, tilting peace It puts, the main face for being directed at driver, camera horizontal view angle and vertical angle of view can be finely tuned according to setting angle.Optionally, Camera can use 6 layers of full glass lens, and setting angle can be 60 ° of horizontal view angle (HFOV), vertical angle of view (VFOV) 45 °, specific setting angle can be depending on vehicle, in order to all behaviors of driver, sensitive chip in collecting vehicle It can be CMOS digital image sensor AR0132AT.The video information acquisition device has integrated level high, low in energy consumption, at low cost The advantages of, wherein camera can well adapt to interior light and shade situation of change using 6 layers of full glass lens.

The speed acquisition device that uses, angle acquisition device are respectively induced for Bus- Speed Monitoring sensor and angular transducer The situation of change of speed and the situation of change of steering wheel angle, wherein Bus- Speed Monitoring sensor is detected using Hall sensor Revolving speed measures the revolving speed of wheel, and then measures speed；Processor is main control chip, signal output end, the vehicle of angular transducer The signal output end of fast detection sensor and the signal output end of video information acquisition device can be with the letters of main control chip The connection of number input terminal, the signal output end of main control chip can be bi-directionally connected with communicating circuit；Wherein, active chip, angle pass Sensor, Bus- Speed Monitoring sensor, video information acquisition device and communicating circuit have power circuit power supply, and there is structure simply may be used By property height, the small feature of such environmental effects.

In one embodiment, the detection operation further include: abnormal speed change drives detection, and processor executes computer journey It is also performed the steps of when sequence and the abnormal speed change driving detection is executed to the running state information of the vehicles of acquisition.

In one embodiment, it is also performed the steps of when processor executes computer program and obtains the first preset time Section in the video information each picture frame, and by preset elliptic contour detection algorithm to each picture frame at Reason obtains each pixel of steering wheel region in each picture frame；According to preset Gauss complexion model and the steering wheel The pixel value of each pixel in region, each pixel for calculating steering wheel region in each picture frame belong to the spy of area of skin color Levy probability；According to the size of each picture frame corresponding characteristic probability and predetermined probabilities threshold value, the current driving shape is identified Whether state is driving condition of diverting one's attention.

In one embodiment, it is also performed the steps of when processor executes computer program and calculates each picture frame The difference of the corresponding characteristic probability of two picture frames of middle arbitrary continuation；If the difference of the characteristic probability is greater than predetermined probabilities threshold Value, it is determined that the current driving condition is driving condition of diverting one's attention.

In one embodiment, it is also performed the steps of when processor executes computer program and obtains the first preset time Each picture frame of the video information in section, and each picture frame is handled by preset face recognition technology, it obtains Take each pixel in mouth region in each picture frame；Calculate the feature of each pixel in mouth region in each picture frame Pixel value；According to each corresponding character pixel value of picture frame and default character pixel threshold value, the current driving shape is identified Whether state is driving condition of diverting one's attention.

In one embodiment, it is also performed the steps of when processor executes computer program and obtains the first preset time Each picture frame of the video information in section, and each picture frame is handled by preset face recognition technology, it counts Calculate the closure frequency of human face characteristic point；If the closure frequency of the human face characteristic point is greater than predeterminated frequency threshold value, it is determined that described Current driving condition is fatigue driving state.

In one embodiment, if human face characteristic point is eyes, following step is also realized when processor executes computer program It is rapid: according to the human eye state in each picture frame, to count the number of winks in first preset time period；The human eye shape State includes opening state and closed state；According to first preset time period and the number of winks, the blink of eyes is calculated Frequency.

In one embodiment, if human face characteristic point is mouth, following step is also realized when processor executes computer program Rapid: according to the mouth states in each picture frame, the mouth counted in first preset time period is closed number；The mouth Bar state includes open configuration and closed state；It is closed number according to first preset time period and the mouth, calculates mouth Bar closure frequency.

In one embodiment, it is also performed the steps of when processor executes computer program and obtains the second preset time The velocity information of vehicle and the angle information of steering wheel in section；According to the velocity information of the vehicle, it is pre- to obtain described second If the characteristic velocity and characteristic acceleration of vehicle in the period；And the angle information according to the steering wheel, obtain described the The feature angular acceleration of vehicle in two preset time periods；According to the characteristic velocity and default characteristic velocity threshold value, the feature Acceleration and default characteristic acceleration threshold value and the feature angular acceleration and default angular acceleration threshold value are worked as described in identification Whether preceding driving condition is abnormal speed change driving condition.

In one embodiment, shown in referring to Fig.1, a kind of vehicles, including above-mentioned driver behavior modeling system are provided System.

In one embodiment, a kind of computer readable storage medium is provided, computer program is stored thereon with, is calculated Machine program performs the steps of when being executed by processor

In one embodiment, the detection operation further include: abnormal speed change drives detection, and computer program is by processor It is also performed the steps of when execution and the abnormal speed change driving detection is executed to the running state information of the vehicles of acquisition.

In one embodiment, when computer program is executed by processor also perform the steps of acquisition first it is default when Between in section the video information each picture frame, and by preset elliptic contour detection algorithm to each picture frame at Reason obtains each pixel of steering wheel region in each picture frame；According to preset Gauss complexion model and the steering wheel The pixel value of each pixel in region, each pixel for calculating steering wheel region in each picture frame belong to the spy of area of skin color Levy probability；According to the size of each picture frame corresponding characteristic probability and predetermined probabilities threshold value, the current driving shape is identified Whether state is driving condition of diverting one's attention.

In one embodiment, it is also performed the steps of when computer program is executed by processor and calculates each image The difference of the corresponding characteristic probability of two picture frames of arbitrary continuation in frame；If the difference of the characteristic probability is greater than predetermined probabilities Threshold value, it is determined that the current driving condition is driving condition of diverting one's attention.

In one embodiment, when computer program is executed by processor also perform the steps of acquisition first it is default when Between in section the video information each picture frame, and each picture frame is handled by preset face recognition technology, Obtain each pixel in mouth region in each picture frame；Calculate the spy of each pixel in mouth region in each picture frame Levy pixel value；According to each corresponding character pixel value of picture frame and default character pixel threshold value, the current driving is identified Whether state is driving condition of diverting one's attention.

In one embodiment, when computer program is executed by processor also perform the steps of acquisition first it is default when Between in section the video information each picture frame, and each picture frame is handled by preset face recognition technology, Calculate the closure frequency of human face characteristic point；If the closure frequency of the human face characteristic point is greater than predeterminated frequency threshold value, it is determined that institute Stating current driving condition is fatigue driving state.

In one embodiment, it is also realized if human face characteristic point is eyes, when computer program is executed by processor following Step: according to the human eye state in each picture frame, the number of winks in first preset time period is counted；The human eye State includes opening state and closed state；According to first preset time period and the number of winks, blinking for eyes is calculated Eye frequency.

In one embodiment, it is also realized if human face characteristic point is mouth, when computer program is executed by processor following Step: according to the mouth states in each picture frame, the mouth counted in first preset time period is closed number；It is described Mouth states include open configuration and closed state；It is closed number according to first preset time period and the mouth, is calculated The closure frequency of mouth.

In one embodiment, when computer program is executed by processor also perform the steps of acquisition second it is default when Between the velocity information of vehicle and the angle information of steering wheel in section；According to the velocity information of the vehicle, described second is obtained The characteristic velocity and characteristic acceleration of vehicle in preset time period；And the angle information according to the steering wheel, described in acquisition The feature angular acceleration of vehicle in second preset time period；According to the characteristic velocity and default characteristic velocity threshold value, the spy Acceleration and default characteristic acceleration threshold value and the feature angular acceleration and default angular acceleration threshold value are levied, described in identification Whether current driving condition is abnormal speed change driving condition.

Those of ordinary skill in the art will appreciate that realizing all or part of the process in above-described embodiment method, being can be with Relevant hardware is instructed to complete by computer program, the computer program can be stored in a non-volatile computer In read/write memory medium, the computer program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein, To any reference of memory, storage, database or other media used in each embodiment provided herein, Including non-volatile and/or volatile memory.Nonvolatile memory may include read-only memory (ROM), programming ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM) or flash memory.Volatile memory may include Random access memory (RAM) or external cache.By way of illustration and not limitation, RAM is available in many forms, Such as static state RAM (SRAM), dynamic ram (DRAM), synchronous dram (SDRAM), double data rate sdram (DDRSDRAM), enhancing Type SDRAM (ESDRAM), synchronization link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic ram (DRDRAM) and memory bus dynamic ram (RDRAM) etc..

Each technical characteristic of above embodiments can be combined arbitrarily, for simplicity of description, not to above-described embodiment In each technical characteristic it is all possible combination be all described, as long as however, the combination of these technical characteristics be not present lance Shield all should be considered as described in this specification.

The several embodiments of the application above described embodiment only expresses, the description thereof is more specific and detailed, but simultaneously It cannot therefore be construed as limiting the scope of the patent.It should be pointed out that coming for those of ordinary skill in the art It says, without departing from the concept of this application, various modifications and improvements can be made, these belong to the protection of the application Range.Therefore, the scope of protection shall be subject to the appended claims for the application patent.

Claims

1. a kind of driver behavior modeling method, which is characterized in that the described method includes:

Detection operation is executed to the video information in the vehicles of acquisition, determines whether the current driving condition of driver is different Normal driving condition；Wherein, the detection operation includes diverting one's attention to drive detection and fatigue driving detection；

If the current driving condition is abnormal driving state, early warning operation corresponding with the abnormal driving state is executed.

2. the method according to claim 1, wherein the detection operates further include: abnormal speed change drives detection, The method also includes:

3. the method according to claim 1, wherein the video information in the vehicles of described pair of acquisition executes Divert one's attention to drive detection, comprising:

Each picture frame of the video information in the first preset time period is obtained, and passes through preset elliptic contour detection algorithm pair Each picture frame is handled, and each pixel of steering wheel region in each picture frame is obtained；

According to the pixel value of preset Gauss complexion model and each pixel of the steering wheel region, each picture frame is calculated Each pixel of middle steering wheel region belongs to the characteristic probability of area of skin color；

According to the size of each picture frame corresponding characteristic probability and predetermined probabilities threshold value, identify that the current driving condition is No is driving condition of diverting one's attention.

4. according to the method described in claim 3, it is characterized in that, according to the corresponding characteristic probability of each picture frame and default The size of probability threshold value identifies whether the current driving condition is driving condition of diverting one's attention, comprising:

If the difference of the characteristic probability is greater than predetermined probabilities threshold value, it is determined that the current driving condition is to divert one's attention to drive shape State.

5. the method according to claim 1, wherein the video information in the vehicles of described pair of acquisition executes Divert one's attention to drive detection, comprising:

Each picture frame of the video information in the first preset time period is obtained, and by preset face recognition technology to described Each picture frame is handled, and each pixel in mouth region in each picture frame is obtained；

According to each corresponding character pixel value of picture frame and default character pixel threshold value, identify that the current driving condition is No is driving condition of diverting one's attention.

6. the method according to claim 1, wherein the video information in the vehicles of described pair of acquisition executes Fatigue driving detection, comprising:

Each picture frame of the video information in the first preset time period is obtained, and by preset face recognition technology to described Each picture frame is handled, and the closure frequency of human face characteristic point is calculated；

If the closure frequency of the human face characteristic point is greater than predeterminated frequency threshold value, it is determined that the current driving condition is driven for fatigue Sail state.

7. according to the method described in claim 6, it is characterized in that, the calculating face is special if human face characteristic point is eyes Levy the closure frequency of point, comprising:

According to the human eye state in each picture frame, the number of winks in first preset time period is counted；The human eye State includes opening state and closed state；

8. according to the method described in claim 6, it is characterized in that, the calculating face is special if human face characteristic point is mouth Levy the closure frequency of point, comprising:

According to the mouth states in each picture frame, the mouth counted in first preset time period is closed number；It is described Mouth states include open configuration and closed state；

9. according to the method described in claim 2, it is characterized in that, the running state information of the vehicles of described pair of acquisition is held The row abnormal speed change drives detection, comprising:

According to the velocity information of the vehicle, the characteristic velocity and feature for obtaining vehicle in second preset time period accelerate Degree；And the angle information according to the steering wheel, obtain the feature angular acceleration of vehicle in second preset time period；

According to the characteristic velocity and default characteristic velocity threshold value, the characteristic acceleration and default characteristic acceleration threshold value, with And whether the feature angular acceleration and default angular acceleration threshold value, the identification current driving condition are that abnormal speed change drives shape State.

10. a kind of driver behavior modeling device, which is characterized in that described device includes:

First detection module executes detection operation for the video information in the vehicles to acquisition, determines working as driver Whether preceding driving condition is abnormal driving state；Wherein, the detection operation includes diverting one's attention to drive detection and fatigue driving detection；

Warning module executes and the abnormal driving state pair if being abnormal driving state for the current driving condition The early warning operation answered.

11. a kind of driver behavior modeling system characterized by comprising video information acquisition device, speed acquisition device, angle Acquisition device, memory and processor are spent, the memory is stored with computer program, and the processor executes the computer The step of any one of claims 1 to 9 the method is realized when program.

12. a kind of vehicles, which is characterized in that including the driver behavior modeling system described in claim 11.

13. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program The step of method described in any one of claims 1 to 9 is realized when being executed by processor.