CN107729870A - The method and apparatus of in-car safety monitoring based on computer vision - Google Patents
The method and apparatus of in-car safety monitoring based on computer vision Download PDFInfo
- Publication number
- CN107729870A CN107729870A CN201711059359.6A CN201711059359A CN107729870A CN 107729870 A CN107729870 A CN 107729870A CN 201711059359 A CN201711059359 A CN 201711059359A CN 107729870 A CN107729870 A CN 107729870A
- Authority
- CN
- China
- Prior art keywords
- image
- car
- limbs
- driver
- different personnel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/19—Sensors therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
Abstract
The invention discloses a kind of method and apparatus of the in-car safety monitoring based on computer vision, it is related to image identification technical field, can solve the problem that the problem of monitoring efficiency of in-car conflict monitoring is low.The method of the present invention mainly includes：Obtain the in-car image shot in preset time period；By carrying out human body skin detection to the in-car image, the limbs image of different personnel in the in-car image is obtained；The limbs image of different personnel carries out optical flow computation in the in-car image adjacent to shooting time, determines movement tendency of the limbs of the different personnel in the preset time period；According to the movement tendency of different personnel in the in-car image, judge whether that limbs conflict occurs.Invention is primarily adapted for use in the scene for carrying out safety monitoring when driving.
Description
Technical field
The present invention relates to image identification technical field, more particularly to a kind of in-car safety monitoring based on computer vision
Method and apparatus.
Background technology
In recent years because the strangers such as share-car trip, windward driving trip share the popularization of trip mode, how to monitor shared
In-car environment during trip, the security for judging stroke, become a urgent problem to be solved.Wherein, in-car personnel punching
Prominent is an important factor for threatening in-car security context.And existing in-car monitoring system be can not automatic detection go out occupant and be
It is no to be conflicted, the video recording of artificial checking monitoring can only be relied on just to can determine that in-car the problem of conflicting with the presence or absence of personnel, led
Cause the ageing and efficiency of in-car safety detection all very low.
The content of the invention
In view of this, the method and apparatus of a kind of in-car safety monitoring based on computer vision provided by the invention, it is main
The monitoring efficiency for being to solve the problems, such as in-car conflict monitoring of syllabus is low.
In order to solve the above problems, present invention generally provides following technical scheme：
In a first aspect, the invention provides a kind of method of the in-car safety monitoring based on computer vision, this method bag
Include following steps：
In-car picture is obtained by camera；
The in-car picture got is analyzed；Wherein analysis includes driver's driving privilege analysis, driver
Driving behavior analysis, passenger's passenger choice behavior analysis；
Judged by analysis in-car whether safe.
Second aspect, the invention provides a kind of device of the in-car safety monitoring based on computer vision, the device bag
Include：
Acquisition module, for obtaining in-car picture by camera；
Analysis module, for analyzing the in-car picture got；Wherein analysis includes driver's driving power
Limit analysis, driver's driving behavior analysis, passenger's passenger choice behavior analysis；
Judge module, it is in-car whether safe for being judged by analysis.
The third aspect, the invention provides a kind of method of the in-car safety monitoring based on computer vision, this method bag
Include：
Obtain the in-car image shot in preset time period；
By carrying out human body skin detection to the in-car image, the limbs figure of different personnel in the in-car image is obtained
Picture；
The limbs image of different personnel carries out optical flow computation in the in-car image adjacent to shooting time, determines the difference
Movement tendency of the limbs of personnel in the preset time period；
According to the movement tendency of different personnel in the in-car image, judge whether that limbs conflict occurs.
Fourth aspect, the present invention also provide a kind of device of the in-car safety monitoring based on computer vision, the device bag
Include：
Acquiring unit, for obtaining the in-car image shot in preset time period；
Skin detection unit, for carrying out human body skin detection to the in-car image, obtain in the in-car image not
With the limbs image of personnel；
Trend determining unit, the limbs image for different personnel in the in-car image adjacent to shooting time carry out light stream
Calculate, determine movement tendency of the limbs of the different personnel in the preset time period；
Conflict judging unit, for the movement tendency according to different personnel in the in-car image, judges whether that limb occurs
Body conflict.
To achieve these goals, the 5th aspect, the invention provides a kind of storage medium, the storage medium is stored with
A plurality of instruction, the instruction be applied to be loaded as processor and as described in terms of performing such as first or three based on computer vision
The method of in-car safety monitoring.
To achieve these goals, the 6th aspect, the invention provides a kind of in-car safety prison based on computer vision
Terminal is surveyed, the terminal includes storage medium and processor；
The processor, it is adapted for carrying out each instruction；
The storage medium, suitable for storing a plurality of instruction；
The instruction be suitable to be loaded as the processor and as described in terms of performing such as first or three based on computer vision
In-car safety monitoring method.
By above-mentioned technical proposal, technical scheme provided by the invention at least has following advantages：
The method and apparatus of in-car safety monitoring provided by the invention based on computer vision, without relying upon artificial next true
Fixed in-car conflicts with the presence or absence of personnel, and the limbs image of different personnel carries out light stream in in-car image only adjacent to shooting time
Calculate, movement tendency of the limbs of in-car different personnel in preset time period is obtained, further according to the motion of these occupants
Trend can in real time and automatically determines in-car whether there occurs limbs conflict；And the limbs image of different personnel is only logical
Cross and the in-car image progress human body skin detection can shot in preset time period is obtained automatically.Utilize the method for the present invention
Full-automatic monitoring in real time can be carried out to in-car security context with device, substantially increases the ageing of in-car safety detection and effect
Rate.
Described above is only the general introduction of technical solution of the present invention, in order to better understand the technological means of the present invention,
And can be practiced according to the content of specification, and in order to allow above and other objects of the present invention, feature and advantage can
Become apparent, below especially exemplified by the embodiment of the present invention.
Brief description of the drawings
By reading the detailed description of hereafter preferred embodiment, it is various other the advantages of and benefit it is common for this area
Technical staff will be clear understanding.Accompanying drawing is only used for showing the purpose of preferred embodiment, and is not considered as to the present invention
Limitation.And in whole accompanying drawing, identical part is denoted by the same reference numerals.In the accompanying drawings：
Fig. 1 shows a kind of stream of the method for in-car safety monitoring based on computer vision provided in an embodiment of the present invention
Journey schematic diagram；
Fig. 2 shows a kind of knot of the device of in-car safety monitoring based on computer vision provided in an embodiment of the present invention
Structure schematic diagram；
Fig. 3 shows the method for another in-car safety monitoring based on computer vision provided in an embodiment of the present invention
Schematic flow sheet；
Fig. 4 shows the method for another in-car safety monitoring based on computer vision provided in an embodiment of the present invention
Flow chart；
Fig. 5 shows the method for another in-car safety monitoring based on computer vision provided in an embodiment of the present invention
Schematic flow sheet；
Fig. 6 shows the method for another in-car safety monitoring based on computer vision provided in an embodiment of the present invention
Schematic flow sheet；
Fig. 7 shows the device of another in-car safety monitoring based on computer vision provided in an embodiment of the present invention
Structural representation；
Fig. 8 shows the device of another in-car safety monitoring based on computer vision provided in an embodiment of the present invention
Structural representation；
Fig. 9 shows the device of another in-car safety monitoring based on computer vision provided in an embodiment of the present invention
Structural representation；
Figure 10 shows the device of another in-car safety monitoring based on computer vision provided in an embodiment of the present invention
Structural representation.
Embodiment
The exemplary embodiment of the disclosure is more fully described below with reference to accompanying drawings.Although the disclosure is shown in accompanying drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here
Limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure
Completely it is communicated to those skilled in the art.
Fig. 1 is the flow signal of the method for the in-car safety monitoring based on computer vision of one embodiment of the invention
Figure.
This method comprises the following steps：
Step S1, in-car picture is obtained by camera；
Step S2, the in-car picture got is analyzed；Wherein analysis includes driver's driving privilege analysis,
Driver's driving behavior analysis, passenger's passenger choice behavior analysis；
Wherein, driver's privilege analysis identifies including Face datection：
Obtain in-car picture；
Major-minor is driven and carries out Face datection, obtains human face region；
Driver is identified, judges whether to possess driving authority.
Wherein, driver's driving behavior analysis includes line-of-sight detection：
Positioned by human eye to obtain driver's binocular position according to in-car picture；
Detection pilot's line of vision direction in real time；
Judge whether driver is not concerned with road surface ahead situation for a long time；
If it is not, the information that gives a warning.
Wherein, passenger's passenger choice behavior analysis includes：
Skin detection：By carrying out color sample to the human face region obtained by Face datection, and it is based on the existing colour of skin
Data analysis, obtain scope of the respective skin color of each occupant in the value of YCrCb color spaces.
Limbs conflict detects：It is in-car every because the camera fixed in advance is positioned at the one known position in in-car front
Individual corresponds to the different regions of each two field picture of camera, by skin detection, can obtain the naked of each occupant
The skin of dew gets rid of the location of known face, it becomes possible to substantially obtain everyone in the position of each two field picture
The position of trunk.Light stream is calculated to front and rear two field pictures.When the limbs that major-minor drives contact with each other, both are judged by algorithm
Whether limbs conflict occurring, for conflict, alert be present, and recording the in-car picture of surrounding time section,
Dispute processing is carried out for third party.
Step S3, judged by analysis in-car whether safe.
Compared with prior art, the present invention has advantages below：The following two approach of generally use in the prior art：Scheme
One, mainly take terminal device gps signal, judge vehicle whether for a long time in abnormality (such as be parked in for a long time some ground
Side), take the measure of alarm logging for the vehicle of exception.Scheme two, by carrying out ONLINE RECOGNITION to driver, judge that driver is
No is me, is realized to human pilot in line justification.Scheme one lacks the analysis to in-car actual conditions, it is impossible to which processing is just
Often the department that occurs multiplies dispute during traveling or short stay, and because lacking in-vehicle information, dispute can not be judged with
Processing.Scheme two only realizes the authentication to driver, lacks the analysis to copilot passenger, and can not handle the two
The scene clashed.And in the solution of the present invention, in-car picture is obtained by camera；To the passenger inside the vehicle got
Analyzed, carried out for driver's safety detection in itself and directly potential for driver and passenger with the situation of driver
Limbs and the safety monitoring of speech conflict, more comprehensively.
Fig. 2 is the structural representation of the device of the in-car safety monitoring based on computer vision of one embodiment of the invention
Figure.
The device includes：
Acquisition module 10, for obtaining in-car picture by camera；
Analysis module 20, for analyzing the in-car picture got；Wherein analysis includes driver's driving
Privilege analysis, driver's driving behavior analysis, passenger's passenger choice behavior analysis；
Wherein, driver's privilege analysis identifies including Face datection：
Obtain in-car picture；
Major-minor is driven and carries out Face datection, obtains human face region；
Driver is identified, judges whether to possess driving authority.
Wherein, driver's driving behavior analysis includes line-of-sight detection：
Positioned by human eye to obtain driver's binocular position according to in-car picture；
Detection pilot's line of vision direction in real time；
Judge whether driver is not concerned with road surface ahead situation for a long time；
If it is not, the information that gives a warning.
Wherein, passenger's passenger choice behavior analysis includes：
Skin detection：By carrying out color sample to the human face region obtained by Face datection, and it is based on the existing colour of skin
Data analysis, obtain scope of the respective skin color of each occupant in the value of YCrCb color spaces.
Limbs conflict detects：It is in-car every because the camera fixed in advance is positioned at the one known position in in-car front
Individual corresponds to the different regions of each two field picture of camera, by skin detection, can obtain the naked of each occupant
The skin of dew gets rid of the location of known face, it becomes possible to substantially obtain everyone in the position of each two field picture
The position of trunk.Light stream is calculated to front and rear two field pictures.When the limbs that major-minor drives contact with each other, both are judged by algorithm
Whether limbs conflict occurring, for conflict, alert be present, and recording the in-car picture of surrounding time section,
Dispute processing is carried out for third party.
Judge module 30, it is in-car whether safe for being judged by analysis.
Compared with prior art, the present invention has advantages below：The following two approach of generally use in the prior art：Scheme
One, mainly take terminal device gps signal, judge vehicle whether for a long time in abnormality (such as be parked in for a long time some ground
Side), take the measure of alarm logging for the vehicle of exception.Scheme two, by carrying out ONLINE RECOGNITION to driver, judge that driver is
No is me, is realized to human pilot in line justification.Scheme one lacks the analysis to in-car actual conditions, it is impossible to which processing is just
Often the department that occurs multiplies dispute during traveling or short stay, and because lacking in-vehicle information, dispute can not be judged with
Processing.Scheme two only realizes the authentication to driver, lacks the analysis to copilot passenger, and can not handle the two
The scene clashed.And in the solution of the present invention, in-car picture is obtained by camera；To the passenger inside the vehicle got
Analyzed, carried out for driver's safety detection in itself and directly potential for driver and passenger with the situation of driver
Limbs and the safety monitoring of speech conflict, more comprehensively.
In order to improve the monitoring efficiency of in-car conflict monitoring, the embodiments of the invention provide a kind of based on computer vision
The method of in-car safety monitoring, as shown in figure 3, this method mainly includes：
101st, the in-car image shot in preset time period is obtained.
Rearview camera is installed using in-car, records in-car image during vehicle traveling or touch at.Afterwards
The center position of in-car front end can be installed depending on camera, such as installed in the position of rearview mirror, driven so as to obtain in-car
The person of sailing and the image of passenger, and in-car different personnel can accordingly be included in image different zones.Judging that in-car is
During no generation limbs conflict, transfer the image that rearview camera records in preset time period and analyzed, by being in image
Whether there occurs limbs conflict to judge occupant for the movement tendency of the personnel of middle different zones.Ensure that to obtain car
The interior accurate movement tendency of personnel, avoid of short duration action as the lasting movement tendency of occupant, such as by passenger to department
Machine pass money payment of short duration action when passenger to impact driver movement tendency, it is necessary to rule of thumb data setting occupant send out
Whether the duration of limb motion, occur according to the limb motion Trend judgement of personnel in the duration during raw limbs conflict
Limbs conflict.
102nd, by carrying out human body skin detection to the in-car image, the limb of different personnel in the in-car image is obtained
Body image.
According to color progress of the span of human skin color in YCrCb color spaces to the in-car image of acquisition
Match somebody with somebody, the pixel that color in image is belonged to the span is defined as skin element, and the region that skin element forms is defined as
The limbs image of occupant.The limbs image for being in image different zones is defined as to the limbs image of different personnel.Will be pre-
If each two field picture that rearview camera is shot in the period carries out human body skin detection respectively, to obtain in-car different personnel
Position of the limbs in each two field picture.
103rd, the limbs image of different personnel carries out optical flow computation in the in-car image adjacent to shooting time, it is determined that described
Movement tendency of the limbs of different personnel in the preset time period.
Carry out optical flow computation to the limbs images of every a pair of consecutive frames, for example, in preset time period rearview camera clap
Limbs image in limbs image and the second two field picture in the first two field picture taken the photograph carries out optical flow computation, obtains in one or two liang of frames
The position change of occupant in time；To the limb in the 3rd two field picture of rearview camera shooting in preset time period
Limbs image in body image and the 4th two field picture carries out optical flow computation, obtains the limbs of the occupant in three or four liang of frame times
Change in location.By that analogy, change in location of each frame time of limbs relative to the previous frame time is obtained, then according to all
Change in location draws out movement tendency of the limbs of occupant in preset time period.
104th, according to the movement tendency of different personnel in the in-car image, judge whether that limbs conflict occurs.
The movement tendency for the limbs for belonging to different personnel is compared, judges the limbs of the different personnel in preset time period
Whether there is the trend persistently to be contacted or have continuous contact action；If, it is determined that there occurs limbs conflict by occupant.
The method of in-car safety monitoring provided by the invention based on computer vision, in-car is determined without relying upon artificial
Conflicting with the presence or absence of personnel, the limbs image of different personnel carries out optical flow computation in in-car image only adjacent to shooting time,
Obtain movement tendency of the limbs of in-car different personnel in preset time period, further according to these occupants movement tendency just
In real time and it can automatically determine in-car whether there occurs limbs conflict；And the limbs image of different personnel is only by pre-
If the in-car image progress human body skin detection can of shooting in the period obtains automatically.Utilize methods and apparatus of the present invention
Full-automatic monitoring in real time can be carried out to in-car security context, substantially increases the ageing and efficiency of in-car safety detection.
Based on above-described embodiment, another embodiment of the present invention further provides another in-car based on computer vision
The method of safety monitoring, referring to the drawings shown in 4, this method mainly includes：
201st, the in-car image shot in preset time period is obtained.
The implementation of step 201 and the step 101 of above-described embodiment are identical, will not be repeated here.
202nd, by carrying out Face datection to the in-car image, the face-image of different personnel is obtained.
Enhancing processing is carried out to the image of acquisition, increases the brightness and contrast of image, and remove the noise spot in image.
Facial feature extraction operation is carried out to the image after processing using convolutional neural networks, obtains each occupant in image
Facial image.
203rd, colour of skin sampling is carried out to the face-image of the different personnel, obtains the colour of skin sample of different personnel.
Color sample is carried out to the facial image of each occupant, the color of sampling is defined as corresponding occupant
Blee, and by blee be used as corresponding to occupant colour of skin sample, car is further determined by colour of skin sample
The limbs colour of skin of interior personnel.
204th, colour of skin span corresponding to each colour of skin sample is determined.
Each colour of skin sample is analyzed based on colour of skin empirical data, it is empty in YCrCb colors to obtain each colour of skin sample
Between in corresponding colour of skin span, the colour of skin span is the value of the limbs colour of skin of occupant corresponding to colour of skin sample
Scope.
205th, the limbs figure that the image for belonging to the colour of skin span in the in-car image is corresponding personnel is determined
Picture.
According to color progress of the colour of skin span of the occupant of acquisition to image-region corresponding to occupant
Match somebody with somebody, color in image-region is belonged to the skin element of occupant corresponding to the pixel determination of colour of skin span, by image
All regions for belonging to colour of skin span skin element composition are defined as the limb of occupant corresponding to image-region in region
Body image.The limbs image of different personnel in each two field picture that rearview camera is shot in preset time period is determined, to obtain
Position of the limbs of different personnel in each two field picture in pick-up.
206th, the limbs image of different personnel carries out optical flow computation in the in-car image adjacent to shooting time, it is determined that described
Movement tendency of the limbs of different personnel in the preset time period.
The implementation of step 206 and the step 103 of above-described embodiment are identical, will not be repeated here.
If the 207th, determine the limbs of the different personnel in the preset time according to the movement tendency of the different personnel
There is the trend persistently to be contacted in section or have continuous contact action, then judge limbs conflict occurs.
If determined according to the movement tendency of the different personnel in preset time period in the image of different personnel limbs into by
Gradually the limbs in close trend or image are in the trend of continuous overlapping, illustrate that occupant will fight or beat
Bucket, that is, judge that there occurs limbs conflict by occupant.
If the 208, judging limbs conflict occurs, limbs conflict warning message is sent out to server.
When judging that limbs conflict occurs for occupant, limbs conflict warning message is sent to server immediately, so as to
Staff checks.
209th, be locally stored the preset time period in-car image and the preset time period before and after time adjacent segments
In-car image.
The duration of the time adjacent segments is pre-set.
For convenience of subsequently by in-car record be supplied to third party handle dispute when check, when judge occupant generation limbs
During conflict, the in-car image of the period clashed is stored in local, subsequently to transfer.And in order to more accurately sentence
Disconnected the reason for clashing, while by the image before clashing and image afterwards while it is stored in local.Can basis
Image how long after demand setting storage clashes preceding image how long and clashed.Such as：When judgement is sent out
During raw conflict, the in-car image of 1 minute before and after the in-car image of preset time period and preset time period is locally stored.
210th, by the in-car of time adjacent segments before and after the in-car image of the preset time period and the preset time period
Image is uploaded to the server.
For convenience of subsequently by in-car record be supplied to third party handle dispute when check, when judge occupant generation limbs
During conflict, by the in-car image of the period clashed and the image of period and this period before this period
The image storage of period is uploaded to server afterwards, subsequently to transfer.The in-car image relevant with in-car conflict can be stood
Server can also be uploaded in special time by being uploaded to server.
Based on above-described embodiment, another embodiment of the present invention further provides another in-car based on computer vision
The method of safety monitoring, referring to the drawings shown in 5, this method mainly includes：
301st, the in-car image shot in preset time period is obtained.
302nd, by carrying out human body skin detection to the in-car image, the limb of different personnel in the in-car image is obtained
Body image.
303rd, the limbs image of different personnel carries out optical flow computation in the in-car image adjacent to shooting time, it is determined that described
Movement tendency of the limbs of different personnel in the preset time period.
304th, according to the movement tendency of different personnel in the in-car image, judge whether that limbs conflict occurs.
Step 301-304 implementation is identical with the step 101-104 of above-described embodiment, will not be repeated here.
305th, by carrying out Face datection to the in-car image, the face-image of driver is obtained.
The image in region corresponding with position of driver is obtained from in-car image, driver's image of acquisition is increased
Strength is managed, and increases the brightness and contrast of image, and remove the noise spot in image.Using convolutional neural networks to processing after
Image carries out facial feature extraction operation, obtains the face-image of the driver in image.
306th, the face-image in the face-image of the driver and identity information storehouse is contrasted, driven described in judgement
Whether the person of sailing has the driving authority of Current vehicle.
The identity information storehouse is used to store the face-image of the driver with different vehicle driving authority and different cars
Vehicles identifications corresponding relation.
The face-image for the driver that the face-image of acquisition and identity information storehouse right of possession limit are driven into vehicle is carried out pair
Than if it is identical to contrast two face-images, it is determined that the current driver for driving vehicle, which has, drives authority；If contrast
Go out two face-image differences, it is determined that the current driver for driving vehicle is without driving authority.
If the driver the 307th, is judged without authority is driven, to server sending permission warning information, with prompting
The driver is without the authority for driving the Current vehicle.
When judging current driver without authority is driven, authority warning information is sent to server immediately, with
Just staff checks.
Based on above-described embodiment, another embodiment of the present invention further provides another in-car based on computer vision
The method of safety monitoring, referring to the drawings shown in 6, this method mainly includes：
401st, the in-car image shot in preset time period is obtained.
402nd, by carrying out human body skin detection to the in-car image, the limb of different personnel in the in-car image is obtained
Body image.
403rd, the limbs image of different personnel carries out optical flow computation in the in-car image adjacent to shooting time, it is determined that described
Movement tendency of the limbs of different personnel in the preset time period.
404th, according to the movement tendency of different personnel in the in-car image, judge whether that limbs conflict occurs.
Step 401-404 implementation is identical with the step 101-104 of above-described embodiment, will not be repeated here.
405th, by carrying out Face datection to the in-car image obtained in real time, the real-time face-image of driver is obtained.
In-car image is obtained in real time, and the image in region corresponding with position of driver is obtained from in-car image, and to driving
The person's of sailing image carries out Face datection, obtains the real-time face-image of driver.
406th, human eye detection is carried out to the face-image, obtains real-time pupil image.
The eye circumference characteristic point and pupil that Ha Er (haar) feature extraction driver's eyes are passed through to the face-image of driver
Position.Eye areas is divided into by nine regions according to eye circumference characteristic point, wherein, the region at center is that driver watches attentively when driving
Position during road surface ahead where pupil, the region of center left and the region of central right are that driver watches left and right attentively when driving
Position during rearview mirror where pupil, other six regions are the driver regions that pupil should not occur when driving.So
, can be with the direction of visual lines of real-time judge driver according to region of the pupil in eyes, and then judge driver whether for a long time
It is not concerned with road surface ahead.
407th, according to the pupil image and the acquisition pupil image at the time of, determine that pupil is in non-watching area
Duration.
The non-watching area includes watching outside rear-view mirror region and dangerous watching area attentively, and the dangerous watching area is car
Region when in driving process except driver's fixating straight ahead residing for pupil and region when watching rearview mirror attentively residing for pupil
Region in addition.
The region of the center left of eye areas and the region of central right are to watch outside rear-view mirror region attentively, except center
Other six regions in the region in region, the region of center left and central right are dangerous watching areas.Scheme when according to in-car
When pupil as detecting driver is in non-watching area, at the time of recording pupil image and obtain pupil image, and really
Pupil is determined in non-watching area residence time；Judge whether driver is dangerous in non-watching area residence time according to pupil
Drive.
408th, judge whether pupil is in the outside rear-view mirror watching area, pupil is in the outside rear-view mirror note if judging
Viewed area, then perform step 409；If judging, pupil is not located at the outside rear-view mirror watching area, performs step 410.
In driving procedure, it is normal driving state that driver pays close attention to outside rear-view mirror within the of short duration time, and driver removes
Watch road surface ahead attentively and watch outside rear-view mirror attentively, paid close attention within the of short duration time and belong to dangerous driving elsewhere.So according to
The time that driver watches outside rear-view mirror attentively judges that driver is in dangerous driving state and watched attentively except road surface ahead and outside rear-view mirror
The time elsewhere judges that driver is in the Rule of judgment difference of dangerous driving state in addition.
If the 409th, the pupil is in the duration of the outside rear-view mirror watching area and watches duration attentively more than outside rear-view mirror
Threshold value, then judge that driver is in dangerous driving state.
In driving procedure, driver is normal driving state in of short duration interior concern rearview mirror, but if driver is closed
Note rearview mirror direction is long just to belong to dangerous driving.Therefore, when the pupil of driver is in continuing for outside rear-view mirror watching area
When duration watches duration threshold value attentively more than outside rear-view mirror set in advance, then judge that driver is in dangerous driving state.Work as judgement
When driver is in dangerous driving state, dangerous driving warning message is sent；Dangerous driving warning message include audible alert or
Vibrations warning.
410th, judge whether pupil is in the dangerous watching area, if judging, pupil is in the dangerous watching area,
Then perform step 411.
If the 411, the pupil is in the duration of the dangerous watching area and watches duration threshold value attentively more than danger, sentence
Disconnected driver is in dangerous driving state.
In driving procedure, driver pays close attention to it except watching road surface ahead attentively and watching outside rear-view mirror attentively within the of short duration time
He belongs to dangerous driving in place.Therefore, the duration that dangerous watching area is in when the pupil of driver exceedes and in advance set
When duration threshold value is watched in fixed danger attentively, then judge that driver is in dangerous driving state.When judging that driver is in dangerous driving
During state, dangerous driving warning message is sent；Dangerous driving warning message includes audible alert or vibrations warning.
Further, one kind is additionally provided as the realization to method shown in above-mentioned Fig. 3-Fig. 6, another embodiment of the present invention
The device of in-car safety monitoring based on computer vision.The device embodiment is corresponding with preceding method embodiment, for ease of readding
Read, present apparatus embodiment is no longer repeated the detail content in preceding method embodiment one by one, it should be understood that this implementation
Device in example can correspond to the full content realized in preceding method embodiment.
Referring to the drawings shown in 7, the device of the in-car safety monitoring based on computer vision mainly includes：
Acquiring unit 51, for obtaining the in-car image shot in preset time period.
Skin detection unit 52, for carrying out human body skin detection to the in-car image, obtain in the in-car image
The limbs image of different personnel.
Trend determining unit 53, the limbs image for different personnel in the in-car image adjacent to shooting time carry out light
Stream calculation, determine movement tendency of the limbs of the different personnel in the preset time period.
Conflict judging unit 54, for the movement tendency according to different personnel in the in-car image, judges whether to occur
Limbs conflict.
Optionally, with reference to the accompanying drawings shown in 8, the skin detection unit 52 includes：
Face detection module 521, for by carrying out Face datection to the in-car image, obtaining the face of different personnel
Image.
Skin sampling module 522, for carrying out colour of skin sampling to the face-image of the different personnel, obtain different personnel
Colour of skin sample.
Flesh tone determination module 523, for determining that the colour of skin of different personnel corresponding to the colour of skin sample of the different personnel takes
It is worth scope.
Limbs determining module 524, for the colour of skin span for determining to belong to the different personnel in the in-car image
Image be corresponding different personnel limbs image.
Optionally, the Trend judgement unit 53 is additionally operable to work as according to determining the movement tendency of the different personnel not
When having the trend of contact in the preset time period with the limbs of personnel or contacted, judge limbs conflict occurs.
Optionally, with reference to the accompanying drawings shown in 8, described device also includes：
Collision alert unit 61, for when judging to occur limbs conflict, sending limbs conflict warning message；
Transmitting element 62, it is additionally operable to the limbs conflict warning message being uploaded to server.
Optionally, with reference to the accompanying drawings shown in 8, described device also includes：
Image storage unit 63, for be locally stored the preset time period in-car image and with the preset time
The in-car image of the isometric front and rear time adjacent segments of section.
The transmitting element 62 is additionally operable to by the in-car image of the preset time period and with the preset time period etc.
The in-car image of long front and rear time adjacent segments is uploaded to the server.
Optionally, with reference to the accompanying drawings shown in 9, described device also includes：
First face detection unit 71, for by carrying out Face datection to the in-car image, obtaining the face of driver
Portion's image.
Authority judging unit 72, for the face-image in the face-image of the driver and identity information storehouse to be carried out
Contrast, judges whether the driver has the driving authority of Current vehicle, and the identity information storehouse, which is used to store, has difference
The corresponding relation of the face-image of the driver of vehicle drive authority and the vehicles identifications of different vehicle.
Authority prewarning unit 73, it is for judging the driver without authority is driven, it is pre- to server sending permission
Alert information, to prompt authority of the driver without the Current vehicle is driven.
Optionally, with reference to the accompanying drawings shown in 10, described device also includes：
Second Face datection unit 81, for by carrying out Face datection to the in-car image obtained in real time, being driven
The real-time face-image of member.
Human eye detection unit 82, for carrying out human eye detection to the face-image, obtain real-time pupil image.
Duration determining unit 83, for according to the pupil image and obtain the pupil image at the time of, determine pupil
It is in the duration of non-watching area.The non-watching area includes watching outside rear-view mirror region and dangerous watching area, institute attentively
State dangerous watching area be in vehicle driving process except driver's fixating straight ahead when pupil residing for region and watch backsight attentively
The region beyond region during mirror residing for pupil.
Danger judgement unit 84, for judging whether driver is in dangerous driving state according to the duration.
Optionally, with reference to the accompanying drawings shown in 10, the danger judgement unit 84 includes：
First danger judgement module 841 and the second danger judgement module 842,
The first danger judgement module 841, for being in continuing for the outside rear-view mirror watching area when the pupil
When duration watches duration threshold value attentively more than outside rear-view mirror, judge that driver is in dangerous driving state；
The second danger judgement module 842, for being in the duration of the dangerous watching area when the pupil
When watching duration threshold value attentively more than danger, judge that driver is in dangerous driving state.
The device of in-car safety monitoring provided by the invention based on computer vision, in-car is determined without relying upon artificial
Conflicting with the presence or absence of personnel, the limbs image of different personnel carries out optical flow computation in in-car image only adjacent to shooting time,
Obtain movement tendency of the limbs of in-car different personnel in preset time period, further according to these occupants movement tendency just
In real time and it can automatically determine in-car whether there occurs limbs conflict；And the limbs image of different personnel is only by right
The in-car image progress human body skin detection can of shooting in preset time period obtains automatically.Utilize the method and dress of the present invention
Full-automatic monitoring in real time can be carried out to in-car security context by putting, and substantially increase the ageing and efficiency of in-car safety detection.
In addition, the embodiment of the present invention can also obtain the face-image and pupillogram of driver according to in-car image
Picture, judge whether driver has the driving authority for driving vehicle using face-image；Judge that driver is using pupil image
No dangerous driving.
In-car safety monitoring terminal based on computer vision includes processor and storage medium, above-mentioned acquiring unit, skin
Skin detection unit, trend determining unit and conflict judging unit etc. are stored in storage medium as program unit, by handling
Device performs the said procedure unit that is stored in storage medium to realize corresponding function.
Kernel is included in processor, is gone in memory to transfer corresponding program unit by kernel.Kernel can set one
Or more, solve the problems, such as that the monitoring efficiency of in-car conflict monitoring is low by adjusting kernel parameter.
Memory may include computer-readable medium in volatile memory, random access memory (RAM) and/
Or the form such as Nonvolatile memory, such as read-only storage (ROM) or flash memory (flashRAM), memory includes at least one storage
Chip.
The embodiments of the invention provide a kind of storage medium, is stored thereon with a plurality of instruction, and the instruction is loaded by processor
And the method for performing in-car safety monitoring of the kind based on computer vision.
The embodiments of the invention provide a kind of processor, the processor is used for operation program, wherein, described program operation
The method that the in-car safety monitoring based on computer vision is planted described in Shi Zhihang.
The embodiments of the invention provide a kind of in-car safety monitoring terminal based on computer vision, the terminal includes depositing
Storage media, processor and storage are realized on a storage medium and the instruction that can run on a processor, during computing device instruction
Following steps：
1st, the in-car image shot in preset time period is obtained.
2nd, by carrying out human body skin detection to the in-car image, the limbs of different personnel in the in-car image are obtained
Image.
3rd, in the in-car image adjacent to shooting time different personnel limbs image carry out optical flow computation, it is determined that it is described not
With movement tendency of the limbs in the preset time period of personnel.
4th, according to the movement tendency of different personnel in the in-car image, judge whether that limbs conflict occurs.
The invention discloses：
A1, a kind of method of the in-car safety monitoring based on computer vision, methods described include：
Obtain the in-car image shot in preset time period；
By carrying out human body skin detection to the in-car image, the limbs figure of different personnel in the in-car image is obtained
Picture；
The limbs image of different personnel carries out optical flow computation in the in-car image adjacent to shooting time, determines the difference
Movement tendency of the limbs of personnel in the preset time period；
According to the movement tendency of different personnel in the in-car image, judge whether that limbs conflict occurs.
A2, the method as described in A1, by carrying out human body skin detection to the in-car image, obtain the in-car image
The limbs image of middle different personnel includes：
By carrying out Face datection to the in-car image, the face-image of different personnel is obtained；
Colour of skin sampling is carried out to the face-image of the different personnel, obtains the colour of skin sample of different personnel；
It is determined that colour of skin span corresponding to each colour of skin sample；
Determine the limbs image that the image for belonging to the colour of skin span in the in-car image is corresponding personnel.
A3, the method as described in A1 or A2, according to the movement tendency of different personnel in the in-car image, judge whether to send out
Raw limbs conflict includes：
If determine the limbs of the different personnel in the preset time period according to the movement tendency of the different personnel
There is the trend persistently to be contacted or have continuous contact action, then judge limbs conflict occurs.
A4, the method as described in A1 or A2, in the movement tendency according to different personnel in the in-car image, judge whether
After generation limbs conflict, methods described also includes：
If judging limbs conflict occurs, limbs conflict warning message is produced, and by the limbs conflict warning message
It is transmitted to server.
A5, the method as described in A4, if judging limbs conflict occurs, methods described also includes：
The time adjacent segments before and after the in-car image of the preset time period and the preset time period is locally stored
In-car image, the duration of the time adjacent segments are pre-set；
And/or the car by time adjacent segments before and after the in-car image of the preset time period and the preset time period
Interior image is uploaded to the server.
A6, the method as described in A1, methods described also include：
By carrying out Face datection to the in-car image, the face-image of driver is obtained；
Face-image in the face-image of the driver and identity information storehouse is contrasted, judges the driver
Whether there is the driving authority of Current vehicle, the identity information storehouse is used to store the driver for driving authority with different vehicle
Face-image and different vehicle vehicles identifications corresponding relation；
If the driver is judged without authority is driven, to server sending permission warning information, with described in prompting
Driver is without the authority for driving the Current vehicle.
A7, the method as described in A1, methods described also include：
By carrying out Face datection to the in-car image obtained in real time, the real-time face-image of driver is obtained；
Human eye detection is carried out to the face-image, obtains real-time pupil image；
At the time of according to the pupil image and obtaining the pupil image, determine that pupil is in continuing for non-watching area
Duration, the non-watching area include watching outside rear-view mirror region and dangerous watching area attentively, and the dangerous watching area is vehicle
Region when in driving process except driver's fixating straight ahead residing for pupil and region when watching rearview mirror attentively residing for pupil with
Outer region；
Judge whether driver is in dangerous driving state according to the duration.
A8, the method as described in A7, judge whether driver includes in dangerous driving state according to the duration：
If the duration that the pupil is in the outside rear-view mirror watching area watches duration threshold value attentively more than outside rear-view mirror,
Then judge that driver is in dangerous driving state；
If the duration that the pupil is in the dangerous watching area watches duration threshold value attentively more than danger, judgement is driven
The person of sailing is in dangerous driving state.
B9, a kind of device of the in-car safety monitoring based on computer vision, described device include：
Acquiring unit, for obtaining the in-car image shot in preset time period；
Skin detection unit, for carrying out human body skin detection to the in-car image, obtain in the in-car image not
With the limbs image of personnel；
Trend determining unit, the limbs image for different personnel in the in-car image adjacent to shooting time carry out light stream
Calculate, determine movement tendency of the limbs of the different personnel in the preset time period；
Conflict judging unit, for the movement tendency according to different personnel in the in-car image, judges whether that limb occurs
Body conflict.
B10, the device as described in B9, the skin detection unit include：
Face detection module, for by carrying out Face datection to the in-car image, obtaining the face figure of different personnel
Picture；
Skin sampling module, for carrying out colour of skin sampling to the face-image of the different personnel, obtain different personnel's
Colour of skin sample；
Flesh tone determination module, for determining the colour of skin value model of different personnel corresponding to the colour of skin sample of the different personnel
Enclose；
Limbs determining module, the figure of the colour of skin span for determining to belong in the in-car image the different personnel
As the limbs image for corresponding different personnel.
B11, the device as described in B9, the Trend judgement unit are additionally operable to when the movement tendency according to the different personnel
Determining the limbs of the different personnel has the trend persistently to be contacted in the preset time period or has continuous contact action
When, judge limbs conflict occurs.
B12, the device as described in B9 or B10, described device also include：
Collision alert unit, for when judging to occur limbs conflict, producing limbs conflict warning message；
Transmitting element, for the limbs conflict warning message to be uploaded into server.
B13, the device as described in B12, described device also include：
Image storage unit, for the in-car image of the preset time period and the preset time period to be locally stored
The in-car image of front and rear time adjacent segments, the duration of the time adjacent segments are pre-set；
The transmitting element is additionally operable to phase before and after the in-car image of the preset time period and the preset time period
The in-car image of adjacent period is uploaded to the server.
B14, the device as described in B9, described device also include：
First face detection unit, for by carrying out Face datection to the in-car image, obtaining the face of driver
Image；
Authority judging unit, for the face-image in the face-image of the driver and identity information storehouse to be carried out pair
Than judging whether the driver has the driving authority of Current vehicle, the identity information storehouse, which is used to store, has different cars
Drive authority driver face-image and different vehicle vehicles identifications corresponding relation；
Authority prewarning unit, it is for judging the driver without authority is driven, to server sending permission early warning
Information, to prompt authority of the driver without the Current vehicle is driven.
B15, the device as described in B9, described device also include：
Second Face datection unit, for by carrying out Face datection to the in-car image obtained in real time, obtaining driver
Real-time face-image；
Human eye detection unit, for carrying out human eye detection to the face-image, obtain real-time pupil image；
Duration determining unit, for according to the pupil image and obtain the pupil image at the time of, determine at pupil
In the duration of non-watching area, the non-watching area includes watching outside rear-view mirror region and dangerous watching area attentively, described
Dangerous watching area be in vehicle driving process except driver's fixating straight ahead when pupil residing for region and watch rearview mirror attentively
When pupil residing for region beyond region；
Danger judgement unit, for judging whether driver is in dangerous driving state according to the duration.
B16, the device as described in B15, the danger judgement unit include：
First danger judgement module and the second danger judgement module,
The first danger judgement module, for being in the duration of the outside rear-view mirror watching area when the pupil
When watching duration threshold value attentively more than outside rear-view mirror, judge that driver is in dangerous driving state；
The second danger judgement module, the duration for being in the dangerous watching area when the pupil are more than
When duration threshold value is watched in danger attentively, judge that driver is in dangerous driving state.
C17, a kind of storage medium, the storage medium are stored with a plurality of instruction, and the instruction is applied to be added by processor
The method for carrying and performing the in-car safety monitoring based on computer vision as described in A1 any one into A8.
D18, a kind of in-car safety monitoring terminal based on computer vision：
The terminal includes storage medium and processor；
The processor, it is adapted for carrying out each instruction；
The storage medium, suitable for storing a plurality of instruction；
It is described instruction be suitable to loaded as the processor and perform as described in A1 any one into A8 based on computer
The method of the in-car safety monitoring of vision.
It should be understood by those skilled in the art that, embodiments herein can be provided as method, system or computer program
Product.Therefore, the application can use the reality in terms of complete hardware embodiment, complete software embodiment or combination software and hardware
Apply the form of example.Moreover, the application can use the computer for wherein including computer usable program code in one or more
The computer program production that usable storage medium is implemented on (including but is not limited to magnetic disk storage, CD-ROM, optical memory etc.)
The form of product.
The application is with reference to the flow according to the method for the embodiment of the present application, equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that can be by every first-class in computer program instructions implementation process figure and/or block diagram
Journey and/or the flow in square frame and flow chart and/or block diagram and/or the combination of square frame.These computer programs can be provided
The processors of all-purpose computer, special-purpose computer, Embedded Processor or other programmable data processing devices is instructed to produce
A raw machine so that produced by the instruction of computer or the computing device of other programmable data processing devices for real
The device for the function of being specified in present one flow of flow chart or one square frame of multiple flows and/or block diagram or multiple square frames.
These computer program instructions, which may be alternatively stored in, can guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works so that the instruction being stored in the computer-readable memory, which produces, to be included referring to
Make the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one square frame of block diagram or
The function of being specified in multiple square frames.
These computer program instructions can be also loaded into computer or other programmable data processing devices so that counted
Series of operation steps is performed on calculation machine or other programmable devices to produce computer implemented processing, so as in computer or
The instruction performed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in individual square frame or multiple square frames.
In a typical configuration, computing device includes one or more processors (CPU), input/output interface, net
Network interface and internal memory.
Memory may include computer-readable medium in volatile memory, random access memory (RAM) and/
Or the form such as Nonvolatile memory, such as read-only storage (ROM) or flash memory (flashRAM).Memory is computer-readable medium
Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer-readable instruction, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase transition internal memory (PRAM), static RAM (SRAM), moved
State random access memory (DRAM), other kinds of random access memory (RAM), read-only storage (ROM), electric erasable
Programmable read only memory (EEPROM), fast flash memory bank or other memory techniques, read-only optical disc read-only storage (CD-ROM),
Digital versatile disc (DVD) or other optical storages, magnetic cassette tape, the storage of tape magnetic rigid disk or other magnetic storage apparatus
Or any other non-transmission medium, the information that can be accessed by a computing device available for storage.Define, calculate according to herein
Machine computer-readable recording medium does not include temporary computer readable media (transitory media), such as data-signal and carrier wave of modulation.
It should also be noted that, term " comprising ", "comprising" or its any other variant are intended to nonexcludability
Comprising so that process, method, commodity or equipment including a series of elements not only include those key elements, but also wrapping
Include the other element being not expressly set out, or also include for this process, method, commodity or equipment intrinsic want
Element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that including key element
Other identical element in process, method, commodity or equipment also be present.
It will be understood by those skilled in the art that embodiments herein can be provided as method, system or computer program product.
Therefore, the application can be using the embodiment in terms of complete hardware embodiment, complete software embodiment or combination software and hardware
Form.Deposited moreover, the application can use to can use in one or more computers for wherein including computer usable program code
The shape for the computer program product that storage media is implemented on (including but is not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
Embodiments herein is these are only, is not limited to the application.To those skilled in the art,
The application can have various modifications and variations.All any modifications made within spirit herein and principle, equivalent substitution,
Improve etc., it should be included within the scope of claims hereof.
The invention also discloses：
E1, a kind of method of the in-car safety monitoring based on computer vision, wherein, this method comprises the following steps：
In-car picture is obtained by camera；
The in-car picture got is analyzed；Wherein analysis includes driver's driving privilege analysis, driver
Driving behavior analysis, passenger's passenger choice behavior analysis；
Judged by analysis in-car whether safe.
E2, the method as described in E1, wherein, driver's privilege analysis identifies including Face datection：
Obtain in-car picture；
Major-minor is driven and carries out Face datection, obtains human face region；
Driver is identified, judges whether to possess driving authority.
E3, the method as described in E1, wherein, driver's driving behavior analysis includes line-of-sight detection：
Positioned by human eye to obtain driver's binocular position according to in-car picture；
Detection pilot's line of vision direction in real time；
Judge whether driver is not concerned with road surface ahead situation for a long time；
If it is not, the information that gives a warning.
E4, the method as described in E1, wherein, passenger's passenger choice behavior analysis includes：
Skin detection：By carrying out color sample to the human face region obtained by Face datection, and it is based on the existing colour of skin
Data analysis, obtain scope of the respective skin color of each occupant in the value of YCrCb color spaces.
Limbs conflict detects：It is in-car every because the camera fixed in advance is positioned at the one known position in in-car front
Individual corresponds to the different regions of each two field picture of camera, by skin detection, can obtain the naked of each occupant
The skin of dew gets rid of the location of known face, it becomes possible to substantially obtain everyone in the position of each two field picture
The position of trunk.Light stream is calculated to front and rear two field pictures.When the limbs that major-minor drives contact with each other, both are judged by algorithm
Whether limbs conflict occurring, for conflict, alert be present, and recording the in-car picture of surrounding time section,
Dispute processing is carried out for third party.
F5, a kind of device of the in-car safety monitoring based on computer vision, wherein, the device includes：
Acquisition module, for obtaining in-car picture by camera；
Analysis module, for analyzing the in-car picture got；Wherein analysis includes driver's driving power
Limit analysis, driver's driving behavior analysis, passenger's passenger choice behavior analysis；
Judge module, it is in-car whether safe for being judged by analysis.
F6, the device as described in F5, wherein, driver's privilege analysis identifies including Face datection：
Obtain in-car picture；
Major-minor is driven and carries out Face datection, obtains human face region；
Driver is identified, judges whether to possess driving authority.
F7, the device as described in F5, wherein, driver's driving behavior analysis includes line-of-sight detection：
Positioned by human eye to obtain driver's binocular position according to in-car picture；
Detection pilot's line of vision direction in real time；
Judge whether driver is not concerned with road surface ahead situation for a long time；
If it is not, the information that gives a warning.
F8, the device as described in F5, wherein, passenger's passenger choice behavior analysis includes：
Skin detection：By carrying out color sample to the human face region obtained by Face datection, and it is based on the existing colour of skin
Data analysis, obtain scope of the respective skin color of each occupant in the value of YCrCb color spaces.
Limbs conflict detects：It is in-car every because the camera fixed in advance is positioned at the one known position in in-car front
Individual corresponds to the different regions of each two field picture of camera, by skin detection, can obtain the naked of each occupant
The skin of dew gets rid of the location of known face, it becomes possible to substantially obtain everyone in the position of each two field picture
The position of trunk.Light stream is calculated to front and rear two field pictures.When the limbs that major-minor drives contact with each other, both are judged by algorithm
Whether limbs conflict occurring, for conflict, alert be present, and recording the in-car picture of surrounding time section,
Dispute processing is carried out for third party.
Claims (10)
- A kind of 1. method of the in-car safety monitoring based on computer vision, it is characterised in that this method comprises the following steps：In-car picture is obtained by camera；The in-car picture got is analyzed；Wherein analysis includes driver's driving privilege analysis, and driver drives Behavioural analysis, passenger's passenger choice behavior analysis；Judged by analysis in-car whether safe.
- 2. a kind of device of the in-car safety monitoring based on computer vision, it is characterised in that the device includes：Acquisition module, for obtaining in-car picture by camera；Analysis module, for analyzing the in-car picture got；Wherein analysis includes driver's driving authority point Analysis, driver's driving behavior analysis, passenger's passenger choice behavior analysis；Judge module, it is in-car whether safe for being judged by analysis.
- A kind of 3. method of the in-car safety monitoring based on computer vision, it is characterised in that methods described includes：Obtain the in-car image shot in preset time period；By carrying out human body skin detection to the in-car image, the limbs image of different personnel in the in-car image is obtained；The limbs image of different personnel carries out optical flow computation in the in-car image adjacent to shooting time, determines the different personnel Movement tendency of the limbs in the preset time period；According to the movement tendency of different personnel in the in-car image, judge whether that limbs conflict occurs.
- 4. according to the method for claim 3, it is characterised in that by carrying out human body skin detection to the in-car image, Obtaining the limbs image of different personnel in the in-car image includes：By carrying out Face datection to the in-car image, the face-image of different personnel is obtained；Colour of skin sampling is carried out to the face-image of the different personnel, obtains the colour of skin sample of different personnel；It is determined that colour of skin span corresponding to each colour of skin sample；Determine the limbs image that the image for belonging to the colour of skin span in the in-car image is corresponding personnel.
- 5. the method according to claim 3 or 4, it is characterised in that in the fortune according to different personnel in the in-car image Dynamic trend, judge whether after limbs conflict occurs, methods described also includes：If judging limbs conflict occurs, limbs conflict warning message is produced, and the limbs conflict warning message is uploaded to Server.
- 6. according to the method for claim 3, it is characterised in that methods described also includes：By carrying out Face datection to the in-car image, the face-image of driver is obtained；Face-image in the face-image of the driver and identity information storehouse is contrasted, whether judges the driver Driving authority with Current vehicle, the identity information storehouse are used for the face for storing the driver that authority is driven with different vehicle The corresponding relation of portion's image and the vehicles identifications of different vehicle；If the driver is judged without authority is driven, to server sending permission warning information, to prompt the driving Member is without the authority for driving the Current vehicle.
- 7. according to the method for claim 3, it is characterised in that methods described also includes：By carrying out Face datection to the in-car image obtained in real time, the real-time face-image of driver is obtained；Human eye detection is carried out to the face-image, obtains real-time pupil image；According to the pupil image and obtain the pupil image at the time of, determine pupil be in non-watching area it is lasting when Long, the non-watching area includes watching outside rear-view mirror region and dangerous watching area attentively, and the dangerous watching area is vehicle row In addition to region during making when driver's fixating straight ahead residing for pupil and region when watching rearview mirror attentively residing for pupil Region；Judge whether driver is in dangerous driving state according to the duration.
- 8. a kind of device of the in-car safety monitoring based on computer vision, it is characterised in that described device includes：Acquiring unit, for obtaining the in-car image shot in preset time period；Skin detection unit, for carrying out human body skin detection to the in-car image, obtain different people in the in-car image The limbs image of member；Trend determining unit, based on the limbs image progress light stream of different personnel in the in-car image adjacent to shooting time Calculate, determine movement tendency of the limbs of the different personnel in the preset time period；Conflict judging unit, for the movement tendency according to different personnel in the in-car image, judges whether that limbs punching occurs It is prominent.
- 9. a kind of storage medium, it is characterised in that the storage medium is stored with a plurality of instruction, and the instruction is applied to by handling Device loads and performs the side of the in-car safety monitoring based on computer vision as described in any one in claim 1,3 to 7 Method.
- A kind of 10. in-car safety monitoring terminal based on computer vision, it is characterised in that：The terminal includes storage medium and processor；The processor, it is adapted for carrying out each instruction；The storage medium, suitable for storing a plurality of instruction；It is described instruction be suitable to loaded as the processor and perform as described in any one in claim 1,3 to 7 based on meter The method of the in-car safety monitoring of calculation machine vision.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710063012.2A CN107341438A (en) | 2017-01-24 | 2017-01-24 | The method and apparatus of in-car safety monitoring based on computer vision |
| CN2017100630122 | 2017-01-24 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107729870A true CN107729870A (en) | 2018-02-23 |
Family
ID=60222644
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710063012.2A Pending CN107341438A (en) | 2017-01-24 | 2017-01-24 | The method and apparatus of in-car safety monitoring based on computer vision |
| CN201711059359.6A Pending CN107729870A (en) | 2017-01-24 | 2017-11-01 | The method and apparatus of in-car safety monitoring based on computer vision |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710063012.2A Pending CN107341438A (en) | 2017-01-24 | 2017-01-24 | The method and apparatus of in-car safety monitoring based on computer vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN107341438A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108986400A (en) * | 2018-09-03 | 2018-12-11 | 深圳市尼欧科技有限公司 | A kind of third party based on image procossing, which multiplies, drives safety automatic-alarming method |
| CN109291794A (en) * | 2018-09-28 | 2019-02-01 | 上汽通用五菱汽车股份有限公司 | Driver status monitoring method, automobile and storage medium |
| CN109501807A (en) * | 2018-08-15 | 2019-03-22 | 初速度（苏州）科技有限公司 | Automatic Pilot pays attention to force detection system and method |
| CN110040096A (en) * | 2019-05-09 | 2019-07-23 | 大陆投资(中国)有限公司 | Aggression detection system and method for vehicle cab |
| CN110288796A (en) * | 2019-06-21 | 2019-09-27 | 浙江大华技术股份有限公司 | Vehicle monitoring method and device, storage medium, electronic device |
| CN110390285A (en) * | 2019-07-16 | 2019-10-29 | 广州小鹏汽车科技有限公司 | System for distraction of driver detection method, system and vehicle |
| CN110414406A (en) * | 2019-07-23 | 2019-11-05 | 广汽蔚来新能源汽车科技有限公司 | Interior object monitoring and managing method, device, system, car-mounted terminal and storage medium |
| CN111008586A (en) * | 2019-11-29 | 2020-04-14 | 上海能塔智能科技有限公司 | Data processing method, device, equipment and storage medium for passenger car conflict detection |
| CN111209814A (en) * | 2019-12-27 | 2020-05-29 | 广东德融汇科技有限公司 | Face recognition campus area early warning method and system for K12 education stage |
| CN111860111A (en) * | 2020-06-01 | 2020-10-30 | 北京嘀嘀无限科技发展有限公司 | Safety monitoring method and device in vehicle journey and storage medium |
| CN112208475A (en) * | 2019-07-09 | 2021-01-12 | 奥迪股份公司 | Safety protection system for vehicle occupants, vehicle and corresponding method and medium |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108449571A (en) * | 2018-01-30 | 2018-08-24 | 深圳市元征软件开发有限公司 | A kind of car monitoring method and equipment |
| CN111064841B (en) * | 2018-10-16 | 2020-11-10 | 北京嘀嘀无限科技发展有限公司 | Message sending method and device and server |
| CN111079474A (en) | 2018-10-19 | 2020-04-28 | 上海商汤智能科技有限公司 | Passenger state analysis method and device, vehicle, electronic device, and storage medium |
| CN109471439A (en) * | 2018-11-23 | 2019-03-15 | 广州小鹏汽车科技有限公司 | Control method for vehicle, device, equipment, storage medium and automobile |
| CN109859270A (en) * | 2018-11-28 | 2019-06-07 | 浙江合众新能源汽车有限公司 | A kind of human eye three-dimensional coordinate localization method and separate type binocular camera shooting device |
| CN111428545A (en) * | 2019-01-10 | 2020-07-17 | 北京嘀嘀无限科技发展有限公司 | Behavior judgment method and device and electronic equipment |
| CN111798588A (en) * | 2019-04-09 | 2020-10-20 | 安克创新科技股份有限公司 | Automobile data recorder, mobile terminal, and in-vehicle monitoring method and device |
| CN110154757A (en) * | 2019-05-30 | 2019-08-23 | 电子科技大学 | The multi-faceted safe driving support method of bus |
| JP2022517152A (en) | 2019-12-16 | 2022-03-07 | ベイジン ディディ インフィニティ テクノロジー アンド ディベロップメント カンパニー リミティッド | Systems and methods for distinguishing between the driver and the occupant in the image captured in the vehicle |
| CN111071015A (en) * | 2019-12-26 | 2020-04-28 | 的卢技术有限公司 | Visual synchronization vehicle window control method and system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1495658A (en) * | 2002-06-30 | 2004-05-12 | 贺贵明 | Driver's face image identification and alarm device and method |
| CN101332362A (en) * | 2008-08-05 | 2008-12-31 | 北京中星微电子有限公司 | Interactive delight system based on human posture recognition and implement method thereof |
| CN101350064A (en) * | 2008-08-29 | 2009-01-21 | 北京中星微电子有限公司 | Method and apparatus for estimating two-dimension human body guise |
| CN101388114A (en) * | 2008-09-03 | 2009-03-18 | 北京中星微电子有限公司 | Method and system for estimating human body attitudes |
| CN104715235A (en) * | 2015-02-10 | 2015-06-17 | 锦州铁道勘察设计院有限公司 | Train driver state recognizing and alarming method |
| CN204904499U (en) * | 2015-08-26 | 2015-12-23 | 中电科二十二所（青岛）天博信息科技公司 | Subway train alarm system |
| CN105292052A (en) * | 2015-11-12 | 2016-02-03 | 福州大学 | Automatic monitoring and alarming system for vehicle traffic safety |
-
2017
- 2017-01-24 CN CN201710063012.2A patent/CN107341438A/en active Pending
- 2017-11-01 CN CN201711059359.6A patent/CN107729870A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1495658A (en) * | 2002-06-30 | 2004-05-12 | 贺贵明 | Driver's face image identification and alarm device and method |
| CN101332362A (en) * | 2008-08-05 | 2008-12-31 | 北京中星微电子有限公司 | Interactive delight system based on human posture recognition and implement method thereof |
| CN101350064A (en) * | 2008-08-29 | 2009-01-21 | 北京中星微电子有限公司 | Method and apparatus for estimating two-dimension human body guise |
| CN101388114A (en) * | 2008-09-03 | 2009-03-18 | 北京中星微电子有限公司 | Method and system for estimating human body attitudes |
| CN104715235A (en) * | 2015-02-10 | 2015-06-17 | 锦州铁道勘察设计院有限公司 | Train driver state recognizing and alarming method |
| CN204904499U (en) * | 2015-08-26 | 2015-12-23 | 中电科二十二所（青岛）天博信息科技公司 | Subway train alarm system |
| CN105292052A (en) * | 2015-11-12 | 2016-02-03 | 福州大学 | Automatic monitoring and alarming system for vehicle traffic safety |
Non-Patent Citations (3)
| Title |
|---|
| 叶芳芳: "《监控视频中的异常行为检测研究》", 《中国博士学位论文全文数据库 信息科技辑》 * |
| 邸巍: "《基于视觉的全天候驾驶员疲劳与精神分散状态检测方法研究》", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 马燚博: "《基于机器视觉的校车异常监测系统》", 《科技资讯》 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109501807A (en) * | 2018-08-15 | 2019-03-22 | 初速度（苏州）科技有限公司 | Automatic Pilot pays attention to force detection system and method |
| CN108986400A (en) * | 2018-09-03 | 2018-12-11 | 深圳市尼欧科技有限公司 | A kind of third party based on image procossing, which multiplies, drives safety automatic-alarming method |
| CN109291794A (en) * | 2018-09-28 | 2019-02-01 | 上汽通用五菱汽车股份有限公司 | Driver status monitoring method, automobile and storage medium |
| CN110040096A (en) * | 2019-05-09 | 2019-07-23 | 大陆投资(中国)有限公司 | Aggression detection system and method for vehicle cab |
| CN110288796A (en) * | 2019-06-21 | 2019-09-27 | 浙江大华技术股份有限公司 | Vehicle monitoring method and device, storage medium, electronic device |
| CN112208475A (en) * | 2019-07-09 | 2021-01-12 | 奥迪股份公司 | Safety protection system for vehicle occupants, vehicle and corresponding method and medium |
| CN112208475B (en) * | 2019-07-09 | 2023-02-03 | 奥迪股份公司 | Safety protection system for vehicle occupants, vehicle and corresponding method and medium |
| CN110390285A (en) * | 2019-07-16 | 2019-10-29 | 广州小鹏汽车科技有限公司 | System for distraction of driver detection method, system and vehicle |
| CN110414406A (en) * | 2019-07-23 | 2019-11-05 | 广汽蔚来新能源汽车科技有限公司 | Interior object monitoring and managing method, device, system, car-mounted terminal and storage medium |
| CN111008586A (en) * | 2019-11-29 | 2020-04-14 | 上海能塔智能科技有限公司 | Data processing method, device, equipment and storage medium for passenger car conflict detection |
| CN111209814A (en) * | 2019-12-27 | 2020-05-29 | 广东德融汇科技有限公司 | Face recognition campus area early warning method and system for K12 education stage |
| CN111209814B (en) * | 2019-12-27 | 2023-05-19 | 广东德融汇科技有限公司 | Face recognition campus area early warning method and system for K12 education stage |
| CN111860111A (en) * | 2020-06-01 | 2020-10-30 | 北京嘀嘀无限科技发展有限公司 | Safety monitoring method and device in vehicle journey and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107341438A (en) | 2017-11-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107729870A (en) | The method and apparatus of in-car safety monitoring based on computer vision | |
| CN107967806B (en) | Vehicle fake-license detection method, device, readable storage medium storing program for executing and electronic equipment | |
| CN106448045B (en) | Safe early warning method, system and mobile unit in driving conditions | |
| Brombacher et al. | Driving event detection and driving style classification using artificial neural networks | |
| CN112781887B (en) | Method, device and system for testing vehicle performance | |
| CN109872510A (en) | Interior Smoke Detection and reporting system and method for Car sharing and the shared vehicle of seating | |
| CN110175686A (en) | Monitor vehicle maintenance qualities | |
| CN110290945A (en) | Record the video of operator and around visual field | |
| CN109711557A (en) | A kind of wheelpath prediction technique, system, computer equipment and storage medium | |
| CN110751530B (en) | Identity information security verification method, server, passenger end and driver end | |
| DE102020122757A1 (en) | SYSTEMS AND PROCEDURES FOR RIDE OPPORTUNITIES USING BLOCKCHAIN | |
| CN109214258A (en) | Lose the detection method and device that the personnel that drive drive in violation of rules and regulations | |
| CN110312639A (en) | Vehicle assistant drive device, vehicle and information processing method | |
| CN106503622A (en) | A kind of vehicle antitracking method and device | |
| CN106809214A (en) | A kind of rear-end collision method for early warning, device and electronic equipment | |
| CN109409259A (en) | Drive monitoring method, device, equipment and computer-readable medium | |
| US11335190B2 (en) | Storage system for vehicle | |
| CN107878385A (en) | Running obstruction warning method and apparatus | |
| CN107303907A (en) | For the apparatus and method for the sleepiness for determining driver | |
| CN108099914A (en) | For controlling the apparatus and method of the stopping of vehicle | |
| CN108437741A (en) | The protection processing method and device of children in vehicles | |
| CN105761326A (en) | Method and device for managing vehicle traveling data, and equipment | |
| DE102006053328B4 (en) | A method of authenticating a driver of a vehicle and associated apparatus | |
| CN108062520A (en) | A kind of driver identification system and method | |
| CN109760695A (en) | Traffic-accident image processing unit based on viewing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 3011, Room 3, Building 27, 25 North Third Ring West Road, Haidian District, Beijing Applicant after: Public Question (Beijing) Information Technology Co., Ltd. Address before: 100094 Beijing Haidian District Shangzhuang Zhenzhuang Road No. 115 Courtyard Level 522 Applicant before: Intelligent information technology (Beijing) Co., Ltd. |
|
| CB02 | Change of applicant information | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180223 |
|
| RJ01 | Rejection of invention patent application after publication |