CN105469073A - Kinect-based call making and answering monitoring method of driver - Google Patents

Kinect-based call making and answering monitoring method of driver Download PDF

Info

Publication number
CN105469073A
CN105469073A CN201510955224.2A CN201510955224A CN105469073A CN 105469073 A CN105469073 A CN 105469073A CN 201510955224 A CN201510955224 A CN 201510955224A CN 105469073 A CN105469073 A CN 105469073A
Authority
CN
China
Prior art keywords
driver
head node
kinect
relative distance
monitoring method
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201510955224.2A
Other languages
Chinese (zh)
Inventor
谢宝
姚莉莉
张传金
纪勇
万海峰
张芝华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ANHUI CREARO TECHNOLOGY Co Ltd
Original Assignee
ANHUI CREARO TECHNOLOGY Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ANHUI CREARO TECHNOLOGY Co Ltd filed Critical ANHUI CREARO TECHNOLOGY Co Ltd
Priority to CN201510955224.2A priority Critical patent/CN105469073A/en
Publication of CN105469073A publication Critical patent/CN105469073A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a Kinect-based call making and answering monitoring method of a driver. The method comprises the following steps: adopting Kinect somatosensory equipment to carry out image acquisition on a driving area, and carrying out face detection on an acquired image through a face strong classifier which finishes training; if a face is in the presence, adopting the Kinect somatosensory equipment to capture the limb movement of the driver, extracting the depth information of the driver, and constructing a human body limb structure bone diagram according to the extracted depth information; according to the human body limb structure bone diagram, independently calculating a relative distance between a head node and a left hand node and the relative distance between the head node and a right hand node; if the relative distance between the head node and the left hand node or the relative distance between the head node and the right hand node is smaller than a limiting value, judging as an abnormal driving state; and if the duration of the abnormal driving state exceeds a preset threshold value, giving an alarm. The Kinect-based call making and answering monitoring method has the advantages of being high in stability and accurate and reliable in detection results and improves driving safety.

Description

A kind of driver based on Kinect plays phone-monitoring method
Technical field
The present invention relates to safe driving monitoring technique field, specifically a kind of driver based on Kinect plays phone-monitoring method.
Background technology
Along with the lifting of people's living standard in this year, automobile progressively enters into the life of people as household requisites.The growth at full speed of automobile pollution has brought very large facility, but also causes various traffic hazard to take place frequently simultaneously, and the life security of people and property all suffer very large loss.According to statistics, the contingency occurrence probability of driving when seeing the mobile phone is 23 times of common driving, and the contingency occurrence probability of driving when making a phone call is 2.8 times of common driving.
Driver how is effectively avoided to play phone under steam, a lot of scholar is proposed some good ideas, such as Chinese patent application CN104573659A discloses " a kind of driver based on svm plays phone-monitoring method ", Chinese patent application CN104156717A discloses " a kind of driver based on image processing techniques drive recognition methods violating the regulations of making a phone call ", this two pieces patented claim is all by the real-time monitoring driving person of camera, judge whether driver is playing phone by the identification of hand and head, in the environment of reality, image is larger by illumination effect, be difficult to detect exactly, Chinese patent application CN104486485A discloses " a kind of active prevent from driving driver play the method for phone ", this patented claim is by detecting car speed, when reaching certain speed, Hall switch detects magnetic flux, and stop driver and play phone, detected by mobile phone signal during this mode, the mobile phone signal of driver or passenger can not be determined, the object that monitoring driving person plays phone cannot be reached.
Summary of the invention
The object of the present invention is to provide a kind of driver based on Kinect to play phone-monitoring method, adopt the method can catch the limb action of driver in real time, thus whether monitoring driving person is playing phone, improving drive safety.
Technical scheme of the present invention is:
Driver based on Kinect plays a phone-monitoring method, comprises the following steps:
(1) adopt Kinect somatosensory device to carry out image acquisition to driver area, and by the face strong classifier of having trained, Face datection is carried out to the image gathered;
(2) judge whether to there is face, if so, then illustrate that driver area exists driver, performs step (3), if not, then illustrate that driver area does not exist driver, returns step (1);
(3) adopt Kinect somatosensory device to catch the limb action of driver, extract the depth information of driver, and build human body limb structure skeletal graph according to the depth information extracted;
(4) according to described human body limb structure skeletal graph, head node and left-hand minutia, relative distance between head node and right-hand minutia is calculated respectively;
(5) judge whether described head node and left-hand minutia or the relative distance between described head node and right-hand minutia are less than a threshold value, if so, be then judged to be abnormal driving state, perform step (6), if not, then step (3) is returned;
(6) judge whether the duration of described abnormal driving state exceedes predetermined threshold value, if so, then reports to the police, if not, then return step (3).
The described driver based on Kinect plays phone-monitoring method, and in step (1), the acquisition of described face strong classifier comprises the following steps:
A, obtain some facial images as training sample;
The haar feature of b, extraction face;
C, obtain face strong classifier by adaboost cascade classifier.
The described driver based on Kinect plays phone-monitoring method, in step (4), adopts following formula to calculate head node and left-hand minutia, relative distance between head node and right-hand minutia respectively:
d 01 = ( x 0 - x 1 ) 2 + ( y 0 - y 1 ) 2 + ( z 0 - z 1 ) 2
d 02 = ( x 0 - x 2 ) 2 + ( y 0 - y 2 ) 2 + ( z 0 - z 2 ) 2
Wherein, d 01represent the relative distance between head node and left-hand minutia, d 02represent the relative distance between head node and right-hand minutia, (x 0, y 0, z 0) represent the coordinate of head node, (x 1, y 1, z 1) represent the coordinate of left-hand minutia, (x 2, y 2, z 2) represent the coordinate of right-hand minutia.
The described driver based on Kinect plays phone-monitoring method, and in step (5), a described threshold value is 20 centimeters.
The described driver based on Kinect plays phone-monitoring method, and in step (6), described predetermined threshold value was 5 seconds.
Beneficial effect of the present invention is:
As shown from the above technical solution, the present invention uses image information and the depth information of Kinect somatosensory device collection driver, judge whether it is playing phone by the limb action of driver, stability of the present invention is high, testing result accurately and reliably, can use under complex environment, remind driver safety to drive, thus improve drive safety.
Accompanying drawing explanation
Fig. 1 is method flow diagram of the present invention;
Fig. 2 is abnormal driving state and the simulation drawing playing telephone state, wherein, Fig. 2 (a) is the simulation drawing of abnormal driving state, and Fig. 2 (b) is the simulation drawing that left hand plays telephone state, and Fig. 2 (c) is the simulation drawing that the right hand plays telephone state.
Embodiment
The present invention is further illustrated below in conjunction with the drawings and specific embodiments.
As shown in Figure 1, a kind of driver based on Kinect plays phone-monitoring method, comprises the following steps:
S1, obtain some facial images as training sample, extract the haar feature of face, then obtain face strong classifier by adaboost cascade classifier.
S2, employing Kinect somatosensory device carry out image acquisition to driver, and by face strong classifier, Face datection is carried out to the image collected, judge whether that face exists, if so, then illustrate to there is driver, enter step S3, if not, then illustrate to there is not driver, circulation step S2, proceed image acquisition and Face datection.
S3, employing Kinect somatosensory device catch the limb action of driver, build human body limb structure skeletal graph according to the depth information of the driver collected.
S4, judge the driving state of driver according to the relative distance between the head node in human body limb structure skeletal graph and left hand, right-hand minutia.
As shown in Figure 2, under normal circumstances, the distance between the head node of driver and left hand, right-hand minutia, more than 50 centimeters, should not be less than 20 centimeters.
Adopt the relative distance between following formulae discovery head node and left hand, right-hand minutia:
d 01 = ( x 0 - x 1 ) 2 + ( y 0 - y 1 ) 2 + ( z 0 - z 1 ) 2
d 02 = ( x 0 - x 2 ) 2 + ( y 0 - y 2 ) 2 + ( z 0 - z 2 ) 2
Wherein, d 01represent the relative distance between head node and left-hand minutia, d 02represent the relative distance between head node and right-hand minutia, (x 0, y 0, z 0) represent the coordinate of head node, (x 1, y 1, z 1) represent the coordinate of left-hand minutia, (x 2, y 2, z 2) represent the coordinate of right-hand minutia.
If the relative distance between S5 driver head node and left hand or right-hand minutia is less than 20 centimeters, is then considered as abnormal driving state, and starts timing, if the duration of this abnormal driving state more than 5 seconds, then reports to the police, send the chimes of doom dripped.
The present invention uses image and the depth information of the Kinect somatosensory device Real-time Obtaining driver of Microsoft, detection and location driver is carried out by image information, the manikin of driver is built by depth information, relative position between head node in the manikin of driver and hand node is calculated, judge whether to be less than a threshold value, and judge whether driver is according to residence time length and play telephone state, if, then send alarm, be similar to the warning device of securing band, send the chimes of doom dripped, prompting driver safety is driven.
The Kinect used has three camera lenses, and middle camera lens is RGB colour TV camera, is used for gathering coloured image; The right and left camera lens is then respectively the 3D structured light degree of depth inductor that infrared transmitter and infrared C MOS video camera are formed, and is used for sampling depth information (in scene, object is to the distance of camera).
The above embodiment is only be described the preferred embodiment of the present invention; not scope of the present invention is limited; under not departing from the present invention and designing the prerequisite of spirit; the various distortion that those of ordinary skill in the art make technical scheme of the present invention and improvement, all should fall in protection domain that claims of the present invention determine.

Claims (5)

1. the driver based on Kinect plays a phone-monitoring method, it is characterized in that, comprises the following steps:
(1) adopt Kinect somatosensory device to carry out image acquisition to driver area, and by the face strong classifier of having trained, Face datection is carried out to the image gathered;
(2) judge whether to there is face, if so, then illustrate that driver area exists driver, performs step (3), if not, then illustrate that driver area does not exist driver, returns step (1);
(3) adopt Kinect somatosensory device to catch the limb action of driver, extract the depth information of driver, and build human body limb structure skeletal graph according to the depth information extracted;
(4) according to described human body limb structure skeletal graph, head node and left-hand minutia, relative distance between head node and right-hand minutia is calculated respectively;
(5) judge whether described head node and left-hand minutia or the relative distance between described head node and right-hand minutia are less than a threshold value, if so, be then judged to be abnormal driving state, perform step (6), if not, then step (3) is returned;
(6) judge whether the duration of described abnormal driving state exceedes predetermined threshold value, if so, then reports to the police, if not, then return step (3).
2. the driver based on Kinect according to claim 1 plays phone-monitoring method, it is characterized in that, in step (1), the acquisition of described face strong classifier comprises the following steps:
A, obtain some facial images as training sample;
The haar feature of b, extraction face;
C, obtain face strong classifier by adaboost cascade classifier.
3. the driver based on Kinect according to claim 1 plays phone-monitoring method, it is characterized in that, in step (4), following formula is adopted to calculate head node and left-hand minutia, relative distance between head node and right-hand minutia respectively:
d 01 = ( x 0 - x 1 ) 2 + ( y 0 - y 1 ) 2 + ( z 0 - z 1 ) 2
d 02 = ( x 0 - x 2 ) 2 + ( y 0 - y 2 ) 2 + ( z 0 - z 2 ) 2
Wherein, d 01represent the relative distance between head node and left-hand minutia, d 02represent the relative distance between head node and right-hand minutia, (x 0, y 0, z 0) represent the coordinate of head node, (x 1, y 1, z 1) represent the coordinate of left-hand minutia, (x 2, y 2, z 2) represent the coordinate of right-hand minutia.
4. the driver based on Kinect according to claim 1 plays phone-monitoring method, it is characterized in that, in step (5), a described threshold value is 20 centimeters.
5. the driver based on Kinect according to claim 1 plays phone-monitoring method, it is characterized in that, in step (6), described predetermined threshold value was 5 seconds.
CN201510955224.2A 2015-12-16 2015-12-16 Kinect-based call making and answering monitoring method of driver Pending CN105469073A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510955224.2A CN105469073A (en) 2015-12-16 2015-12-16 Kinect-based call making and answering monitoring method of driver

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510955224.2A CN105469073A (en) 2015-12-16 2015-12-16 Kinect-based call making and answering monitoring method of driver

Publications (1)

Publication Number Publication Date
CN105469073A true CN105469073A (en) 2016-04-06

Family

ID=55606744

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510955224.2A Pending CN105469073A (en) 2015-12-16 2015-12-16 Kinect-based call making and answering monitoring method of driver

Country Status (1)

Country Link
CN (1) CN105469073A (en)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106022242A (en) * 2016-05-13 2016-10-12 哈尔滨工业大学（威海） Driver call making identification method in intelligent transportation system
CN106530730A (en) * 2016-11-02 2017-03-22 重庆中科云丛科技有限公司 Traffic violation detection method and system
CN109086729A (en) * 2018-08-13 2018-12-25 成都盯盯科技有限公司 Communication behavior detection method, device, equipment and storage medium
CN109492602A (en) * 2018-11-21 2019-03-19 华侨大学 A kind of process clocking method and system based on human body limb language
CN109886150A (en) * 2019-01-29 2019-06-14 上海佑显科技有限公司 A kind of driving behavior recognition methods based on Kinect video camera
CN109948450A (en) * 2019-02-22 2019-06-28 深兰科技（上海）有限公司 A kind of user behavior detection method, device and storage medium based on image
CN109963140A (en) * 2017-12-25 2019-07-02 深圳超多维科技有限公司 Nakedness-yet stereoscopic display method and device, equipment and computer readable storage medium
CN110688921A (en) * 2019-09-17 2020-01-14 东南大学 Method for detecting smoking behavior of driver based on human body action recognition technology
CN111301280A (en) * 2018-12-11 2020-06-19 北京嘀嘀无限科技发展有限公司 Dangerous state identification method and device
CN111461020A (en) * 2020-04-01 2020-07-28 浙江大华技术股份有限公司 Method and device for identifying behaviors of insecure mobile phone and related storage medium
CN111553190A (en) * 2020-03-30 2020-08-18 浙江工业大学 Image-based driver attention detection method
CN111815899A (en) * 2019-04-12 2020-10-23 泰州市康平医疗科技有限公司 Target distance real-time measuring method and storage medium
CN113361343A (en) * 2021-05-21 2021-09-07 上海可深信息科技有限公司 Deep learning based call receiving and making behavior detection method
CN113591661A (en) * 2021-07-24 2021-11-02 深圳市铁越电气有限公司 Call-making behavior prediction method and system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103366506A (en) * 2013-06-27 2013-10-23 北京理工大学 Device and method for automatically monitoring telephone call behavior of driver when driving
CN104573659A (en) * 2015-01-09 2015-04-29 安徽清新互联信息科技有限公司 Driver call-making and call-answering monitoring method based on svm
CN104732251A (en) * 2015-04-23 2015-06-24 郑州畅想高科股份有限公司 Video-based method of detecting driving state of locomotive driver

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103366506A (en) * 2013-06-27 2013-10-23 北京理工大学 Device and method for automatically monitoring telephone call behavior of driver when driving
CN104573659A (en) * 2015-01-09 2015-04-29 安徽清新互联信息科技有限公司 Driver call-making and call-answering monitoring method based on svm
CN104732251A (en) * 2015-04-23 2015-06-24 郑州畅想高科股份有限公司 Video-based method of detecting driving state of locomotive driver

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
SEAN L.GALLAHAN等: "Detecting and Mitigating Driver Distraction with Motion Capture Technology: Distracted Driving Warning System", 《PROCEEDINGS OF THE 2013 IEEE SYSTEMS AND INFORMATION》 *
王志良等: "《物联网技术综合实训教程》", 30 April 2014, 机械工业出版社 *

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106022242B (en) * 2016-05-13 2019-05-03 哈尔滨工业大学（威海） Driver's making and receiving calls recognition methods in intelligent transportation system
CN106022242A (en) * 2016-05-13 2016-10-12 哈尔滨工业大学（威海） Driver call making identification method in intelligent transportation system
CN106530730A (en) * 2016-11-02 2017-03-22 重庆中科云丛科技有限公司 Traffic violation detection method and system
CN109963140A (en) * 2017-12-25 2019-07-02 深圳超多维科技有限公司 Nakedness-yet stereoscopic display method and device, equipment and computer readable storage medium
CN109086729A (en) * 2018-08-13 2018-12-25 成都盯盯科技有限公司 Communication behavior detection method, device, equipment and storage medium
CN109492602A (en) * 2018-11-21 2019-03-19 华侨大学 A kind of process clocking method and system based on human body limb language
CN111301280A (en) * 2018-12-11 2020-06-19 北京嘀嘀无限科技发展有限公司 Dangerous state identification method and device
CN109886150A (en) * 2019-01-29 2019-06-14 上海佑显科技有限公司 A kind of driving behavior recognition methods based on Kinect video camera
CN109948450A (en) * 2019-02-22 2019-06-28 深兰科技（上海）有限公司 A kind of user behavior detection method, device and storage medium based on image
CN111815899A (en) * 2019-04-12 2020-10-23 泰州市康平医疗科技有限公司 Target distance real-time measuring method and storage medium
CN111815899B (en) * 2019-04-12 2021-02-12 汇金智融(深圳)科技有限公司 Target distance real-time measuring method and storage medium
CN110688921A (en) * 2019-09-17 2020-01-14 东南大学 Method for detecting smoking behavior of driver based on human body action recognition technology
CN111553190A (en) * 2020-03-30 2020-08-18 浙江工业大学 Image-based driver attention detection method
CN111461020A (en) * 2020-04-01 2020-07-28 浙江大华技术股份有限公司 Method and device for identifying behaviors of insecure mobile phone and related storage medium
CN111461020B (en) * 2020-04-01 2024-01-19 浙江大华技术股份有限公司 Recognition method, equipment and related storage medium for unsafe mobile phone behavior
CN113361343A (en) * 2021-05-21 2021-09-07 上海可深信息科技有限公司 Deep learning based call receiving and making behavior detection method
CN113591661A (en) * 2021-07-24 2021-11-02 深圳市铁越电气有限公司 Call-making behavior prediction method and system

Similar Documents

Publication Publication Date Title
CN105469073A (en) Kinect-based call making and answering monitoring method of driver
CN101639894B (en) Method for detecting train driver behavior and fatigue state on line and detection system thereof
CN101577812B (en) Method and system for post monitoring
CN104013414B (en) A kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone
CN104637246B (en) Driver multi-behavior early warning system and danger evaluation method
CN105844128B (en) Identity recognition method and device
CN103366565B (en) Method and system of detecting pedestrian running red light based on Kinect
CN103279737B (en) A kind of behavioral value method of fighting based on space-time interest points
CN106295551A (en) A kind of personal security cap wear condition real-time detection method based on video analysis
CN102622600A (en) High-speed train driver alertness detecting method based on face image and eye movement analysis
CN103366506A (en) Device and method for automatically monitoring telephone call behavior of driver when driving
CN101847206A (en) Pedestrian traffic statistical method and system based on traffic monitoring facilities
CN111325048B (en) Personnel gathering detection method and device
CN101178769B (en) Health protecting equipment and realization method thereof
CN109543577A (en) A kind of fatigue driving detection method for early warning based on facial expression feature
CN104700088A (en) Gesture track recognition method based on monocular vision motion shooting
CN106331605B (en) A kind of floods detection system and method based on video
CN106599873A (en) Figure identity identification method based on three-dimensional attitude information
CN102914286A (en) Method for automatically detecting user sitting posture based on handheld equipment
CN101183477A (en) Automatic door bell and implementing method thereof
CN104284149A (en) Electric line dangerous source point oversize vehicle shape change detection video recognition method
CN111383248A (en) Method and device for judging red light running of pedestrian and electronic equipment
CN104392201B (en) A kind of human body tumble recognition methods based on omnidirectional vision
CN109308467A (en) Traffic accident prior-warning device and method for early warning based on machine learning
CN203885510U (en) Driver fatigue detection system based on infrared detection technology

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: 230088, 12 Dream Garden Road, hi tech Zone, Anhui, Hefei

Applicant after: ANHUI CREARO TECHNOLOGY CO., LTD.

Address before: 230088, 12 Dream Garden Road, hi tech Zone, Anhui, Hefei

Applicant before: Anhui Crearo Technology Co., Ltd.

COR Change of bibliographic data
RJ01 Rejection of invention patent application after publication

Application publication date: 20160406

RJ01 Rejection of invention patent application after publication