CN104317397A - Vehicle-mounted man-machine interactive method - Google Patents

Vehicle-mounted man-machine interactive method Download PDF

Info

Publication number
CN104317397A
CN104317397A CN201410542613.8A CN201410542613A CN104317397A CN 104317397 A CN104317397 A CN 104317397A CN 201410542613 A CN201410542613 A CN 201410542613A CN 104317397 A CN104317397 A CN 104317397A
Authority
CN
China
Prior art keywords
image
camera
gesture
vehicle
processing module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201410542613.8A
Other languages
Chinese (zh)
Inventor
沙文瀚
王建辉
李芳珍
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chery Automobile Co Ltd
Original Assignee
SAIC Chery Automobile Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SAIC Chery Automobile Co Ltd filed Critical SAIC Chery Automobile Co Ltd
Priority to CN201410542613.8A priority Critical patent/CN104317397A/en
Publication of CN104317397A publication Critical patent/CN104317397A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20Movements or behaviour, e.g. gesture recognition
    • G06V40/28Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107Static hand or arm
    • G06V40/113Recognition of static hand signs

Landscapes

  • Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Image Analysis (AREA)

Abstract

The invention belongs to the technical field of intelligent control for vehicles, and particularly relates to a vehicle-mounted man-machine interactive method. The method comprises the following steps of (A) starting a camera; (B) acquiring an image in a cab by using the camera, and recognizing a gesture from the image acquired by the camera through an image processing module, wherein the image at least comprises driver information; (C) if the recognized gesture does not exist, repeating the step B, and if the recognized gesture exists, entering a step D; (D) matching the recognized gesture and outputting a corresponding control command by using the image processing module; (E) receiving the control command and correspondingly controlling a vehicle by using an executive module; (F) repeating the step B, if the gesture exists and does not change, continuously repeating the step B, if the gesture exists and changes, entering the step D, and if the recognized gesture does not exist, stopping outputting the control command and repeating the step B. According to the vehicle-mounted man-machine interactive method, the operation is accurate, a risk brought by pressing a key by a driver in the driving process is avoided, and the driving experience and comfort of the driver are improved.

Description

Vehicle-mounted man-machine interaction method
Technical field
The invention belongs to Vehicular intelligent control technology field, particularly the vehicle-mounted man-machine interaction method of one.
Background technology
Day by day popularizing and widely using along with automobile, the comfortableness of automobile and security become the problem that user more and more pays close attention to, people are in the process of steering vehicle, touching button is often needed to go to control vehicle, such as air-conditioning switch, air-out size, the selection of music, volume regulate, play time-out, etc.
Most of vehicle that Vehicles Collected from Market is sold, due to the limited space near instrument desk, most switch all concentrates on console, and button is intensive and position is close.User is when being easy to push the wrong without manual manipulation direct when eye observation; Condition of road surface emergency case is more, if user pays close attention to button operation in the process of driving to travel, because miss important vehicle condition information in the short time of concern button, may cause major traffic accidents.Conventional solution adopts multi-functional steering wheel, but multi-functional steering wheel involves river air bag system common clock spring, and the higher and function quantity of cost is limited to, and can not arbitrarily expand and change.
Summary of the invention
The object of the present invention is to provide a kind of vehicle-mounted man-machine interaction method, by identifying that driver's gesture controls vehicle, avoiding driver to pay close attention to button and grasping the accident risk brought.
For realizing above object, the technical solution used in the present invention is: a kind of vehicle-mounted man-machine interaction method, comprise the steps: that (A) starts camera, (B) camera obtains drives off-the-air picture, at least includes driver information in this image; Image processing module identifies gesture from the image that camera gets; (C) if without identification gesture, repeat step B; If there is identification gesture, enter step D; (D) identification gesture is mated with the gesture template of its storage by image processing module; According to the gesture template matched, export corresponding steering order; (E) execution module controls accordingly to vehicle after receiving steering order; (F) repeat step B, if gesture exists and do not change, then continue to repeat step B; If gesture exists and changes, then enter step D; If without identification gesture, stop exporting steering order and repeating step B.
Compared with prior art, there is following technique effect in the present invention: by arranging camera and image processing module, identifies the gesture of driver, when driver needs to operate, only need to make corresponding gesture facing to camera, can complete operation; By gesture identification, operation accurately, and avoids driver in driving conditions and need to bow the risk that button brings, also improve simultaneously driver drive experience and comfort level, and car load is intelligent.
Accompanying drawing explanation
Fig. 1 is process flow diagram of the present invention;
Fig. 2 is theory diagram of the present invention.
Embodiment
Below in conjunction with Fig. 1 to Fig. 2, the present invention is described in further detail.
Consult Fig. 1, a kind of vehicle-mounted man-machine interaction method, comprise the steps: that (A) starts camera, (B) camera obtains drives off-the-air picture, at least includes driver information in this image; Image processing module identifies gesture from the image that camera gets; (C) if without identification gesture, repeat step B; If there is identification gesture, enter step D; (D) identification gesture is mated with the gesture template of its storage by image processing module; According to the gesture template matched, export corresponding steering order; (E) execution module controls accordingly to vehicle after receiving steering order; (F) repeat step B, if gesture exists and do not change, then continue to repeat step B; If gesture exists and changes, then enter step D; If without identification gesture, stop exporting steering order and repeating step B.User in use, can according to the custom of oneself and need by gesture template and corresponding steering order pre-set, conventional adjustment as vehicle music volume adjusting, change the adjustment etc. of song, air-conditioning air volume.
The scope caught due to camera is larger, in use, the eyes of driver can not leave road surface, gesture is made in the region that directly can capture facing to camera, input to image processing module and carry out identifying and export corresponding steering order after camera catches, avoid driver and observe the accident potential that button may bring, similarly, this mode of operation is for driver, and it is more convenient to operate.
Further, in described step B, image processing module obtains identification gesture as follows: (B1) camera obtains the image not containing driver in pilothouse, and using this image as initial background image; (B2) image containing driver got by camera and initial background image compare computing, obtain foreground image; (B3) the people's hand portion in identification foreground image also extracts staff image; (B4) filtering is carried out to staff image, edge treated, rim detection, edge extracting obtain digitized staff graphical information, image recognition is become graphical information, has a lot of algorithm to realize, here just not at detailed description.Here by arranging initial background image, and from foreground image, identifying people's hand portion, the calculated amount of identification can be reduced.Initial background image can, after camera installs, when driver is not on seat, carry out gathering and being stored in image processing module.
Camera can according to object movement and static state, analyze and whether have picture in driver and the car when non-driver, along with the change of stationary article in car, camera or image processing module are understood self study and upgrade initial background image, to reduce calculated amount during staff identification.In described step B2, if there is stagnant zone in foreground image and rest time meets threshold value, then this region is added in initial background image, namely described stagnant zone processes the identical region of the foreground image that obtains at every turn, and popular point is said, in the image got when camera exactly, the image in certain region continues for some time and does not change, just the image in this region is considered as background image, before carrying out staff identification, is removed.When it should be noted that camera starts at every turn, the background image of acquiescence is exactly the image starting most to store, and afterwards when carrying out gesture identification, the background image changed is only as the judgement in this driving procedure.
As preferred version of the present invention, start camera as follows: after (A1) car load battery feed is connected, system reset; (A2) system carries out self-inspection; (A3) when ignition switch ON shelves power supply is connected, when after enabling signal vehicle being detected, camera starts; When ignition switch ON shelves power supply disconnects, after ALARM signal being detected, camera starts.Here ALARM signal is provided with, even if can ensure that the ignition switch ON shelves power supply of vehicle disconnects, also can send ALARM signal at any time by vehicle-bone 3 G network camera is started, obtain video or image information in car, to reach the object of remote monitor vehicle.
Further, described image processing module analyzes the brightness of the image that camera gets, if brightness is less than the threshold value of setting, then enables infrared light compensating lamp.For night or sleety weather, when in car during insufficient light, the image light that camera obtains is darker, smudgy, be unfavorable for the identification of gesture, therefore ensure that camera obtains the sharpness of picture by infrared light compensating lamp, further, the light of infrared light compensating lamp can not be identified by the human eye, and has no effect to driver.
Because driver is sitting on the seat in pilothouse, therefore driver is when doing gesture, scope is limited, in order to ensure that camera can capture the gesture of driver, under described camera can being positioned at air-conditioner air outlet, focal length 10 ~ 40cm place before camera of camera.In addition, when carrying out gesture identification, exceeding the image outside effective focal length scope, directly using algorithm filtration treatment, reduce calculated amount.
Consult as Fig. 2, preferably, the control signal that described image processing module also exports can be CAN signal, LIN signal, PIN signal, CAN and LIN two kinds of networks that system is reserved and 6 tunnel PIN drive conventional control assembly, can meet the demand of most users.

Claims (7)

1. a vehicle-mounted man-machine interaction method, comprises the steps:
(A) camera is started,
(B) camera obtains and drives off-the-air picture, at least includes driver information in this image; Image processing module identifies gesture from the image that camera gets;
(C) if without identification gesture, repeat step B; If there is identification gesture, enter step D;
(D) identification gesture is mated with the gesture template of its storage by image processing module; According to the gesture template matched, export corresponding steering order;
(E) execution module controls accordingly to vehicle after receiving steering order;
(F) repeat step B, if gesture exists and do not change, then continue to repeat step B; If gesture exists and changes, then enter step D; If without identification gesture, stop exporting steering order and repeating step B.
2. vehicle-mounted man-machine interaction method as claimed in claim 1, it is characterized in that: in described step B, image processing module obtains identification gesture as follows:
(B1) camera obtains the image not containing driver in pilothouse, and using this image as initial background image;
(B2) image containing driver got by camera and initial background image compare computing, obtain foreground image;
(B3) the people's hand portion in identification foreground image also extracts staff image;
(B4) filtering is carried out to staff image, edge treated, rim detection, edge extracting obtain digitized staff graphical information.
3. vehicle-mounted man-machine interaction method as claimed in claim 2, it is characterized in that: in described step B2, if there is stagnant zone in foreground image and rest time meets threshold value, then be added in initial background image by this region, namely described stagnant zone processes the identical region of the foreground image that obtains at every turn.
4. vehicle-mounted man-machine interaction method as claimed in claim 3, is characterized in that: start camera as follows:
(A1) after car load battery feed is connected, system reset;
(A2) system carries out self-inspection;
(A3) when ignition switch ON shelves power supply is connected, when after enabling signal vehicle being detected, camera starts; When ignition switch ON shelves power supply disconnects, after ALARM signal being detected, camera starts.
5. vehicle-mounted man-machine interaction method as claimed in claim 4, is characterized in that: described image processing module analyzes the brightness of the image that camera gets, if brightness is less than the threshold value of setting, then enables infrared light compensating lamp.
6. vehicle-mounted man-machine interaction method as claimed in claim 5, is characterized in that: under described camera is positioned at air-conditioner air outlet, focal length 10 ~ 40cm place before camera of camera.
7. vehicle-mounted man-machine interaction method as claimed in claim 6, is characterized in that: the control signal that described image processing module also exports can be CAN signal, LIN signal, PIN signal.
CN201410542613.8A 2014-10-14 2014-10-14 Vehicle-mounted man-machine interactive method Pending CN104317397A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410542613.8A CN104317397A (en) 2014-10-14 2014-10-14 Vehicle-mounted man-machine interactive method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410542613.8A CN104317397A (en) 2014-10-14 2014-10-14 Vehicle-mounted man-machine interactive method

Publications (1)

Publication Number Publication Date
CN104317397A true CN104317397A (en) 2015-01-28

Family

ID=52372636

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410542613.8A Pending CN104317397A (en) 2014-10-14 2014-10-14 Vehicle-mounted man-machine interactive method

Country Status (1)

Country Link
CN (1) CN104317397A (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105224088A (en) * 2015-10-22 2016-01-06 东华大学 A kind of manipulation of the body sense based on gesture identification vehicle-mounted flat system and method
CN106527674A (en) * 2015-09-14 2017-03-22 上海羽视澄蓝信息科技有限公司 Human-computer interaction method, equipment and system for vehicle-mounted monocular camera
CN106886275A (en) * 2015-12-15 2017-06-23 比亚迪股份有限公司 The control method of car-mounted terminal, device and vehicle
CN106933346A (en) * 2017-01-20 2017-07-07 深圳奥比中光科技有限公司 The zoning methods and equipment in in-car manipulation space
WO2017147877A1 (en) * 2016-03-03 2017-09-08 邱琦 Image-based identification method
CN108482276A (en) * 2018-02-02 2018-09-04 斑马网络技术有限公司 Vehicle screen method for information display and its system
CN108710436A (en) * 2018-06-27 2018-10-26 常州星宇车灯股份有限公司 A kind of control system based on TOF gesture identifications
CN108710857A (en) * 2018-05-22 2018-10-26 深圳前海华夏智信数据科技有限公司 People's vehicle recognition methods based on infrared light filling and device
CN108860025A (en) * 2018-06-12 2018-11-23 重庆延锋安道拓汽车部件系统有限公司 Seat adjuster and adjusting method based on gesture identification
CN109366497A (en) * 2018-11-12 2019-02-22 奇瑞汽车股份有限公司 On-vehicle machines people, the control method of on-vehicle machines people, device and storage medium
CN109934999A (en) * 2017-12-16 2019-06-25 北京嘀嘀无限科技发展有限公司 A kind of shared motor bicycle customized information reminding method and system
CN110889306A (en) * 2018-09-07 2020-03-17 广州汽车集团股份有限公司 Vehicle-mounted gesture recognition method and system based on camera
CN112069914A (en) * 2020-08-14 2020-12-11 杭州鸿泉物联网技术股份有限公司 Vehicle control method and system

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101201934A (en) * 2006-12-15 2008-06-18 财团法人工业技术研究院 Method for subsection of video object
CN102509073A (en) * 2011-10-17 2012-06-20 上海交通大学 Static target segmentation method based on Gauss background model
CN102968797A (en) * 2012-12-03 2013-03-13 广州盈可视电子科技有限公司 Image foreground and background segmenting method and device
CN103175287A (en) * 2013-04-22 2013-06-26 清华大学 Energy-saving control method and device for detecting character movement for air conditioner based on background modeling
US20130162535A1 (en) * 2007-10-22 2013-06-27 Mitsubishi Electric Corporation Manipulation input device which detects human hand manipulations from captured motion images
CN103294190A (en) * 2012-02-06 2013-09-11 福特全球技术公司 Recognition system interacting with vehicle controls through gesture recognition
CN103440475A (en) * 2013-08-14 2013-12-11 北京博思廷科技有限公司 Automatic teller machine user face visibility judging system and method
CN103455144A (en) * 2013-08-22 2013-12-18 深圳先进技术研究院 Vehicle-mounted man-machine interaction system and method
CN103631157A (en) * 2012-08-29 2014-03-12 杨尧任 Device of employing hand gesture recognition technology to control vehicle electric appliance and application method thereof
CN103729858A (en) * 2013-12-13 2014-04-16 广州中国科学院先进技术研究所 Method for detecting article left over in video monitoring system
US20140185872A1 (en) * 2012-12-28 2014-07-03 Hyundai Motor Company Method and system for recognizing hand gesture using selective illumination

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101201934A (en) * 2006-12-15 2008-06-18 财团法人工业技术研究院 Method for subsection of video object
US20130162535A1 (en) * 2007-10-22 2013-06-27 Mitsubishi Electric Corporation Manipulation input device which detects human hand manipulations from captured motion images
CN102509073A (en) * 2011-10-17 2012-06-20 上海交通大学 Static target segmentation method based on Gauss background model
CN103294190A (en) * 2012-02-06 2013-09-11 福特全球技术公司 Recognition system interacting with vehicle controls through gesture recognition
CN103631157A (en) * 2012-08-29 2014-03-12 杨尧任 Device of employing hand gesture recognition technology to control vehicle electric appliance and application method thereof
CN102968797A (en) * 2012-12-03 2013-03-13 广州盈可视电子科技有限公司 Image foreground and background segmenting method and device
US20140185872A1 (en) * 2012-12-28 2014-07-03 Hyundai Motor Company Method and system for recognizing hand gesture using selective illumination
CN103175287A (en) * 2013-04-22 2013-06-26 清华大学 Energy-saving control method and device for detecting character movement for air conditioner based on background modeling
CN103440475A (en) * 2013-08-14 2013-12-11 北京博思廷科技有限公司 Automatic teller machine user face visibility judging system and method
CN103455144A (en) * 2013-08-22 2013-12-18 深圳先进技术研究院 Vehicle-mounted man-machine interaction system and method
CN103729858A (en) * 2013-12-13 2014-04-16 广州中国科学院先进技术研究所 Method for detecting article left over in video monitoring system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
XIAODONG CAI,FALAH ALI, E. STIPIDIS: "Background Modeling for Detecting Move-then-stop Arbitrary-long time Video Objects", 《IMAGE ANALYSIS FOR MULTIMEDIA INTERACTIVE SERVICES, 2009. WIAMIS "09. 10TH WORKSHOP ON》 *

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106527674A (en) * 2015-09-14 2017-03-22 上海羽视澄蓝信息科技有限公司 Human-computer interaction method, equipment and system for vehicle-mounted monocular camera
CN105224088A (en) * 2015-10-22 2016-01-06 东华大学 A kind of manipulation of the body sense based on gesture identification vehicle-mounted flat system and method
CN106886275A (en) * 2015-12-15 2017-06-23 比亚迪股份有限公司 The control method of car-mounted terminal, device and vehicle
CN106886275B (en) * 2015-12-15 2020-03-20 比亚迪股份有限公司 Control method and device of vehicle-mounted terminal and vehicle
WO2017147877A1 (en) * 2016-03-03 2017-09-08 邱琦 Image-based identification method
CN106933346A (en) * 2017-01-20 2017-07-07 深圳奥比中光科技有限公司 The zoning methods and equipment in in-car manipulation space
CN106933346B (en) * 2017-01-20 2019-07-26 深圳奥比中光科技有限公司 The zoning methods and equipment in car manipulation space
CN109934999A (en) * 2017-12-16 2019-06-25 北京嘀嘀无限科技发展有限公司 A kind of shared motor bicycle customized information reminding method and system
CN108482276A (en) * 2018-02-02 2018-09-04 斑马网络技术有限公司 Vehicle screen method for information display and its system
CN108710857A (en) * 2018-05-22 2018-10-26 深圳前海华夏智信数据科技有限公司 People's vehicle recognition methods based on infrared light filling and device
CN108860025A (en) * 2018-06-12 2018-11-23 重庆延锋安道拓汽车部件系统有限公司 Seat adjuster and adjusting method based on gesture identification
CN108860025B (en) * 2018-06-12 2024-05-14 安道拓(重庆)汽车部件有限公司 Seat adjusting device and adjusting method based on gesture recognition
CN108710436A (en) * 2018-06-27 2018-10-26 常州星宇车灯股份有限公司 A kind of control system based on TOF gesture identifications
CN110889306A (en) * 2018-09-07 2020-03-17 广州汽车集团股份有限公司 Vehicle-mounted gesture recognition method and system based on camera
CN109366497A (en) * 2018-11-12 2019-02-22 奇瑞汽车股份有限公司 On-vehicle machines people, the control method of on-vehicle machines people, device and storage medium
CN112069914A (en) * 2020-08-14 2020-12-11 杭州鸿泉物联网技术股份有限公司 Vehicle control method and system

Similar Documents

Publication Publication Date Title
CN104317397A (en) Vehicle-mounted man-machine interactive method
TWI741512B (en) Method, device and electronic equipment for monitoring driver's attention
CN108725357B (en) Parameter control method and system based on face recognition and cloud server
CN102707801B (en) Vehicle-mounted control system for identifying and control method thereof
CN109552340B (en) Gesture and expression control for vehicles
CN104461005B (en) A kind of Vehicular screen method of controlling switch
US11524578B2 (en) Control method and control device for vehicle display device
CN110114825A (en) Speech recognition system
KR101490908B1 (en) System and method for providing a user interface using hand shape trace recognition in a vehicle
CN105701442B (en) Passenger car stopping prompting method and device
US20200070848A1 (en) Method and System for Initiating Autonomous Drive of a Vehicle
CN108423004A (en) A kind of binocular identification driver status detecting system and method
CN104859559B (en) The control method and device of interior environment
CN109614953A (en) A kind of control method based on image recognition, mobile unit and storage medium
CN110070058A (en) A kind of vehicle-mounted gesture identifying device and system
CN105835761A (en) Method and device for controlling on and off of atmosphere lamps in car
CN107284449A (en) A kind of traffic safety method for early warning and system, automobile, readable storage medium storing program for executing
CN113591659A (en) Gesture control intention recognition method and system based on multi-modal input
CN114987500A (en) Driver state monitoring method, terminal device and storage medium
KR20120048190A (en) Vehicle control system using motion recognition
CN108459520B (en) Automatic display method for vehicle and system thereof
CN107139724A (en) Vehicular screen dynamic display method and its vehicle-mounted central control system for vehicle
CN109376682A (en) A kind of intelligent video camera head and fatigue state recognition method
CN111511599A (en) Method for operating an auxiliary system and auxiliary system for a motor vehicle
CN114701503A (en) Method, device and equipment for adjusting driving behavior of vehicle driver and storage medium

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20150128