CN105590084A - Robot human face detection tracking emotion detection system - Google Patents
Robot human face detection tracking emotion detection system Download PDFInfo
- Publication number
- CN105590084A CN105590084A CN201410606320.1A CN201410606320A CN105590084A CN 105590084 A CN105590084 A CN 105590084A CN 201410606320 A CN201410606320 A CN 201410606320A CN 105590084 A CN105590084 A CN 105590084A
- Authority
- CN
- China
- Prior art keywords
- face
- robot
- human face
- emotion
- tracking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a robot human face detection tracking emotion detection system. The system comprises a human face detection emotion recognition system and a human face tracking system, the two are connected with a master control computer system, the human face detection emotion recognition system comprises an acquisition human face expression database, an expression feature database construction module, a feature database reconstruction module and an expression recognition module, and the human face tracking system is connected with a digital steering engine control system. The robot human face detection tracking emotion detection system can realize autonomous real-time detection and tracking of a human face, emotion recognition and simulation of a robot and provides basic visual image data for other robot visual functions.
Description
Technical field
The face that the present invention relates to a kind of home services robot detects to be followed the tracks of and emotion recognition system.
Background technology
The recognition of face of existing home services robot is limited in one's ability, especially emotion recognition ability is very limited, cannot carry out omnibearing identification to old man in family or children's affective state, such as the face recognition module in patent of invention [1] [2], the facial image only absorbing by described camera head and pre-stored facial image compare to identify absorbed facial image, but recognition capability is limited, during particularly for the more facial image of storage, cannot effectively complete the identification of face. Invention [3] although in help the elderly/disabled aiding robot there is anti-theft monitoring, safety inspection, monitoring diagnosis and treatment, auxiliary walking, article carrying, family electric control, sanitation and hygiene, home entertaining, give the correct time waken, the function such as children education, but there is no the function of face detection and emotion recognition, cannot effectively identify the old man in family and child's state. Invention [4] is mainly by analyzing speech and face, carry out affective feature extraction, decision-making goes out the emotion that robot will express, and export by corresponding output equipment, produce facial expression and the body language of robot, but this system had not both had to realize the autokinetic movement of robot, there is no effectively to judge old man and child's behavior state yet. Invention [5] mainly provides a kind of mood of robot to generate and expression system, and the affective state aspect that detects to identify old man and children by face for how does not relate to. Invention [6] has proposed a kind of face based on multi-feature fusion location and has known method for distinguishing with head pose, can realize the face motion analysis under variation illumination condition, mobile context, but not by face motion analysis, solve people's emotion recognition problem.
[1] there is the intelligent robot of recognition of face, publication number: CN201163417Y, invention designer: Chen Zhenxiong; Yang Zhou; Li Na; The little equality of skin
[2] there is the safety protection robot of recognition of face, publication number: CN202985566U, invention designer: Wang Yun
[3] help the elderly/disabled aiding robot, publication number: CN202397747U, invention designer: Xiao Nanfeng
[4] emotional robot system, publication number: CN101474481A, inventor: Wang Zhiliang, Liu Yaofeng, Xie Lun, Zhao Hui etc.
[5] robot emotion emotion generates and an expression system, publication number: CN103218654A, inventor: Du Zhenjun, Xu Fang, Qu Daokui, Jia Kai, Zheng Chunhui, Tan Boyue etc.
[6] face based on multi-feature fusion location and head pose recognition methods, publication number: CN1601549A, inventor: Tan Tieniu, Wei Yucheng, Zhou Chao.
Summary of the invention
The technical problem to be solved in the present invention: face detection and emotion recognition system for present domestic robot application are mainly made up of removable camera and fixing camera, just realize merely face identification system, intelligent degree is limited, and cannot perception the problem of other information around, a kind of robot face identification system that can carry out face tracking and emotion recognition is provided.
Technical solution of the present invention:
A kind of robot face detects follows the tracks of emotion detection system, it is characterized in that: comprise that face detects emotion recognition system and face tracking system, both are all connected with host computer system, face detects emotion recognition system and comprises that collection face expression database, expressive features storehouse build module, feature database reconstructed module and Expression Recognition module, and face tracking system is connected with digital rudder controller control system.
Robot according to claim 1 face detects follows the tracks of emotion detection system, it is characterized in that: the method for work of face tracking system comprises four steps: step 1, on current goal position, set up online grader; Step 2, grader traversal search in candidate region that next frame utilizes previous step to set up, generates confidence map; Step 3. in confidence map, search for the reposition of maximum as target place; Step 4, at maximum position, place chooses sample, utilizes the new samples obtaining to upgrade grader, utilizes grader repeating step two and the step 3 after upgrading in next frame.
Beneficial effect of the present invention:
The present invention is mainly used in robotic vision perception and control, the face position in detected image and then the head rotation of control and robot direct of travel, and according to the variation of human face's expression, the emulation of expressing one's feelings by led dot matrix. The present invention can realize robot autonomous real-time detection and follow the trail of face, emotion recognition and emulation, and provides basic visual image data for other robot visual performance. In order to take into account robustness, accuracy and the real-time of face tracking and recognizer, utilize people's face detection algorithm to realize the automatic location of face target, then carry out face tracking taking CAMShift as track algorithm. Before CAMShift follows the tracks of, introduce prefilter, filtering class area of skin color, can effectively improve the tracking robustness of face, solves CAMShift and easily trace into the problem of class area of skin color.
Brief description of the drawings:
Fig. 1 is that the present invention forms structural representation.
Fig. 2 is the fundamental diagram of face detection and emotion recognition system.
Fig. 3 is the trace flow figure of face tracking system.
Detailed description of the invention:
Invention relates generally to the Intelligent home service robot in home environment, comprise telecontrol equipment, image processing module, control device, expression simulator, as shown in Figure 1, host computer system respectively connecting base plate controller, face detects emotion recognition system, communication module and audio amplifier to entirety composition structure; Chassis controller is connected with slave computer control system, controls the machine operation of the left and right wheels on chassis. Communication module connects respectively the Numeralize Drive Scheme of robot head and the step motor control system of robot arm, Numeralize Drive Scheme is connected with face tracking system, under the information of face tracking, 2 steering wheel work of system digits electric machine control system control head, 1 steering wheel is responsible for the rotation up and down of control neck, the left-right rotation of 1 steering wheel control head, thus eyeball (rotation of camera I and II) can be controlled.
The face of robot detects with emotion recognition system and forms by gathering face expression database, expressive features storehouse structure module, feature database reconstructed module and 4 parts of Expression Recognition module,
1) gather face expression database
Face expression database acquisition module completes and from camera, automatically carries out face detection and location, expression district, generates the training plan image set extracting for expressive features. This module need to be rotated face by human eye detection equally and according to face geometry feature, expression district accurately be located.
2) expressive features storehouse builds module
Expressive features storehouse builds module and uses PCA (PCA) dimensionality reduction to remove image redundancy information, extracts expressive features, and expressive features is saved as to file.
3) feature database reconstructed module
Feature database reconstructed module is used based on apart from Hash method (DBH), primitive character storehouse being reconstructed into structurized Hash table, as improving Expression Recognition efficiency.
4) Expression Recognition module
Expression Recognition module gathers color image frames from camera, changed into gray level image and use histogram equalization to carry out brightness normalization to gray level image, then carry out the steps such as accurately locate in face detection, human eye detection, face rotation, expression district, expressive features extraction and extract image expressive features, in last feature database after reconstruct, carry out k arest neighbors classification (KNN) and identify human face expression, carry out guidance machine people with this behavior of old man and children is predicted.
Home services robot face tracking system based on online pre-filtering CAMShift algorithm is described:
The face tracking system of described robot can be divided into five steps: (1) sets up online grader classified filtering on current goal position; (2) next frame utilizes grader traversal search in candidate region that previous step is set up, and sets up face tracking color of object histogram; (3) generate color probability distribution image according to histogram, and search for therein the reposition of maximum as target place; (4) choose sample at maximum position place, utilize the new samples obtaining to upgrade grader, judge whether target search restrains, as convergence, solve target sizes adaptive problem and be updated in line filter; (5), as target search does not restrain, repeating step (1), (2), (3) and (4), until target search convergence solves target sizes adaptive problem and is updated in line filter, export target position and size; Detailed process as shown in Figure 3.
Claims (2)
1. a robot face detects and follows the tracks of emotion detection system, it is characterized in that: comprise that face detects emotion recognition system and face tracking system, both are all connected with host computer system, face detects emotion recognition system and comprises that collection face expression database, expressive features storehouse build module, feature database reconstructed module and Expression Recognition module, and face tracking system is connected with digital rudder controller control system.
2. robot according to claim 1 face detects and follows the tracks of emotion detection system, it is characterized in that: the method for work of face tracking system comprises four steps: step 1, on current goal position, set up online grader; Step 2, grader traversal search in candidate region that next frame utilizes previous step to set up, generates confidence map; Step 3. in confidence map, search for the reposition of maximum as target place; Step 4, at maximum position, place chooses sample, utilizes the new samples obtaining to upgrade grader, utilizes grader repeating step two and the step 3 after upgrading in next frame.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410606320.1A CN105590084A (en) | 2014-11-03 | 2014-11-03 | Robot human face detection tracking emotion detection system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410606320.1A CN105590084A (en) | 2014-11-03 | 2014-11-03 | Robot human face detection tracking emotion detection system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105590084A true CN105590084A (en) | 2016-05-18 |
Family
ID=55929656
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410606320.1A Pending CN105590084A (en) | 2014-11-03 | 2014-11-03 | Robot human face detection tracking emotion detection system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105590084A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106346475A (en) * | 2016-11-01 | 2017-01-25 | 上海木爷机器人技术有限公司 | Robot and robot control method |
| CN106355242A (en) * | 2016-09-26 | 2017-01-25 | 苏州小璐机器人有限公司 | Interactive robot on basis of human face detection |
| CN106426180A (en) * | 2016-11-24 | 2017-02-22 | 深圳市旗瀚云技术有限公司 | Robot capable of carrying out intelligent following based on face tracking |
| CN107817793A (en) * | 2017-09-28 | 2018-03-20 | 金子旭 | Forward type tracks robot automatically |
| CN111382608A (en) * | 2018-12-28 | 2020-07-07 | 广州盈可视电子科技有限公司 | Intelligent detection system with emotion recognition function |
| CN111428666A (en) * | 2020-03-31 | 2020-07-17 | 齐鲁工业大学 | Intelligent family accompanying robot system and method based on rapid face detection |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101540890A (en) * | 2009-04-28 | 2009-09-23 | 南京航空航天大学 | Method for obtaining a clear face image of a moving human body in a monitored video |
| US20130078600A1 (en) * | 2011-08-29 | 2013-03-28 | Worcester Polytechnic Institute | System and method of pervasive developmental disorder interventions |
| CN103679203A (en) * | 2013-12-18 | 2014-03-26 | 江苏久祥汽车电器集团有限公司 | Robot system and method for detecting human face and recognizing emotion |
-
2014
- 2014-11-03 CN CN201410606320.1A patent/CN105590084A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101540890A (en) * | 2009-04-28 | 2009-09-23 | 南京航空航天大学 | Method for obtaining a clear face image of a moving human body in a monitored video |
| US20130078600A1 (en) * | 2011-08-29 | 2013-03-28 | Worcester Polytechnic Institute | System and method of pervasive developmental disorder interventions |
| CN103679203A (en) * | 2013-12-18 | 2014-03-26 | 江苏久祥汽车电器集团有限公司 | Robot system and method for detecting human face and recognizing emotion |
Non-Patent Citations (1)
| Title |
|---|
| 朱建清: ""智能视频监控系统中的人脸检测和人脸跟踪技术的研究"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106355242A (en) * | 2016-09-26 | 2017-01-25 | 苏州小璐机器人有限公司 | Interactive robot on basis of human face detection |
| CN106346475A (en) * | 2016-11-01 | 2017-01-25 | 上海木爷机器人技术有限公司 | Robot and robot control method |
| CN106426180A (en) * | 2016-11-24 | 2017-02-22 | 深圳市旗瀚云技术有限公司 | Robot capable of carrying out intelligent following based on face tracking |
| CN107817793A (en) * | 2017-09-28 | 2018-03-20 | 金子旭 | Forward type tracks robot automatically |
| CN111382608A (en) * | 2018-12-28 | 2020-07-07 | 广州盈可视电子科技有限公司 | Intelligent detection system with emotion recognition function |
| CN111428666A (en) * | 2020-03-31 | 2020-07-17 | 齐鲁工业大学 | Intelligent family accompanying robot system and method based on rapid face detection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105590084A (en) | Robot human face detection tracking emotion detection system | |
| CN103679203B (en) | Robot system and method for detecting human face and recognizing emotion | |
| CN104635574B (en) | A kind of early education towards child is accompanied and attended to robot system | |
| Zhang et al. | A novel approach to driving fatigue detection using forehead EOG | |
| CN103955215B (en) | Automatic obstacle avoidance trolley based on gesture identification and control device and control method | |
| CN204322085U (en) | A kind of early education towards child is accompanied and attended to robot | |
| CN105719431A (en) | Fatigue driving detection system | |
| CN202995348U (en) | An intelligent wheelchair controlling system | |
| CN109623835B (en) | Wheelchair manipulator system based on multi-mode information fusion | |
| CN102831380A (en) | Body action identification method and system based on depth image induction | |
| CN109658503B (en) | Driver behavior intention detection method fused with electroencephalogram signals | |
| CN109009887A (en) | A kind of man-machine interactive navigation system and method based on brain-computer interface | |
| CN107301406A (en) | Fast face angle recognition method based on deep learning | |
| CN106074021A (en) | Intelligent wheelchair system based on brain-computer interface and method of operating thereof | |
| CN108682119A (en) | Method for detecting fatigue state of driver based on smart mobile phone and smartwatch | |
| CN103198616A (en) | Method and system for detecting fatigue driving based on head and neck movement feature recognition of driver | |
| CN107544675A (en) | Brain control formula virtual reality method | |
| CN104524772A (en) | Roller coaster passenger state real-time detection method | |
| CN112631173A (en) | Brain-controlled unmanned platform cooperative control system | |
| CN110364254A (en) | A kind of automated external defibrillator intelligent assistance system and method | |
| WO2022139943A3 (en) | Machine learning for ear disease diagnosis assistance | |
| CN106445165A (en) | Head-mounted type controlling system based on electroencephalogram eeg analytic technique | |
| CN112364817A (en) | AI control system based on facial expression psychophysiological parameters | |
| CN104850735A (en) | Activity recognition method based on stack own coding | |
| CN110108510A (en) | Based on embedded system automobile electronics intelligent checking system and its method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20170503 Address after: 150000 Heilongjiang province science and technology innovation of Harbin city high tech Industrial Development Zone, a new show on Street Plaza building 20, room 178 No. A205 Applicant after: Heilongjiang best technology Co., Ltd. Address before: 550018 678 building, Jinyang hi tech Industrial Park, Guiyang hi tech Zone, Guizhou Applicant before: GUIZHOU YFSH TECHNOLOGY ROBOT CO., LTD. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160518 |
|
| RJ01 | Rejection of invention patent application after publication |