CN102176191A - Television control method based on sight-tracking - Google Patents
Television control method based on sight-tracking Download PDFInfo
- Publication number
- CN102176191A CN102176191A CN2011100703785A CN201110070378A CN102176191A CN 102176191 A CN102176191 A CN 102176191A CN 2011100703785 A CN2011100703785 A CN 2011100703785A CN 201110070378 A CN201110070378 A CN 201110070378A CN 102176191 A CN102176191 A CN 102176191A
- Authority
- CN
- China
- Prior art keywords
- televisor
- beholder
- control
- eye tracking
- people
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- User Interface Of Digital Computer (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a television (TV) control method based on sight-tracking, which firstly installs two cameras which can clearly shoot human eyes pupil, and sets a TV sight control triggering button on the TV to start the control of the sight-tracking, and specifically comprises the following steps: (1) locking and tracking human faces of all viewers, detecting the human faces in real time in images of the TV viewers shot by the cameras, locking and tracking all detected human faces; (2) estimating fixation points of all viewers, respectively locating the human eyes in each tracked human face region to realize the fixation point estimation of multiple users; and (3) using the sight to realize the TV control, determining a TV control button selected by the viewer according to an estimated fixation point position of the viewer, thereby, enabling the TV to perform the corresponding action to realize the intelligent control to the TV. People can naturally, conveniently and rapidly control the TV by using the TV control method.
Description
Technical field
The present invention relates to a kind of televisor control method, belong to televisor and control technical field based on eye tracking.
Background technology
Along with the develop rapidly of science and technology, intelligent control technology becomes the research direction that receives much concern in recent years as high-tech a kind of sign.Variously arise at the historic moment based on emerging intelligent control technologies such as input of eye tracking, gesture or speech recognitions.Wherein based on the eye tracking intelligent control technology, because it is developed by leaps and bounds in the intelligent control technology field naturally, rapidly, easily and effectively.Its range of application is also also progressively come into daily life from initial psychology of vision research, the auxiliary fields such as commercial advertisement test and appraisal, robot control of developing into of disabled person.
Televisor all is an amusement article indispensable in people's daily life all the time, development along with the digital to television machine, television function from strength to strength, its complexity of controlling is also improving gradually, and traditional infrared remote control more and more can not satisfy the requirement of people for TV control.And existing tangible gap aspect the diversity of the simplicity of gesture and TV control based on the control mode of gesture; Simultaneously, the difficulty that then realizes of voice-based control mode in the technology that also exists aspect the determinacy of the expression diversity of same semanteme and steering order.Therefore, realize a kind of natural, accurate, convenient and swift and have interesting intelligent TV machine control method and just can make people can realize the brand-new enjoyment that high-tech product brings to us better.
But eye tracking technology substituting for key and mouse are realized the operation control to computing machine, and same, eye tracking also can be used for controlling of televisor.Yet, because the people far and generally has a plurality of people apart from televisor and watches simultaneously when watching TV programme, therefore, than the application of eye tracking in computer control, adopt eye tracking realize to controlling of televisor be a kind of under middle distance (about 3 meters) environment the man-machine control technology applicable to the multi-user, all different in computer control with eye tracking to hardware, algorithm and control operation mode, being controlled at of this middle distance has more important Research Significance in the real world applications.
Summary of the invention
Control the deficiency of technology at having televisor now, the invention provides a kind of can the realization natural, the convenient, fast and intelligent televisor control method of controlling of televisor based on eye tracking.
Televisor control method based on eye tracking of the present invention, the video camera that can clearly take the human eye pupil at first is housed on televisor, so that realize the accurate estimation of fixation point, and a televisor sight line is set on televisor controls trigger button and start controlling of eye tracking; Specifically may further comprise the steps:
(1) all beholder people's face locking trackings carry out in the TV viewer's of shot by camera image in real time that people's face detects, and locking and follow the tracks of all detected people's faces;
The specific implementation step of above-mentioned steps (1) is:
Start video camera when the A televisor is opened, take the scene that the beholder sees TV in real time;
B carries out people's face and detects in captured scene, lock-in detection to each human face region and it is carried out real-time follow-up;
C has new beholder to enter when detecting by people's face in the scene, then lock people's face of this beholder and it is followed the tracks of;
(2) all beholder's fixation points are estimated, carry out the human eye location respectively in each human face region of being followed the tracks of, realize that multi-user's fixation point is estimated;
The specific implementation step of above-mentioned steps (2) is:
A as structure, gray scale, shape etc., orients each TV viewer's position of human eye according to the human eye characteristic from the human face region of following the tracks of;
B analyzes each TV viewer's human eye pupil feature, estimates all beholders' fixation point position in real time;
(3) utilize sight line realization televisor to control, according to the beholder's who estimates fixation point position, determine the TV control knob that the beholder selects, thereby make televisor carry out corresponding action, realize the intellectuality of televisor is controlled;
The specific implementation step of above-mentioned steps (3) is:
The A beholder controls trigger button by sight line trigger television eye tracking, and Presentation Function is controlled button on television screen simultaneously;
B stares the correlation function button on the screen, realizes TV channel switching, volume adjusting or picture color adjusting etc. are controlled;
After C controlled and finishes, all function controlling button on the screen-concealing was controlled trigger button up to the televisor eye tracking and is triggered once more;
When D has a plurality of observers to watch program simultaneously, only allow a beholder TV to be controlled by sight line at every turn; Have only when previous beholder control finish after, could respond next beholder controls trigger button to the televisor eye tracking triggering;
During F closing television machine, video camera stops to take simultaneously.
The present invention is used for eye tracking the control of televisor, under the many observers' of middle distance complex situations, carry out fixation point location accurately, and the operations such as channel switch, picture adjustment or volume adjustment of TV are controlled in the position that utilizes fixation point, the application of this method makes people carry out nature, to control quickly and easily televisor, enjoys for people's life provides the intelligentized enjoyment of high-tech.
Description of drawings
Fig. 1 televisor hardware setting synoptic diagram.
The screen interface that Fig. 2 sight line is controlled in the process shows.
Fig. 3 televisor control method of the present invention process flow diagram.
Wherein: 1, video camera, 2, video camera, 3, the televisor switch, 4, the televisor eye tracking controls trigger button.
Embodiment
The present invention is based on the televisor control method of eye tracking, be applied to the scene that indoor middle distance is observed the user more.With as shown in Figure 1 system is that example illustrates.Two video cameras 1 and 2 that can clearly take the human eye pupil at first are housed on televisor,, and a televisor sight line is set on televisor controls trigger button 4 and start controlling of eye tracking so that realize the accurate estimation of fixation point.These two video cameras should be able to (be generally 2 to 4 meters) the human eye pupil is implemented clear shooting about 3 meters.
Being that example is introduced method of the present invention to controlling of multifunctional digital TV.
When televisor switch 3 is opened, start video camera 1 and 2, take the scene that the user watches televisor in real time.In captured scene, carry out people's face according to face characteristic and detect, lock-in detection to everyone face and it is followed the tracks of.In detecting scene, there is new beholder to add, detects by people's face immediately, lock this user's human face region and it is included within the tracking target scope.
In each human face region of being followed the tracks of,, orient the position of human eye that each televisor is watched the user, and estimate human eye pupil parameter according to characteristics such as the structure of human eye, shape, gray scales.According to coordinate position and the feature such as shape and structure of location human eye pupil on space and image, estimate all in real time and watch the current fixation point position of user.
When the beholder need carry out televisor when controlling, the beholder utilizes sight line trigger television eye tracking to control trigger button 4, Presentation Function is controlled button on television screen, as master menu selection keys such as video request program, TV programme, music, picture browsings, shown in Fig. 2 (a).The beholder selects corresponding operation by staring different buttons.
When the volume of picture color that needs to regulate TV in the process of watching or video or converted channel, the trigger television eye tracking is controlled trigger button 4, buttons such as master menu, volume, colourity, brightness, saturation degree, channel switching, numeric keypad then can appear, shown in Fig. 2 (b).Stare master menu and can return the interface shown in Fig. 2 (a), stare volume, colourity, brightness, saturation degree or channel switching push button, can select the module that will be provided with, the digital keys of staring again on the numeric keypad just can be set to corresponding module oneself satisfied degree.If the unit numeral is directly stared the corresponding digital key, if dibit numeral or three bit digital then will be stared "--" or "---" earlier and be stared numerical key again.As want volume is transferred to 23, then at first stare " volume " button, stare again "--", and then stare " 2 " key and " 3 " key respectively and can finish regulation and control volume.
" a finishing " button is all arranged in the operation and control interface, as Fig. 2 (a) (b) as shown in.Control finish after, stare " finishing " button on the screen, all function controlling buttons are controlled trigger button up to the televisor eye tracking and are triggered once more on the automatic screen-concealing of system.
Watch program simultaneously as a plurality of beholders, only allow a beholder TV to be controlled at every turn by sight line; Have only when previous beholder control finish after, system just can respond next beholder controls trigger button to the televisor eye tracking triggering.That is to say to have only previous beholder A to stare after " finishing " button, next observer B controls staring of trigger button to the televisor eye tracking and is only an effective trigger signals, and beholder B could control televisor.
During the closing television machine, stop video camera 1 and 2 automatically and take.
Adopt above-mentioned televisor control method based on eye tracking, people needn't depend on telepilot again, and only depend on moving of sight line just can realize controlling to televisor, control than using a teleswitch, this method is more natural, more direct, not only realized manipulation function, also increased enjoyment simultaneously for people watch TV.In addition, the control mode of this man-machine interaction has also been expanded the space of the more amusement functions of exploitation for televisor.
Claims (4)
1. televisor control method based on eye tracking, it is characterized in that, the video camera that can clearly take the human eye pupil at first is housed on televisor, so that realize the accurate estimation of fixation point, and a televisor sight line is set on televisor controls trigger button and start controlling of eye tracking; Specifically may further comprise the steps:
(1) all beholder people's face locking trackings carry out in the TV viewer's of shot by camera image in real time that people's face detects, and locking and follow the tracks of all detected people's faces;
(2) all beholder's fixation points are estimated, carry out the human eye location respectively in each human face region of being followed the tracks of, realize that multi-user's fixation point is estimated;
(3) utilize sight line realization televisor to control, according to the beholder's who estimates fixation point position, determine the TV control knob that the beholder selects, thereby make televisor carry out corresponding action, realize the intellectuality of televisor is controlled.
2. the televisor control method based on eye tracking according to claim 1 is characterized in that, the specific implementation step of described step (1) is:
Start video camera when the A televisor is opened, take the scene that the beholder sees TV in real time;
B carries out people's face and detects in captured scene, lock-in detection to each human face region and it is carried out real-time follow-up;
C has new beholder to enter when detecting by people's face in the scene, then lock people's face of this beholder and it is followed the tracks of.
3. the televisor control method based on eye tracking according to claim 1 is characterized in that, the described specific implementation step of stating step (2) is:
A as structure, gray scale, shape etc., orients each TV viewer's position of human eye according to the human eye characteristic from the human face region of following the tracks of;
B analyzes each TV viewer's human eye pupil feature, estimates all beholders' fixation point position in real time.
4. the televisor control method based on eye tracking according to claim 1 is characterized in that, the described specific implementation step of stating step (3) is:
The A beholder controls trigger button by sight line trigger television eye tracking, and Presentation Function is controlled button on television screen simultaneously;
B stares the correlation function button on the screen, realizes TV channel switching, volume adjusting or picture color adjusting etc. are controlled;
After C controlled and finishes, all function controlling button on the screen-concealing was controlled trigger button up to the televisor eye tracking and is triggered once more;
When D has a plurality of observers to watch program simultaneously, only allow a beholder TV to be controlled by sight line at every turn; Have only when previous beholder control finish after, could respond next beholder controls trigger button to the televisor eye tracking triggering;
During F closing television machine, video camera stops to take simultaneously.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011100703785A CN102176191A (en) | 2011-03-23 | 2011-03-23 | Television control method based on sight-tracking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011100703785A CN102176191A (en) | 2011-03-23 | 2011-03-23 | Television control method based on sight-tracking |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102176191A true CN102176191A (en) | 2011-09-07 |
Family
ID=44519378
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011100703785A Pending CN102176191A (en) | 2011-03-23 | 2011-03-23 | Television control method based on sight-tracking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102176191A (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102693022A (en) * | 2011-12-12 | 2012-09-26 | 苏州科雷芯电子科技有限公司 | Vision tracking and voice identification mouse system |
| CN102915115A (en) * | 2012-09-25 | 2013-02-06 | 上海华勤通讯技术有限公司 | Method for adjusting display frame by eye |
| CN103366381A (en) * | 2013-08-06 | 2013-10-23 | 山东大学 | Sight line tracking correcting method based on space position |
| CN103631364A (en) * | 2012-08-20 | 2014-03-12 | 联想(北京)有限公司 | Control method and electronic device |
| CN103823560A (en) * | 2014-02-24 | 2014-05-28 | 联想(北京)有限公司 | Control method and device |
| CN103853419A (en) * | 2012-12-04 | 2014-06-11 | 联想(北京)有限公司 | Method and electronic equipment for information processing |
| CN104065986A (en) * | 2013-03-18 | 2014-09-24 | 中兴通讯股份有限公司 | Method and device used for controlling television based on eyeball movement |
| CN104182046A (en) * | 2014-08-22 | 2014-12-03 | 京东方科技集团股份有限公司 | Eye control reminding method, eye control image display method and display system |
| CN104679226A (en) * | 2013-11-29 | 2015-06-03 | 上海西门子医疗器械有限公司 | Non-contact medical control system and method and medical device |
| CN104837049A (en) * | 2014-02-06 | 2015-08-12 | 三星电子株式会社 | User terminal apparatus, display apparatus, and control methods thereof |
| CN106527689A (en) * | 2016-10-13 | 2017-03-22 | 广州视源电子科技股份有限公司 | User interface interaction method and system for virtual reality system |
| CN106686467A (en) * | 2017-03-02 | 2017-05-17 | 东莞市志诚电器有限公司 | Lie-flat type audio-video watching intelligent system |
| CN107368191A (en) * | 2012-01-04 | 2017-11-21 | 托比股份公司 | For watching interactive system attentively |
| WO2018027415A1 (en) * | 2016-08-06 | 2018-02-15 | 吕秋萍 | Adjustment method for technology automatically adjusting brightness of television, and television |
| WO2018027418A1 (en) * | 2016-08-06 | 2018-02-15 | 吕秋萍 | Method for deactivating technology for adjusting television set-up based on human face, and television |
| WO2018027414A1 (en) * | 2016-08-06 | 2018-02-15 | 吕秋萍 | Method for deactivating brightness adjustment technology based on market feedback, and television |
| CN108170346A (en) * | 2017-12-25 | 2018-06-15 | 广东欧珀移动通信有限公司 | Electronic device, interface display methods and Related product |
| CN108595009A (en) * | 2012-02-29 | 2018-09-28 | 联想(北京)有限公司 | A kind of man-machine interaction control method and electric terminal |
| CN114585990A (en) * | 2019-11-05 | 2022-06-03 | 美光科技公司 | Presentation enhancement based in part on eye tracking |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1694043A (en) * | 2004-04-29 | 2005-11-09 | 国际商业机器公司 | System and method for selecting and activating a target object using a combination of eye gaze and key presses |
| CN101405680A (en) * | 2006-03-23 | 2009-04-08 | 皇家飞利浦电子股份有限公司 | Hotspots for eye track control of image manipulation |
| CN101866215A (en) * | 2010-04-20 | 2010-10-20 | 复旦大学 | Human-computer interaction device and method adopting eye tracking in video monitoring |
-
2011
- 2011-03-23 CN CN2011100703785A patent/CN102176191A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1694043A (en) * | 2004-04-29 | 2005-11-09 | 国际商业机器公司 | System and method for selecting and activating a target object using a combination of eye gaze and key presses |
| CN101405680A (en) * | 2006-03-23 | 2009-04-08 | 皇家飞利浦电子股份有限公司 | Hotspots for eye track control of image manipulation |
| CN101866215A (en) * | 2010-04-20 | 2010-10-20 | 复旦大学 | Human-computer interaction device and method adopting eye tracking in video monitoring |
Non-Patent Citations (1)
| Title |
|---|
| 胡畔等: "《基于视线跟踪技术的眼控鼠标》", 《天津师范大学学报（自然科学版）》 * |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102693022A (en) * | 2011-12-12 | 2012-09-26 | 苏州科雷芯电子科技有限公司 | Vision tracking and voice identification mouse system |
| CN107368191B (en) * | 2012-01-04 | 2020-09-25 | 托比股份公司 | System for gaze interaction |
| CN107368191A (en) * | 2012-01-04 | 2017-11-21 | 托比股份公司 | For watching interactive system attentively |
| CN108595009A (en) * | 2012-02-29 | 2018-09-28 | 联想(北京)有限公司 | A kind of man-machine interaction control method and electric terminal |
| CN108595009B (en) * | 2012-02-29 | 2020-12-18 | 联想(北京)有限公司 | Man-machine interaction control method and electronic terminal |
| CN103631364A (en) * | 2012-08-20 | 2014-03-12 | 联想(北京)有限公司 | Control method and electronic device |
| CN103631364B (en) * | 2012-08-20 | 2017-06-27 | 联想(北京)有限公司 | A kind of control method and electronic equipment |
| CN102915115A (en) * | 2012-09-25 | 2013-02-06 | 上海华勤通讯技术有限公司 | Method for adjusting display frame by eye |
| CN103853419A (en) * | 2012-12-04 | 2014-06-11 | 联想(北京)有限公司 | Method and electronic equipment for information processing |
| CN103853419B (en) * | 2012-12-04 | 2017-08-29 | 联想(北京)有限公司 | The method and electronic equipment of information processing |
| CN104065986A (en) * | 2013-03-18 | 2014-09-24 | 中兴通讯股份有限公司 | Method and device used for controlling television based on eyeball movement |
| CN103366381A (en) * | 2013-08-06 | 2013-10-23 | 山东大学 | Sight line tracking correcting method based on space position |
| CN104679226A (en) * | 2013-11-29 | 2015-06-03 | 上海西门子医疗器械有限公司 | Non-contact medical control system and method and medical device |
| CN104837049A (en) * | 2014-02-06 | 2015-08-12 | 三星电子株式会社 | User terminal apparatus, display apparatus, and control methods thereof |
| CN103823560B (en) * | 2014-02-24 | 2017-01-04 | 联想(北京)有限公司 | A kind of control method and device |
| CN103823560A (en) * | 2014-02-24 | 2014-05-28 | 联想(北京)有限公司 | Control method and device |
| CN104182046A (en) * | 2014-08-22 | 2014-12-03 | 京东方科技集团股份有限公司 | Eye control reminding method, eye control image display method and display system |
| WO2018027414A1 (en) * | 2016-08-06 | 2018-02-15 | 吕秋萍 | Method for deactivating brightness adjustment technology based on market feedback, and television |
| WO2018027418A1 (en) * | 2016-08-06 | 2018-02-15 | 吕秋萍 | Method for deactivating technology for adjusting television set-up based on human face, and television |
| WO2018027415A1 (en) * | 2016-08-06 | 2018-02-15 | 吕秋萍 | Adjustment method for technology automatically adjusting brightness of television, and television |
| CN106527689A (en) * | 2016-10-13 | 2017-03-22 | 广州视源电子科技股份有限公司 | User interface interaction method and system for virtual reality system |
| CN106686467A (en) * | 2017-03-02 | 2017-05-17 | 东莞市志诚电器有限公司 | Lie-flat type audio-video watching intelligent system |
| CN108170346A (en) * | 2017-12-25 | 2018-06-15 | 广东欧珀移动通信有限公司 | Electronic device, interface display methods and Related product |
| CN114585990A (en) * | 2019-11-05 | 2022-06-03 | 美光科技公司 | Presentation enhancement based in part on eye tracking |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102176191A (en) | Television control method based on sight-tracking | |
| CN107506037B (en) | Method and device for controlling equipment based on augmented reality | |
| CN105763917B (en) | A kind of control method and system of terminal booting | |
| CN106020478B (en) | A kind of intelligent terminal control method, device and intelligent terminal | |
| CN105425964B (en) | A kind of gesture identification method and system | |
| CN107390863B (en) | Device control method and device, electronic device and storage medium | |
| EP3077867B1 (en) | Optical head mounted display, television portal module and methods for controlling graphical user interface | |
| WO2013185714A1 (en) | Method, system, and computer for identifying object in augmented reality | |
| US9971491B2 (en) | Gesture library for natural user input | |
| GB2483168A (en) | Controlling movement of displayed object based on hand movement and size | |
| US8599200B2 (en) | Information processing apparatus, display controlling method, and display controlling apparatus | |
| US11016558B2 (en) | Information processing apparatus, and information processing method to determine a user intention | |
| KR100692526B1 (en) | Gesture recognition apparatus and methods for automatic control of systems | |
| JP2013101242A (en) | Image processor, display control method and program | |
| US20180316911A1 (en) | Information processing apparatus | |
| EP4252099A1 (en) | Single-handed microgesture inputs | |
| Dhule et al. | Computer vision based human-computer interaction using color detection techniques | |
| CN112399239B (en) | Video playing method and device | |
| CN109600555A (en) | A kind of focusing control method, system and photographing device | |
| CN111596760A (en) | Operation control method and device, electronic equipment and readable storage medium | |
| CN111656313A (en) | Screen display switching method, display device and movable platform | |
| Zhang et al. | A strong tracking nonlinear robust filter for eye tracking | |
| CN109240492A (en) | The method for controlling studio packaging and comment system by gesture identification | |
| US20180366089A1 (en) | Head mounted display cooperative display system, system including dispay apparatus and head mounted display, and display apparatus thereof | |
| WO2018083737A1 (en) | Display device and remote operation controller |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20110907 |