CN101510121A - Interface roaming operation method and apparatus based on gesture identification - Google Patents
Interface roaming operation method and apparatus based on gesture identification Download PDFInfo
- Publication number
- CN101510121A CN101510121A CNA2009101033571A CN200910103357A CN101510121A CN 101510121 A CN101510121 A CN 101510121A CN A2009101033571 A CNA2009101033571 A CN A2009101033571A CN 200910103357 A CN200910103357 A CN 200910103357A CN 101510121 A CN101510121 A CN 101510121A
- Authority
- CN
- China
- Prior art keywords
- chip
- gesture identification
- template
- user
- gesture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- User Interface Of Digital Computer (AREA)
- Position Input By Displaying (AREA)
Abstract
The invention provides an interface roaming operation method based on the gesture recognition and a device thereof; the device consists of a camera, an infrared diode, a CPU, a north bridge controller, a south bridge controller, a display chip, a display bridging chip, an IO chip, a BIOS chip, a programmable or a dedicated gesture recognition chip, and a bearing circuit board, wherein the camera is connected with the south bridge controller or the IO chip, or the gesture recognition chip; after the gesture image of the user is shot, the information is sent out by a USB bus or an IEEE1394 bus or an SPI bus; and then processing is carried out by a gesture recognition software program which is implemented by the CPU, or the double-click command is recognized by the gesture recognition chip for triggering corresponding interface roaming operating instructions.
Description
Technical field
The present invention relates to a kind of interface roaming operation method and device, be specifically related to a kind of motion of hand and attitude utilized and replace mouse to carry out the man-machine interaction method and the structure of roam operation based on gesture identification.
Background technology
Traditional man-machine interaction of computing machine is based on mouse-keyboard, and mouse mainly is used in figure man-machine interface mutual, has fast and accurate mutual effect.But the various pointing devices that comprise touchpad, trackpoint, trackball, touch-screen all need to hold or push a certain mouse apparatus, and it is moved on a certain plane or curved surface, define the freedom of man-machine interaction, its on-the-spot demonstration that can not well be applicable to large format (projector etc.) propaganda is used.In addition, move under the conditions such as (walking about), car and boat vibration environment, the interactive efficiency of all kinds of conventional mouse roaming operation methods also is subjected to severely restricts.In recent years, computer vision technique has developed to such an extent that reach its maturity, under this background, comes " explanation " people's gesture motion by computer vision, and then the various actions in triggering alternately also become possibility.
The technical solution adopted in the present invention is: propose a kind of interface roaming operation method and device based on gesture identification, by camera, infrared diode, CPU, the north bridge controller, the south bridge controller, display chip, show bridging chip, the IO chip, the BIOS chip, gesture identification chip able to programme or special-purpose gesture identification chip, and the bearer circuit plate constitutes, it is characterized in that: camera links to each other with south bridge controller or IO chip, or camera links to each other with gesture identification chip able to programme or special-purpose gesture identification chip, and this chip links to each other by PS/2 or usb bus and south bridge controller or IO chip or north and south bridge integrated chip or system's SOC chip.After photographing user's images of gestures, send this information by usb bus or IEEE1394 bus or spi bus, the gesture recognition software program of carrying out via CPU is handled then, the interface roaming operation in the discriminating user gesture interaction.Or after by the gesture identification chip gesture video flowing being resolved, instruction is double-clicked in identification, and by the PS/2 or the usb bus interface of chip internal the mouse event instruction is sent.
Described camera can be made a video recording at infrared band, can cooperate the infrared diode light source works, described infrared diode is an option means, the configuration of infrared diode can be on the graphoscope frame, on the keyboard plane, on the head-mounted display, in order to user's hand is carried out infrared radiation, avoid of the interference of extraneous visible light, to improve discrimination to camera.
Described gesture recognition software program, gesture identification chip able to programme or special-purpose gesture identification chip are by low tomographic image pretreatment module, the middle level is cut apart and tracking module, high-rise template matches module constitutes, the low tomographic image pretreatment module of image input, this module possesses linear filtering and edge detection feature, the middle level cut apart with tracking module in order to video flowing is divided into single image or the pixel segmentation with target signature in single image is come out, and the coordinate of these character pixels followed the tracks of, high-rise template matches module adopts the hidden Markov template that character pixel is carried out the implied meaning coupling, and output obtains the implication of gesture interface roaming operation.
Described gesture interface roaming action type is characterized by definite action of the left hand or the right hand, this action is mated by default template or User-defined template, when template is preset in use, user's hand motion need conform to the default hand exercise model in the template and can carry out the behavior sign to roam operation, and User-defined template is obtained by user's training.
The training process of described User-defined template is, the user makes a video recording by the hand exercise process that this device will set in the time, video flowing to gained hangs down the tomographic image pre-service, the middle level is cut apart and tracking, high-rise template matches, obtain a characteristic point data at last and gather the matching relationship that characterizes hand motion and roam operation, training process need repeat repeatedly.In the double click operation process of hand, the user can expand existing preset matching template and self-defined matching template, after catching the operation by human hand action, the gesture identification chip mates it and preset matching template and self-defined matching template, if can't mate then offer the implied meaning of user option, and it is added in matching template with this action representative of artificial selection.
Described roam operation comprises that the location of mouse, left button are clicked, left button double-click, right-click, roller, drag operation, the two dimension rotation of man-machine interface, three-dimensional rotation, amplifies, dwindles.
The invention has the beneficial effects as follows: a kind of interface roaming operation method and the device that are proposed based on gesture identification, can allow the gesture motion of people according to nature, need not the touch mouse device and control it and carry out plane or non-plane motion, can carry out the roam operation of similar conventional mouse.
Beneficial effect of the present invention also comprises: a kind of interface roaming operation method and the device based on gesture identification that are proposed, its described User Defined matching template training and extended method, more traditional fixed form has greater flexibility, can better satisfy individual habit of user, fitness is higher.
Beneficial effect of the present invention also comprises: a kind of interface roaming operation method and the device based on gesture identification that are proposed, the roam operation that it can be realized, comprise not only that location, the left button of conventional mouse on graphical interface of user clicked, left button double-click, right-click, roller, drag operation, can also be single gesture realize man-machine interface two dimension rotation, three-dimensional rotation, operational motion such as amplify, dwindle, compared to traditional mouse-operated, more convenient quick.
Description of drawings
Fig. 1 is a software implementation structure synoptic diagram of the present invention.
Fig. 2 is a hardware implementation structure synoptic diagram of the present invention.
Fig. 3 is a self-defined template training process flow diagram of the present invention.
Embodiment
Below in conjunction with accompanying drawing the present invention is further described.
Fig. 1 is a software implementation structure synoptic diagram of the present invention.Comprise CPU (1), north bridge controller (2), graphic presentation chip (3), figure bridging chip (4), south bridge controller (5), IO chip (6), System on Chip/SoC (7), camera (8), infrared LED (9).Its inner interconnected relationship is: CPU (1) links to each other with north bridge controller (2), north bridge controller (2) links to each other with south bridge controller (5), south bridge controller (5) links to each other with IO chip (6), and the USB of south bridge controller (5) or System on Chip/SoC (7) or IEEE1394 or spi bus link to each other with camera (8).
Fig. 2 is a hardware implementation structure synoptic diagram of the present invention.Except that the described content of Fig. 1, also has gesture identification chip (11), the gesture identification chip passes through USB or IEEE1394 or spi bus and links to each other with south bridge controller or System on Chip/SoC (7), and the gesture identification chip also passes through USB or IEEE1394 or spi bus or user definition IO bus and links to each other with camera.
Fig. 3 is a self-defined template training process flow diagram of the present invention.The training process of the User-defined template of its description is, the user carries out video camera (12) by this device in a certain Preset Time, video flowing to gained hangs down tomographic image pre-service (13), the middle level is cut apart and is followed the tracks of (14), high-rise template matches (15), high-rise template comprises default template (16) and User-defined template (17), if cutting apart with the characteristic point data set of gained as a result of tracking (14) back, the middle level can mate with default template (16) or User-defined template (17), then the match is successful triggers roaming event, otherwise carry out Matching Model expansion (19), promptly offer the implied meaning of user option, and it is added in the matching template with this action representative of artificial selection.
Claims (8)
1. interface roaming operation method and device based on a gesture identification, by camera, infrared diode, CPU, north bridge controller, south bridge controller, display chip, demonstration bridging chip, IO chip, BIOS chip, and the bearer circuit plate constitutes, it is characterized in that: camera links to each other with south bridge controller or IO chip, after photographing user's images of gestures, send this information by usb bus or IEEE1394 bus or spi bus, the gesture recognition software program of carrying out via CPU is handled then, the interface roaming operation in the discriminating user gesture interaction.
2. a kind of interface roaming operation method and device according to claim 1 based on gesture identification, also can be by camera, infrared diode, CPU, the north bridge controller, the south bridge controller, display chip, bridging chip, the IO chip, the BIOS chip, gesture identification chip able to programme or special-purpose gesture identification chip, and the bearer circuit plate constitutes, it is characterized in that: camera links to each other with gesture identification chip able to programme or special-purpose gesture identification chip, this chip links to each other by PS/2 or usb bus and south bridge controller or IO chip or north and south bridge integrated chip or system's SOC chip, after camera photographs user's images of gestures, send this image information by usb bus or IEEE1394 bus or spi bus or self-defined IO data line to gesture identification chip able to programme or special-purpose gesture identification chip, after chip resolves the gesture video flowing thus, instruction is double-clicked in identification, and by the PS/2 or the usb bus interface of chip internal the mouse event instruction is sent.
3. a kind of interface roaming operation method and device according to claim 1 based on gesture identification, it is characterized in that: described camera can be made a video recording at infrared band, can cooperate the infrared diode light source works, described infrared diode is an option means, the configuration of infrared diode can be on the graphoscope frame, on the keyboard plane, on the head-mounted display, in order to user's hand is carried out infrared radiation, avoid of the interference of extraneous visible light, to improve discrimination to camera.
4. a kind of interface roaming operation method and device according to claim 1 based on gesture identification, it is characterized in that: described gesture recognition software program, gesture identification chip able to programme or special-purpose gesture identification chip are by low tomographic image pretreatment module, the middle level is cut apart and tracking module, high-rise template matches module constitutes, the low tomographic image pretreatment module of image input, this module possesses linear filtering and edge detection feature, the middle level cut apart with tracking module in order to video flowing is divided into single image or the pixel segmentation with target signature in single image is come out, and the coordinate of these character pixels followed the tracks of, high-rise template matches module adopts the hidden Markov template that character pixel is carried out the implied meaning coupling, and output obtains the implication of gesture interface roaming operation.
5. a kind of interface roaming operation method and device according to claim 1 based on gesture identification, it is characterized in that: gesture interface roaming action type is characterized by definite action of the left hand or the right hand, this action is mated by default template or User-defined template, when template is preset in use, user's hand motion need conform to the default hand exercise model in the template and can carry out the behavior sign to roam operation, and User-defined template is obtained by user's training.
6. a kind of interface roaming operation method and device according to claim 5 based on gesture identification, it is characterized in that: the training process of described User-defined template is, the user makes a video recording by the hand exercise process that this device will set in the time, low tomographic image pre-service during the video flowing of gained carry out claim 4, the middle level is cut apart and tracking, high-rise template matches, obtain a characteristic point data at last and gather the matching relationship that characterizes hand motion and roam operation, training process need repeat repeatedly.
7. a kind of interface roaming operation method and device according to claim 5 based on gesture identification, it is characterized in that: in the double click operation process of hand, the user can expand existing preset matching template and self-defined matching template, after catching the operation by human hand action, the gesture identification chip mates it and preset matching template and self-defined matching template, if can't mate then offer the implied meaning of user option, and it is added in matching template with this action representative of artificial selection.
8. a kind of interface roaming operation method and device according to claim 5 based on gesture identification, it is characterized in that: described roam operation comprises that the location of mouse, left button are clicked, left button double-click, right-click, roller, drag operation, the two dimension rotation of man-machine interface, three-dimensional rotation, amplifies, dwindles.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2009101033571A CN101510121A (en) | 2009-03-12 | 2009-03-12 | Interface roaming operation method and apparatus based on gesture identification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2009101033571A CN101510121A (en) | 2009-03-12 | 2009-03-12 | Interface roaming operation method and apparatus based on gesture identification |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101510121A true CN101510121A (en) | 2009-08-19 |
Family
ID=41002533
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNA2009101033571A Pending CN101510121A (en) | 2009-03-12 | 2009-03-12 | Interface roaming operation method and apparatus based on gesture identification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101510121A (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102200831A (en) * | 2010-03-25 | 2011-09-28 | 索尼公司 | Gesture input device, gesture input method and program |
| CN102253573A (en) * | 2010-05-18 | 2011-11-23 | 鸿富锦精密工业（深圳）有限公司 | Front projection device and front projection control method thereof |
| CN102323855A (en) * | 2010-04-30 | 2012-01-18 | 微软公司 | The connector that reshapes with stiffness variable |
| CN102662462A (en) * | 2012-03-12 | 2012-09-12 | 中兴通讯股份有限公司 | Electronic device, gesture recognition method and gesture application method |
| CN102662471A (en) * | 2012-04-09 | 2012-09-12 | 沈阳航空航天大学 | Computer vision mouse |
| CN102860837A (en) * | 2011-07-08 | 2013-01-09 | 株式会社东芝 | Image processing system, image processing device, image processing method, and medical image diagnostic device |
| CN103514437A (en) * | 2012-12-24 | 2014-01-15 | Tcl集团股份有限公司 | Three-dimensional hand gesture recognition device and three-dimensional hand gesture recognition method |
| CN103777753A (en) * | 2013-11-12 | 2014-05-07 | 广州新节奏智能科技有限公司 | Novel portable body sensing and controlling device and application method thereof |
| CN104508601A (en) * | 2012-05-30 | 2015-04-08 | 寇平公司 | Head-worn computer with improved virtual display function |
| CN101661329B (en) * | 2009-09-22 | 2015-06-03 | 北京中星微电子有限公司 | Operating control method and device of intelligent terminal |
| CN104820472A (en) * | 2015-05-13 | 2015-08-05 | 瑞声声学科技(深圳)有限公司 | A mobile terminal |
| CN105407262A (en) * | 2014-09-16 | 2016-03-16 | 洪永川 | Camera |
| CN105404387A (en) * | 2014-09-16 | 2016-03-16 | 洪永川 | Gesture remote control system for intelligent terminal |
| CN105487643A (en) * | 2014-09-16 | 2016-04-13 | 洪永川 | Interactive intelligent terminal system |
| CN108616712A (en) * | 2018-04-18 | 2018-10-02 | 深圳中电数码显示有限公司 | A kind of interface operation method, device, equipment and storage medium based on camera |
| WO2020019730A1 (en) * | 2018-07-27 | 2020-01-30 | 深圳市志海和科技有限公司 | Keyboard |
| CN111461059A (en) * | 2020-04-21 | 2020-07-28 | 哈尔滨拓博科技有限公司 | Multi-zone multi-classification extensible gesture recognition control device and control method |
| CN114885140A (en) * | 2022-05-25 | 2022-08-09 | 华中科技大学 | Multi-screen splicing immersion type projection picture processing method and system |
-
2009
- 2009-03-12 CN CNA2009101033571A patent/CN101510121A/en active Pending
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101661329B (en) * | 2009-09-22 | 2015-06-03 | 北京中星微电子有限公司 | Operating control method and device of intelligent terminal |
| CN102200831A (en) * | 2010-03-25 | 2011-09-28 | 索尼公司 | Gesture input device, gesture input method and program |
| US9539510B2 (en) | 2010-04-30 | 2017-01-10 | Microsoft Technology Licensing, Llc | Reshapable connector with variable rigidity |
| CN102323855A (en) * | 2010-04-30 | 2012-01-18 | 微软公司 | The connector that reshapes with stiffness variable |
| CN102323855B (en) * | 2010-04-30 | 2015-11-25 | 微软技术许可有限责任公司 | There is the connector reshaped of stiffness variable |
| CN102253573A (en) * | 2010-05-18 | 2011-11-23 | 鸿富锦精密工业（深圳）有限公司 | Front projection device and front projection control method thereof |
| CN102860837A (en) * | 2011-07-08 | 2013-01-09 | 株式会社东芝 | Image processing system, image processing device, image processing method, and medical image diagnostic device |
| CN102860837B (en) * | 2011-07-08 | 2016-08-10 | 东芝医疗系统株式会社 | Image processing system, device, method and medical diagnostic imaging apparatus |
| CN102662462A (en) * | 2012-03-12 | 2012-09-12 | 中兴通讯股份有限公司 | Electronic device, gesture recognition method and gesture application method |
| CN102662462B (en) * | 2012-03-12 | 2016-03-30 | 中兴通讯股份有限公司 | Electronic installation, gesture identification method and gesture application process |
| CN102662471B (en) * | 2012-04-09 | 2015-02-18 | 沈阳航空航天大学 | Computer vision mouse |
| CN102662471A (en) * | 2012-04-09 | 2012-09-12 | 沈阳航空航天大学 | Computer vision mouse |
| CN104508601A (en) * | 2012-05-30 | 2015-04-08 | 寇平公司 | Head-worn computer with improved virtual display function |
| CN103514437A (en) * | 2012-12-24 | 2014-01-15 | Tcl集团股份有限公司 | Three-dimensional hand gesture recognition device and three-dimensional hand gesture recognition method |
| CN103777753A (en) * | 2013-11-12 | 2014-05-07 | 广州新节奏智能科技有限公司 | Novel portable body sensing and controlling device and application method thereof |
| CN105487643B (en) * | 2014-09-16 | 2019-05-07 | 深圳市冠凯科技有限公司 | Interactive intelligent terminal system |
| CN105487643A (en) * | 2014-09-16 | 2016-04-13 | 洪永川 | Interactive intelligent terminal system |
| CN105404387A (en) * | 2014-09-16 | 2016-03-16 | 洪永川 | Gesture remote control system for intelligent terminal |
| CN105407262A (en) * | 2014-09-16 | 2016-03-16 | 洪永川 | Camera |
| CN104820472A (en) * | 2015-05-13 | 2015-08-05 | 瑞声声学科技(深圳)有限公司 | A mobile terminal |
| CN108616712A (en) * | 2018-04-18 | 2018-10-02 | 深圳中电数码显示有限公司 | A kind of interface operation method, device, equipment and storage medium based on camera |
| CN108616712B (en) * | 2018-04-18 | 2020-11-10 | 深圳中电数码显示有限公司 | Camera-based interface operation method, device, equipment and storage medium |
| WO2020019730A1 (en) * | 2018-07-27 | 2020-01-30 | 深圳市志海和科技有限公司 | Keyboard |
| CN111461059A (en) * | 2020-04-21 | 2020-07-28 | 哈尔滨拓博科技有限公司 | Multi-zone multi-classification extensible gesture recognition control device and control method |
| CN114885140A (en) * | 2022-05-25 | 2022-08-09 | 华中科技大学 | Multi-screen splicing immersion type projection picture processing method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101510121A (en) | Interface roaming operation method and apparatus based on gesture identification | |
| US10394334B2 (en) | Gesture-based control system | |
| CN102854983B (en) | A kind of man-machine interaction method based on gesture identification | |
| Sugano et al. | Aggregaze: Collective estimation of audience attention on public displays | |
| WO2020078017A1 (en) | Method and apparatus for recognizing handwriting in air, and device and computer-readable storage medium | |
| US20180173313A1 (en) | Detecting, representing, and interpreting three-space input: gestural continuum subsuming freespace, proximal, and surface-contact modes | |
| CN108845668B (en) | Man-machine interaction system and method | |
| Zhu et al. | Real-time hand gesture recognition with Kinect for playing racing video games | |
| Yin et al. | Toward natural interaction in the real world: Real-time gesture recognition | |
| CN109145802B (en) | Kinect-based multi-person gesture man-machine interaction method and device | |
| CN107066081B (en) | Interactive control method and device of virtual reality system and virtual reality equipment | |
| Geer | Will gesture recognition technology point the way? | |
| JP2014029656A (en) | Image processor and image processing method | |
| Sharma et al. | Air-swipe gesture recognition using OpenCV in Android devices | |
| CN109189219A (en) | The implementation method of contactless virtual mouse based on gesture identification | |
| CN104714650B (en) | A kind of data inputting method and device | |
| CN107390881A (en) | A kind of gestural control method | |
| JP2016099643A (en) | Image processing device, image processing method, and image processing program | |
| CN106468944A (en) | Non-contact control system and method | |
| Lew et al. | Human-computer intelligent interaction: a survey | |
| Jain et al. | Gestarlite: An on-device pointing finger based gestural interface for smartphones and video see-through head-mounts | |
| Annachhatre et al. | Virtual Mouse Using Hand Gesture Recognition-A Systematic Literature Review | |
| CN203070205U (en) | Input equipment based on gesture recognition | |
| CN111860086A (en) | Gesture recognition method, device and system based on deep neural network | |
| Raees et al. | Thumb inclination-based manipulation and exploration, a machine learning based interaction technique for virtual environments |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Open date: 20090819 |