CN101038523A - Mouse system based on visual tracking and voice recognition - Google Patents
Mouse system based on visual tracking and voice recognition Download PDFInfo
- Publication number
- CN101038523A CN101038523A CNA2007100399967A CN200710039996A CN101038523A CN 101038523 A CN101038523 A CN 101038523A CN A2007100399967 A CNA2007100399967 A CN A2007100399967A CN 200710039996 A CN200710039996 A CN 200710039996A CN 101038523 A CN101038523 A CN 101038523A
- Authority
- CN
- China
- Prior art keywords
- mouse
- module
- speech recognition
- light spot
- phonetic order
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
A mouse system based on the vision tracking and sound recognition, which belongs to the computer technology field, according to the invention, a pick-up head dynamically tracks the light spot of the far infrared mark on the imaging surface by detection, a light spot positioning sub-module converts the motion on the image into the movement on the desktop of the mouse, and transmits the location information to an information feedback module, then the information feedback module displays the mouse cursor position transmitted from a mouse driving module via a computer display and provides correspondent information prompt to the user, a speech instruction processing module receives the speech instruction from the user via a microphone which is connected with the main frame of the computer, and a speech recognition sub-module recognizes the input speech signal, and the correspondent mouse incident is driven and incited by the classification result. The invention has simple structure, convenient use, wide application and provides a convenient human-machine interface to the computer system
Description
Technical field
The present invention is a kind of mouse system of field of computer technology, specifically is a kind of mouse system based on vision track and speech recognition.
Background technology
In traditional computer system, conventional keyboard, mouse are topmost input-output device, the interactive mode that present most of computer system adopts is: keyboard and mouse-display-keyboard and mouse, be that operating personnel are that input reaches prompting for information about and asks for instructions to computing machine by keyboard, mouse, and computing machine provide great deal of related information and prompting to ask for instructions etc. by display to the people.Present most of computer operating system and application software all are based on this interactive mode exploitation.Though the pattern of this man-machine interaction can satisfy computer user's day to day operation needs, but using traditional mouse-keyboard under many situations is inappropriate as input-output device, for example the computer system of physical disabilities' use just should not also have most of multimedia entertainment equipment also to be inconvenient to use traditional mouse-keyboard as input-output device with traditional mouse-keyboard.The input-output device that many special uses also occurred at present, for example Chinese character hand-written plate, forceball, data glove and 3D mouse etc.But these equipment have following weak point: complex structure, cost an arm and a leg, only be suitable for specialized field and use, versatility is not strong, can't with traditional mouse-keyboard compatibility.
Find by prior art documents, number of patent application is called " the cursor vision point choosing method and the device of form " for the 00134143.X name, this technology has proposed a kind of mouse control method easily, the method that mainly is to use the information Control cursor of mouse relative position on the table that relatively moves of pupil to move, its pupil tracking mainly is a relative shift of determining pupil by the method for posting dynamic tracking.Because the mouse position location is the method for relative positioning, so the location of cursor of mouse is subjected to the influence of noise disturbance bigger, robustness is not strong.In further retrieval, Shang Weijian has the mouse system of reporting the sort of robust based on the vision track technology.
Summary of the invention
The present invention is directed to prior art problems and deficiency, proposed a kind of mouse system based on vision track and speech recognition.The present invention can real-time follow-up detects moving of infraluminescence mark by camera, and with the position of this mouse beacon cursor, receives phonetic order by microphone, thereby is provide convenience close friend's man-machine interface of computer system.
The present invention is achieved by the following technical solutions, the present invention includes: mouse-driven module, information feedback module and phonetic order processing module; The mouse-driven module is made of camera, infraluminescence mark and cursor positioning submodule, camera is by detecting the light spot image of following the tracks of infrared mark, can move the position of dynamic tracking hot spot on the camera imaging plane, be that cursor of mouse showing motion desktop on hot spot in the movement conversion on the image by the cursor positioning submodule then, and the positional information of mouse is passed to the information feedback module; The information feedback module shows by the position of the cursor of mouse that graphoscope transmits the mouse-driven module, and gives the corresponding information indicating of user; The phonetic order processing module is made up of microphone and speech recognition submodule, when the information feedback module shows that cursor of mouse arrives user's interested position, the phonetic order processing module receives the phonetic order that the user sends by the microphone that is connected with main frame, by the speech recognition submodule input speech signal is discerned, excited and drive corresponding mouse event response (left button is clicked, right-click, left button double-click) by classification results.
Cursor positioning submodule in the described mouse-driven module, its input information are camera image, resolution H * W, monitor resolution L * M, infrared mark facula position coordinate (a, b), be output as mouse cursor position (A, B).This cursor positioning submodule can be the motion of cursor of mouse on the demonstration desktop with the conversion of motion of infraluminescence mark, and its transport function is:
Speech recognition submodule in the described phonetic order processing module, it is input as the linear predictor coefficient (LPCC) that microphone receives voice signal, be output as the Classification and Identification result of phonetic order, classification results can drive corresponding operating-system resources and realize corresponding mouse event response (left button is clicked, right-click, left button double-click).
Compared with prior art, system architecture of the present invention is simple, realize that easily antinoise is disturbed, strong robustness, and the mouse positioning error is less than 2 pixels, and the discrimination of phonetic order is then up to 90%.Compared with prior art, it is convenient, cheap to use equipment and material to buy; System of the present invention highly versatile easy to use not only can use separately, can also cooperate traditional mouse-keyboard to use together.
Description of drawings
Fig. 1 forms synoptic diagram for the present invention.
Fig. 2 is a system flow block diagram of the present invention.
Embodiment
Below in conjunction with accompanying drawing embodiments of the invention are elaborated: present embodiment has provided detailed embodiment and process being to implement under the prerequisite with the technical solution of the present invention, but protection scope of the present invention is not limited to following embodiment.
As shown in Figure 1, present embodiment comprises: mouse-driven module, information feedback module and phonetic order processing module; Described mouse-driven module is made up of infraluminescence mark 1, camera 2, infrared fileter 3 and cursor positioning submodule, the information feedback module is formed by being connected on the main frame 5 display 6, and the phonetic order processing module is formed by being connected on the main frame microphone 4 and speech recognition submodule.Wherein cursor positioning, two submodules of speech recognition all are the application modules on the main frame 5.The mouse-driven module is made of the camera 2 that infrared fileter 3 is housed, infraluminescence mark 1 and cursor positioning submodule, camera 2 is by detecting the light spot image of following the tracks of infrared mark 1, can move the position of dynamic tracking hot spot on the camera imaging plane, be that cursor of mouse showing motion desktop on hot spot in the movement conversion on the image by the cursor positioning submodule then, and the positional information of mouse is passed to the information feedback module; The information feedback module shows by the position of the cursor of mouse that graphoscope 6 transmits the mouse-driven module, and gives the corresponding information indicating of user; When the information feedback module shows that cursor of mouse arrives user's interested position, the phonetic order processing module receives the phonetic order that the user sends by the microphone 4 that is connected with main frame 5, by the speech recognition submodule input speech signal is discerned, excited and drive corresponding mouse event response (left button is clicked, right-click, left button double-click) by classification results.
As shown in Figure 2, present embodiment at first uses microphone 4 to gather all instruction voice signals that the user uses, and the linear predictor coefficient (LPCC) that extracts these voice signals stores on the disk of main frame 5, as the received pronunciation sample.Then, the hand-held infraluminescence mark 1 of user moves in camera 2 visual ranges, infrared fileter 3 is housed on the camera lens of camera 2 in the mouse-driven module, can only respond to the light that infrared mark 1 sends, adopt the method that adaptive threshold is cut apart to carry out the infrared light spot detection, obtain the picture position coordinate of infrared light spot by the method for asking center of gravity.Learn the position coordinates of infrared light spot on the former frame image, in a less image coordinate scope, detect the infrared light spot position of back one two field picture as benchmark, thereby realize the dynamic tracking of infrared light spot, the cursor positioning submodule moves the position of infraluminescence mark 1 and is converted into cursor of mouse in the motion that shows on the desktop.The information feedback module will be presented on the display 6 by the mouse cursor position that the mouse-driven module obtains.Common user moves by the cursor of mouse on the observation display 6, judges whether to send phonetic order; The phonetic order processing module is behind the voice signal that receives microphone 4 collections, and the speech recognition submodule will be imported phonetic order and drive the mouse event response with nearest neighbor method classification (is classification center with the received pronunciation sample) and with classification results.
(left mouse button is clicked to carry out conventional mouse action by the mouse system that uses present embodiment on the computing machine that Windows XP operating system is housed; Right mouse button is clicked; Left mouse button is double-clicked) 100 times, record phonetic order correct execution rate of the present invention greater than 86%, the mouse positioning error is 1.6 pixels, by these two indexs as can be seen present embodiment have robust, accurate superperformance.
Claims (4)
1, a kind of mouse system based on vision track and speech recognition is characterized in that, comprising: mouse-driven module, information feedback module and phonetic order processing module; The mouse-driven module is made of camera, infraluminescence mark and cursor positioning submodule, camera is by detecting the light spot image of following the tracks of infrared mark, move the position of dynamic tracking hot spot on the camera imaging plane, be that cursor of mouse showing motion desktop on hot spot in the movement conversion on the image by the cursor positioning submodule then, and the positional information of mouse is passed to the information feedback module; The information feedback module shows by the position of the cursor of mouse that graphoscope transmits the mouse-driven module, and gives the corresponding information indicating of user; The phonetic order processing module is made up of microphone and speech recognition submodule, when the information feedback module shows that cursor of mouse arrives user's interested position, the phonetic order processing module receives the phonetic order that the user sends by the microphone that is connected with main frame, by the speech recognition submodule input speech signal is discerned, excited and drive corresponding mouse event response by classification results.
2, the mouse system based on vision track and speech recognition according to claim 1, it is characterized in that, speech recognition submodule in the described phonetic order processing module, it is input as the linear predictor coefficient that microphone receives voice signal, be output as the Classification and Identification result of phonetic order, classification results drives corresponding operating-system resources and realizes the response of corresponding mouse event, comprises that left button is clicked, right-click, left button double-click.
3, the mouse system based on vision track and speech recognition according to claim 1, it is characterized in that, described microphone is gathered all instruction voice signals that the user uses, and the linear predictor coefficient that extracts these voice signals stores on the disk of main frame, as the received pronunciation sample.
4, the mouse system based on vision track and speech recognition according to claim 1, it is characterized in that, in the described mouse-driven module, the infraluminescence mark moves in the camera visual range, on the camera lens of camera infrared fileter is housed, can only respond to the light that infrared mark sends, adopt the method that adaptive threshold is cut apart to carry out the infrared light spot detection, obtain the picture position coordinate of infrared light spot by the method for asking center of gravity; Learn the position coordinates of infrared light spot on the former frame image, in a little image coordinate scope, detect the infrared light spot position of back one two field picture, thereby realize the dynamic tracking of infrared light spot as benchmark.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2007100399967A CN100449468C (en) | 2007-04-26 | 2007-04-26 | Mouse system based on visual tracking and voice recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2007100399967A CN100449468C (en) | 2007-04-26 | 2007-04-26 | Mouse system based on visual tracking and voice recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101038523A true CN101038523A (en) | 2007-09-19 |
| CN100449468C CN100449468C (en) | 2009-01-07 |
Family
ID=38889457
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2007100399967A Expired - Fee Related CN100449468C (en) | 2007-04-26 | 2007-04-26 | Mouse system based on visual tracking and voice recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100449468C (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101887316A (en) * | 2010-07-28 | 2010-11-17 | 郭小卫 | Method for acquiring indication information by mouse and mouse applying same |
| CN101430625B (en) * | 2007-11-07 | 2011-07-27 | 夏普株式会社 | Display system and method for detecting pointed position |
| WO2012063247A1 (en) * | 2010-11-12 | 2012-05-18 | Hewlett-Packard Development Company, L . P . | Input processing |
| CN102662475A (en) * | 2012-04-18 | 2012-09-12 | 华南理工大学 | Novel mouse based on computer vision and interaction method |
| CN102693022A (en) * | 2011-12-12 | 2012-09-26 | 苏州科雷芯电子科技有限公司 | Vision tracking and voice identification mouse system |
| CN102945672A (en) * | 2012-09-29 | 2013-02-27 | 深圳市国华识别科技开发有限公司 | Voice control system for multimedia equipment, and voice control method |
| CN103294196A (en) * | 2013-05-15 | 2013-09-11 | 陈创举 | Head-operated mouse |
| CN103870149A (en) * | 2012-12-18 | 2014-06-18 | 联想(北京)有限公司 | Data processing method and electronic equipment |
| CN103902023A (en) * | 2012-12-25 | 2014-07-02 | 苏州薇思雨软件科技有限公司 | Novel voice-activated mouse |
| CN104714630A (en) * | 2013-12-12 | 2015-06-17 | 联想(北京)有限公司 | Gesture recognition method and system and computer |
| CN104731549A (en) * | 2015-04-09 | 2015-06-24 | 徐敏 | Voice recognition man-machine interaction device based on mouse and method thereof |
| CN106383603A (en) * | 2016-09-23 | 2017-02-08 | 安徽声讯信息技术有限公司 | Voice control system based on voice mouse |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2242590A (en) * | 1989-11-22 | 1991-10-02 | Marconi Gec Ltd | Visual tracking apparatus |
| JPH07234694A (en) * | 1994-02-21 | 1995-09-05 | N T T Data Tsushin Kk | Automatic reception device |
| CN1357862A (en) * | 2000-12-06 | 2002-07-10 | 英业达股份有限公司 | Cursor clicking and selecting method and device in windows |
| US20040133381A1 (en) * | 2003-01-02 | 2004-07-08 | Ping-Ho Chen | Control scheme for spatial and level searching of a panoramic stabilized periscope |
| CN1760892A (en) * | 2004-10-12 | 2006-04-19 | 刘本林 | Unit of header label in use for man-machine interface of computer |
-
2007
- 2007-04-26 CN CNB2007100399967A patent/CN100449468C/en not_active Expired - Fee Related
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101430625B (en) * | 2007-11-07 | 2011-07-27 | 夏普株式会社 | Display system and method for detecting pointed position |
| CN101887316A (en) * | 2010-07-28 | 2010-11-17 | 郭小卫 | Method for acquiring indication information by mouse and mouse applying same |
| WO2012063247A1 (en) * | 2010-11-12 | 2012-05-18 | Hewlett-Packard Development Company, L . P . | Input processing |
| CN102693022A (en) * | 2011-12-12 | 2012-09-26 | 苏州科雷芯电子科技有限公司 | Vision tracking and voice identification mouse system |
| CN102662475A (en) * | 2012-04-18 | 2012-09-12 | 华南理工大学 | Novel mouse based on computer vision and interaction method |
| CN102945672B (en) * | 2012-09-29 | 2013-10-16 | 深圳市国华识别科技开发有限公司 | Voice control system for multimedia equipment, and voice control method |
| CN102945672A (en) * | 2012-09-29 | 2013-02-27 | 深圳市国华识别科技开发有限公司 | Voice control system for multimedia equipment, and voice control method |
| CN103870149A (en) * | 2012-12-18 | 2014-06-18 | 联想(北京)有限公司 | Data processing method and electronic equipment |
| CN103870149B (en) * | 2012-12-18 | 2017-08-29 | 联想(北京)有限公司 | Data processing method and electronic equipment |
| CN103902023A (en) * | 2012-12-25 | 2014-07-02 | 苏州薇思雨软件科技有限公司 | Novel voice-activated mouse |
| CN103294196A (en) * | 2013-05-15 | 2013-09-11 | 陈创举 | Head-operated mouse |
| CN104714630A (en) * | 2013-12-12 | 2015-06-17 | 联想(北京)有限公司 | Gesture recognition method and system and computer |
| CN104714630B (en) * | 2013-12-12 | 2017-12-29 | 联想(北京)有限公司 | Gesture identification method, system and computer |
| CN104731549A (en) * | 2015-04-09 | 2015-06-24 | 徐敏 | Voice recognition man-machine interaction device based on mouse and method thereof |
| CN104731549B (en) * | 2015-04-09 | 2017-12-19 | 安徽咪鼠科技有限公司 | A kind of speech recognition man-machine interaction method based on mouse |
| CN106383603A (en) * | 2016-09-23 | 2017-02-08 | 安徽声讯信息技术有限公司 | Voice control system based on voice mouse |
Also Published As
| Publication number | Publication date |
|---|---|
| CN100449468C (en) | 2009-01-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100449468C (en) | Mouse system based on visual tracking and voice recognition | |
| CN1322329B (en) | Imput device using scanning sensors | |
| Stiefmeier et al. | Combining motion sensors and ultrasonic hands tracking for continuous activity recognition in a maintenance scenario | |
| CN103513811B (en) | Touch trajectory tracking method | |
| US20100211918A1 (en) | Web Cam Based User Interaction | |
| CN1530876A (en) | Handwriting path identifying system and method | |
| Munich et al. | Visual input for pen-based computers | |
| WO2004061751A3 (en) | Compact optical pointing apparatus and method | |
| CN101869484A (en) | Medical diagnosis device having touch screen and control method thereof | |
| Robinson et al. | The LivePaper system: augmenting paper on an enhanced tabletop | |
| KR20140114224A (en) | Information input device and method | |
| CN101609364B (en) | Input information identification equipment of remote control computer and use method | |
| CN201383135Y (en) | Multiple touch control interactive device | |
| CN1746837A (en) | Electronic display system positioning method and positioner thereof | |
| CN1755588A (en) | Interaction-based computer interfacing method and device | |
| CN116070173B (en) | Finger reading method and system for cross-modal task instruction understanding | |
| CN1744017A (en) | Infrared and ultrasonic intelligent electronic white board | |
| CN215932585U (en) | Screen writing device | |
| CN214870610U (en) | Robot for intelligent analysis and laser navigation | |
| CN2757227Y (en) | Infrared and supersonic intelligent electronic white plate | |
| Boato et al. | Hand tracking and trajectory analysis for physical rehabilitation | |
| Mali et al. | Design and Implementation of Hand Gesture Assistant Command Control Video Player Interface for Physically Challenged People | |
| CN202523038U (en) | Multipoint touch control system based on ARM (Advanced RSIC Machine) processor | |
| CN2613815Y (en) | Photoelectric mouse | |
| KR20030021987A (en) | Wireless finger mouse system based on image processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20090107 Termination date: 20110426 |