201128541 六、發明說明： 【發明所屬之技術領域】 本發明提供一種手勢辨識之互動裝置，尤指一種利用視 的的取像裝置檢測空間中的單手或是雙手的運作模式的手語的辨 識方法。 【先前技術】 在電子設備發展史，鍵盤、滑鼠與觸控板出現，解決輸入 ^問題。不過’對初學者而言，學會使用這些輸人介面門楼相| 尚，而且所佔空間不小，例如，筆記型電腦或手機一半體積都& 鍵盤佔據。若能省下鍵盤空間，自然能提升產品可攜性，最可行 的方式，就是直接於面板以觸控方式進行操作。因此，對於指向 性裝置(Pointing Stick)與觸控板(Touch Pad)技術有日益廣泛的^用 需求，希望能用以取代傳統的滑鼠、鍵盤的操作功能，由ς 勢s吾s的形恶簡單且不易辨識錯誤，一些學術研究機構開始著手 ，展可辨識手勢語言的技術，讓觸控面板操作更人性化且將可讓 夕隻手指、手掌及多位使用者同時使用，達到手觸的最 。 觸控技術發展，其實已行之有年，'然而，自iPhGne推出，為觸^ 市場巧下了」枚震撼彈，電子產業瞭解到，觸控螢幕並非只是ί 眾^場，其貫足以成為主流輸入介面。iPh〇ne證明好的觸控介面， 確貫能取代大多數鍵盤功能，並賦與使用者更直覺、便利的操作 體驗；以更大的面板替代鍵盤，還能設計出更輕薄、時尚造型； 加亡完全採賴態面板技術賴心缝、滑輪._·等機械零件 故障問題。 it產品將步入多點觸控(Multi_t〇uch)新時代，雖可運用電阻/ 電容的方式建置多關控錄電腦，但是此種方式_控點數無 法很夕，車乂不適合多位使用者同時使用。而較新穎的技術則是用 ，荷编合元件(CCD)取像電腦視覺的方式建置，除了觸控點數 非常多外’亦不限於單—使时，可财隻手指、其他筆控方式 同時使用，達到難iTGUCh的最高境界，其應用 201128541 目前’在電腦作業系統中，已經開發出觸控的相容模式， 所以各家業者無不競相投入手勢識別的發展，欲定義出不同 的操作手勢。觸控螢幕的應用越來越多’促使各項技術發展也相 當快1 ° iPhone改採投射電容式技術實現多點觸控功能，讓大家 將注意力轉移到新的觸控技術。激發觸控螢幕朝向多點觸控互動 技術激烈競爭，有多種不同技術正在發展，包括電容式、表面音 f式及，電式等。由於近年來影像感測元件及影像處理速度提 高，使得影像光電式觸控互動螢幕受到重視，有許多廠商提出新 的貫驗系統或產品。 目則的光學觸控是直接觸摸螢幕，會有汙損的問題以及裝設 Ξ難If有的鍵盤操作，者需要注視於顯示器與鍵盤之間切 換。^疋需要外掛滑鼠、手寫版、Touch Pad等控制器才能操作。 德且ϋϋ:視窗控_手勢是各家觸控系統所力爭之地；日 W乃〜作手勢的感測系統將越被使用者所接受。然而觸 控仍又限於有限的面積才可以實施，並 設計方宰，研私、、=創5又出一種能夠解決上述不足和缺陷的發明 dr種手勢辨識之互動裝置。 互動要^ Γί在於提供—種手勢辨識之互動裝置，該 影像s 置，藉由攝取的 指標，，與呼叫出㈣、##^：細益上的一筒含)以上的，，多媒體 本發明之另目翻虛擬触像難功能。 方法，該互動裝置㈣手語游標控制的辨識 是瞼）直接操作。#不而任何掛裝置’只需使用者的手（或 為達到本發明的上述目的， ^:本發明提供一手勢辨識之互過技術方案加以實 i si固(含)以上的取像裝置捕捉位^ ί的iif y視窗上 據，使感測系統能夠偵測到空二3的手指位置之圖項數 节的手彳日的動作；手指的動作則 201128541 包含有單手與雙手動作的操 π〇 ，啟動，式、滑鼠移動、滑鼠左“、f ^的操作模式包括 mmim 記€數=====則是利用設定的特定201128541 VI. Description of the Invention: [Technical Field] The present invention provides an interactive device for gesture recognition, and more particularly to an identification of a sign language using an image capturing device for detecting one-handed or two-handed operation modes in a space. method. [Prior Art] In the history of electronic devices, keyboards, mice and touchpads appeared to solve the input problem. However, for beginners, learning to use these input interface is still, and the space is not small, for example, half of the notebook or mobile phone is & keyboard occupied. If you can save keyboard space, you can naturally improve the portability of the product. The most feasible way is to operate the touch panel directly on the panel. Therefore, there is an increasing demand for Pointing Stick and Touch Pad technology, and it is hoped that it can replace the operation functions of the traditional mouse and keyboard, and the shape of the sho Evil is simple and difficult to identify mistakes. Some academic research institutions have begun to develop techniques for recognizing gesture language, making touch panel operation more user-friendly and allowing simultaneous use of fingers, palms and multiple users. the most. The development of touch technology has actually been going on for a long time. 'However, since the launch of iPhGne, it has been made for the touch of the market." The electronics industry has learned that the touch screen is not just a gamut, it is enough to become Mainstream input interface. iPh〇ne proves that a good touch interface can replace most keyboard functions and give users a more intuitive and convenient operation experience. With a larger panel instead of a keyboard, it can also be designed to be lighter and thinner. Adding and destroying completely immersed panel technology, Lai Xinsu, pulley, _· and other mechanical parts failure problems. It products will step into the new era of multi-touch (Multi_t〇uch), although the use of resistors / capacitors to build a multi-control computer, but this way _ control points can not be very good, the rut is not suitable for multiple Users use at the same time. The more novel technology is used, and the CCD is built in the way of computer vision. In addition to the very large number of touch points, it is not limited to single-time, but only for fingers and other pens. The way to use at the same time, to reach the highest level of iTGUCh, its application 201128541 Currently 'in the computer operating system, has developed a compatible mode of touch, so all the players are competing to invest in the development of gesture recognition, to define different Action gestures. The application of touch screens is increasing~ The development of various technologies is also relatively fast. 1 ° The iPhone has adopted the projected capacitive technology to realize multi-touch function, so that everyone can turn their attention to the new touch technology. Inspire the touch screen to face multi-touch interactive technology. There are many different technologies under development, including capacitive, surface acoustic, and electrical. Due to the increasing speed of image sensing components and image processing in recent years, image photoelectric touch interactive screens have received attention, and many manufacturers have proposed new inspection systems or products. The optical touch of the target is to directly touch the screen, there will be problems of fouling and the keyboard operation of the installation If you need to watch the switch between the display and the keyboard. ^疋 Need a mouse, handwritten version, Touch Pad and other controllers to operate. De and ϋϋ: Windows Control _ Gesture is the place where various touch systems strive; the W-to-gesture sensing system will be more accepted by users. However, the control is still limited to a limited area before it can be implemented, and the design of the slaughter, research and development, and the creation of an interactive device capable of solving the above-mentioned deficiencies and defects. The interaction is to provide an interactive device for gesture recognition, which is set by the ingested indicator, and the call out (four), ##^: a package on the fine, or above, the multimedia invention It is also difficult to turn over the virtual touch. Method, the identification of the interactive device (4) sign language cursor control is 睑) direct operation. #不不挂装置"only the user's hand (or in order to achieve the above object of the present invention, ^: the present invention provides a gesture recognition of the mutual technical solution to capture the image capture device The iif y window of the ^ ^ ί enables the sensing system to detect the movement of the hand of the finger position of the finger position of the empty two 3; the action of the finger is 201128541, including the one-handed and the two-handed action 〇π〇, start, type, mouse movement, mouse left ", f ^ operation mode including mmim record number ===== is to use the specific setting
號’感;系統:宜併攏作為感測啟動訊 離，當感測系統感應到食指%指的移曰動^為游標的移動距 動結束心游標將停止於“感=^再=失時，則設定為移 滑鼠右鍵的模式則是利用小指 ;，，此時游標則停留於原 就越小，適合於精準控制以近時，相對移動的比例 示相對移動比例2: ϊ合二==，的角 表相對移動比例越大視^速移動^拇指越向外張代 變化二與食指間角度的 食指、中指的移動比麵式則是湘雙手的梅指、 比例；食指與中&二指與食指間的角度代表相對移動 ===，指與中指分開時則顯 歧下·縣侧代m左鍵點^: 感測特徵，：雔^的片：拇指與(食指及申指)間的虎口做為 手的虎口向内7、::虎 201128541 旋轉模式則是利用雙手的拇指與(食指及中指）間的虎口做 ^測特徵，當雙手的虎口相對旋轉方向時則表示視窗進行3〇的旋 到雙手虎口特徵後，雙手虎口的旋轉角度即帶動視窗 角轉動作：當雙手虎口特徵消失後，旋轉動作即停止於最後 上述說明可知，在視窗制這塊領域中，手語的動作是相 1 的，除了必須要使偵測的時間縮短且具有高正確辨識率， I具有祕作的特性。本發明手勢顺之聽裝置是將手注予 二短ΐ時間被偵測到’縮短視窗的運作時^ *，、、曼稭由間化的手浯刼作，始使用者得以熟練視窗的操作。 另外本發明還提供了多媒體功能列操作模式： 啟動模式：以使用者自行設定的(或DEFUAL)的手 L加I，2,3,4,5,6,7,8,9等）或搭配語音(如芝麻開門)作為啟動暗 結束模式：以使用者自行設定的(或DEFUAL)的手 味ί H3,4,5,6,7,8,9等）或搭配語音(如芝麻開門)作為έ士束# :^搭配的程式被關閉時，原對應的多媒體功能列’:、自上; ，作模式：視被呼叫_最新執行程式為何，會呼 功容許使用者自行設定）。如以下模式·· j多媒 顯示器最上層；b.仍然以上述單手游標控制ΪΪ f實式3 —，G·亦可在設定功能列時增設語音控制功能。 具體t詳構，現配合附圖結合本發明 以上 個(含） 圖像以探查感測點，檢查此指 個=指向感測_=置==¾的； 描、.會-個手语的不同的連續的指向動作發生時，顯示裝^會^ 201128541 手操作模式的手語的操作有j手與雙 者的手（或是臉）直接操作。不而任何掛褒置，只需使用 鍵、：右目滑鼠左 式包括^移動比例、縮放模式與旋轉模式等貢項目目而又手的操作模 的特定記號，如剪刀、石頭、布則是利用設定 S!f *· a應起偵點，進人感啦、统的運作模式。 μ為 攏作芩中指併 再移動。為移動結束訊號，游標將停止於最後感應點，不 下弯所示，滑鼠右鍵的模式則是利用無名指與小指 則=原先的出號時，此時游標 攏或;滑鼠左鍵的模式則是利用拇指向食指靠 游標:留於======號時，此時 ^r=，適合於精上== 動;藉由Ϊ以：；;目=例f大，適合於游標的快速移 有不同的變化Γ 角度㈣化’即可使游標的相對移動速度 手操作Μ式則疋可作為緣圖製作講稿與電子書的觸控電腦 [S3 201128541 的使用功能，包括有蒋叙 請參見第ό圖所示，、縮放模式與旋轉模式等項目。 指、食指、中指做為感·、s移動比例模式則是利用雙手的梅 移動比例；食指與^的位食指間的角度代表相對 攏時為帶動游標位移的 置=代表游私訊號，當食指與甲指併 移終止的訊號；當拇护=二艮扣與中指分開時則顯示游標位 點選；此作用方式與下彎動作則代表滑鼠右鍵 中指调的虎口做雙手的拇指與(食指及No. 'Sense; system: should be close together as a sensing start signal, when the sensing system senses the shift of the index finger % finger ^ for the moving distance of the cursor, the heart cursor will stop at "Sense = ^ then = lost time, The mode set to move the right mouse button is to use the little finger;,, at this time, the cursor stays at the original smaller, suitable for precise control to the near time, the relative movement ratio shows the relative movement ratio 2: ϊ合二==, The greater the relative movement ratio of the angle table, the faster the movement of the thumb, the more the outward change of the thumb, the change of the index finger and the middle finger of the angle between the index finger and the index finger is the plum finger and the proportion of the hands of the Xiang; the index finger and the middle & The angle between the two fingers and the index finger represents the relative movement ===, when the finger is separated from the middle finger, then the difference is made. The county side generation m left key point ^: Sensing characteristics: 雔^ The film: thumb and (index finger and index finger The tiger's mouth as the hand's tiger's mouth is inward. 7,:: Tiger 201128541 The rotation mode is to use the tiger's mouth between the thumb and the index finger and the middle finger to make the test feature. When the tiger's mouth of the two hands is relatively rotated, After the window is rotated to the characteristics of the two-handed tiger's mouth, the angle of rotation of the two-handed tiger's mouth is Moving window corner movement: When the characteristics of the two-handed tiger mouth disappear, the rotation action stops at the last description. In the field of window system, the sign language action is phase 1, except that the detection time must be shortened and It has a high correct recognition rate, and I has the characteristics of a secret. The gesture of the gesture-sounding device of the present invention is that the hand is injected for two short time times and is detected as 'shortening the operation of the window ^,,, The user can master the operation of the window. In addition, the present invention also provides a multimedia function column operation mode: Start mode: user-set (or DEFUAL) hand L plus I, 2, 3, 4 , 5,6,7,8,9, etc.) or with voice (such as sesame open) as the start dark end mode: user-defined (or DEFUAL) hand ί H3,4,5,6,7, 8,9, etc. or with voice (such as sesame open door) as a gentleman bundle # : ^ matching program is closed, the original corresponding multimedia function column ':, from the top;, mode: as called _ latest executive program Why, it will allow the user to set it himself.) The following mode·· j multimedia display The upper layer; b. still with the above one-handed cursor control ΪΪ f real 3 -, G · can also add a voice control function when setting the function column. The specific t detailed structure, now with the figure combined with the above (including) figure of the present invention Like to probe the sensing point, check this finger = point to sense _ = set == 3⁄4; when the different continuous pointing actions of the tracing, sign, and sign language occur, the display will be ^^^^^^^^^^^ The operation of the sign language has direct manipulation of the hands of the j hands and the faces of the two hands (or the face). No matter what the hanging device is, just use the key: the right eye mouse left type includes the movement ratio, the zoom mode and the rotation mode. The specific marks of the operation mode of the hand, such as scissors, stones, and cloth, are used to set the S!f *· a to start the detection point, enter the sense of the person, and the operation mode. μ is the middle finger and moves again. In order to move the end signal, the cursor will stop at the last sensing point, not shown in the corner. The mode of the right mouse button is to use the ring finger and the little finger = the original number, then the cursor is closed or the left mouse button mode is Is to use the thumb to the index finger by the cursor: when the ====== number, at this time ^r=, suitable for fine == move; by Ϊ to :;; 目 = example f large, suitable for cursors There are different changes in the fast movement. The angle (four) can be used to make the cursor's relative movement speed. The hand-operated method can be used as a margin map to make a script and an e-book touch computer [S3 201128541, including the use of Jiang Shu. See the figure in the figure, zoom mode and rotation mode. The index, the index finger, the middle finger are used as the sense, and the s moving ratio mode is the proportion of the plum movement using the hands; the angle between the index finger and the index finger of the ^ represents the relative displacement of the cursor to represent the displacement of the cursor = representative of the private signal, when The index finger and the nail finger move and terminate the signal; when the thumb protector = the second button is separated from the middle finger, the cursor position is selected; the action mode and the downward bend action represent the thumb of the mouse mouth of the right mouse button. (index finger and
請參見第收縮時，代表視窗鏡頭縮小。 中指)間的虎口做二♦轉拉式則是利用雙手的梅指與(食指及 示視窗進行二t轉; 直覺是未來的主流，其誕生是為了基於消費者對於 是為传靡/求’而本發贼窗手語的辨識方法的研發目的就 疋為使觸控電腦的應収為便捷。 *曰的就See Shrinking, which represents a reduction in the window shot. The middle finger between the tiger's mouth to do two ♦ pull-type is to use the hands of the plum fingers and (index finger and display window for two t turn; intuition is the mainstream of the future, its birth is based on the consumer is for the rumors / seeking 'The development purpose of the identification method of the thief window sign language is to make the acceptance of the touch computer convenient.
本發明還提供了多媒體功能列操作模式： 社f動模式：以使用者自行設定的(或defual)的手勢(剪刀石 號布加1，2,3,4,5,6,7,8,9等）或搭配語音(如芝麻開門)作為啟動暗 、，，σ束模式：以使用者自行設定的(或DEFUAL)的手勢(剪刀石 =1，2,3,4,5,6,7,8,9等）或搭配語音(如芝麻開門)作為結 备所搭配的程式被關閉時，原對應的多媒體功能列，自動關 操作模式：視被呼叫時的最新執行程式為何，會 ^的f媒體功能列(容許使用者自行設定）。如以下模式：a.將多媒 里功能列顯示在顯示器最上層；b.仍然以上述單手游標控制與點 ^方式操作功能列；c.亦可在設定功能列時增設語音控制功^’’’。 8 [S] 201128541 體控=點選P〇WERP〇iNT檀案後，啟動功能列，會產生多媒 啟動全縣播綱是以雙轉做為感測； 以單選後ic⑽放大並_，邮應骨座， 式控制s周整滑座’以握拳作為結束指令. 啟動^量大小聲則是在點選後lc〇N放大並’ 滑座’以早手操作模式控制調 見敏 錐丰;;，、_彳是在點選後出現螢幕===手或 螢幕鍵盤位置時該按鍵自動放Α, ^丰=標出現在 抬得動作輯倾錄被^並上 為使熟悉該項技術者如實瞭解本發明 功效，所舉之實施例及附圖僅 明目的與 明加以任何限制，本發明還可尚有其他不對本發 熟悉泣匕項技術者在不脫離本案發明精神下進行其以) 應視為本案申。請專利範圍的等效實施。 “樣式貫鈀，均 【圖式簡單說明】 第1圖為Perspective Pixel之技術屌理· =本發明啟動方式之手勢示意圖； ί ^圖為本發明料移動之手勢示意圖； 苐4圖為本發明滑鼠右鍵之手音 第5圖為本發明滑鼠左鍵之手勢示g圖: ί 發明游標相對移動速度;制之手勢干. J:圖為本發明雙手移動比例模式之手勢：忍圖， 第9圖為本發明雙手旋轉模式 【主要元件符號說明】 亍羿不思圖。The invention also provides a multimedia function column operation mode: a social mode: a user-set (or defual) gesture (scissors stone cloth plus 1, 2, 3, 4, 5, 6, 7, 8 9 etc.) or with voice (such as sesame open) as the start dark,, σ beam mode: user-set (or DEFUAL) gesture (scissors stone = 1, 2, 3, 4, 5, 6, 7 , 8, 9, etc.) or with the voice (such as sesame open door) as the program is closed, the original corresponding multimedia function column, automatic shutdown mode: depending on the latest execution program when called, will ^ f Media function column (allows the user to set it himself). For example, the following modes: a. Display the multimedia function column on the top of the display; b. Still use the one-hand cursor control and point mode operation function column; c. You can also add voice control function when setting the function column. ''. 8 [S] 201128541 Body Control = Click on the P〇WERP〇iNT Tan case, start the function column, it will generate the multimedia start. The county broadcast is based on double rotation as the sensing; after the single selection, ic(10) is enlarged and _, Mail should be the pedestal, control s week, the whole slide 'with the fist to end the instruction. Start the amount of sound is the lc〇N after the point is selected and the 'slider' is controlled by the early hand operation mode. ;;,,_彳 is the screen that appears when the screen appears after the click === hand or screen keyboard position, ^ Feng = standard appears in the lift action series is recorded and is used to make the technology familiar The present invention is not limited to the purpose of the present invention, and the present invention may be carried out without any limitation to the present invention. ) should be regarded as the application of this case. Please request an equivalent implementation of the patent scope. "Pattern Palladium, both [Simple Description] Figure 1 is a technical diagram of Perspective Pixel · = schematic diagram of the gesture of the startup mode of the present invention; ί ^ is a schematic diagram of the gesture of the material movement of the present invention; The right hand of the right mouse button 5 is the gesture of the left mouse button of the present invention: ί Invented the relative movement speed of the cursor; the gesture is dry. J: The figure is the gesture of the hand movement mode of the invention: Figure 9 is the two-handed rotation mode of the present invention [the main component symbol description] 亍羿不思图.
9 [SI9 [SI