TW201237674A - Device and method for controlling mouse cursor by head - Google Patents

Device and method for controlling mouse cursor by head Download PDF

Info

Publication number
TW201237674A
TW201237674A TW100108134A TW100108134A TW201237674A TW 201237674 A TW201237674 A TW 201237674A TW 100108134 A TW100108134 A TW 100108134A TW 100108134 A TW100108134 A TW 100108134A TW 201237674 A TW201237674 A TW 201237674A
Authority
TW
Taiwan
Prior art keywords
coordinates
image
coordinate
cursor
center
Prior art date
Application number
TW100108134A
Other languages
Chinese (zh)
Other versions
TWI480764B (en
Inventor
Ching-Hua Chiu
Yung-Kuan Chan
Meng-Hsiun Tsai
Original Assignee
Nat Univ Chung Hsing
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nat Univ Chung Hsing filed Critical Nat Univ Chung Hsing
Priority to TW100108134A priority Critical patent/TWI480764B/en
Publication of TW201237674A publication Critical patent/TW201237674A/en
Application granted granted Critical
Publication of TWI480764B publication Critical patent/TWI480764B/en

Links

Landscapes

  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)

Abstract

This invention relates to a device and method for controlling mouse cursor by head. About the method, it includes the following steps: (1) preparing step; (2) screen coordinates setting step; (3) referential frame coordinates capturing step; (4) coordinates capturing step for the midpoint between the eyebrows and two eyes' centers; (5) 3 dimensional coordinates converting step; (6) triangle center cursor-controlling step; (7) finishing step. By these steps, two image capturing devices capture the related coordinates of the referential frame first. Then, the referential frame is removed away. After which, two image capturing deices capture the related coordinates of the user's face. By doing the 3 dimensional coordinates conversion, the coordinates of the midpoint between the eyebrows and two eyes' centers can be obtained. These three points form an invisible triangle. A normal line can be extended from the triangle's center and then intersects with the screen of the display. The intersected point would be used as a cursor. When the user's head moves, the intersected point which is functioned as a cursor will move accordingly as so to send out a desired command. So, the cursor is controlled by head movement in this invention. The user's hand is not necessary to leave the keyboard. In addition, it is suitable for the user who has no hands.

Description

201237674 六、發明說明: 【發明所屬之技術領域】 本,明係有關-種控制游標之袋置及方法 控制游標之裝置及方法’其兼具移動臉部 種^ 不需離開鍵盤相當方便與利於手部殘缺者使料手π 【先前技術】 ’ 目則較普遍的微軟視窗系統,都是以滑鼠 1動並執行命令,對-般整天使用電腦的上班於^面 對按鍵的位置,從鍵盤移到滑鼠上亦技對滑 鍵,移動過程麻煩而耗時。 石汉r 而目前市面上雖然有頭控型的游標控制裝 ,型專 M2漏號之可赋影像追縱置中= 疋ί必需要頭上掛兩個光源,以供攝影機擷取兩個光i的位置 而轉換為游標位置,這樣的裝置產生以一問題: 胁[1]3必需縣充足電量。傳朗控式裝置在個時的第-,點’疋^需保持光源的電量充足,萬—使用到—半因電g不 J造成光量減弱甚至完全無法發光,職套綠完全無法使 用0 [2] 容易造成頭部不舒服。傳統頭控式裝置必需要在頭上 控一組光源,長時間使用下來,會造成頭不舒服。 [3] 控制不易。傳統頭控式裝置的點選要件, 預定時嶋即座標點保持_,才由系關 ^為點選扎々,如此讓使用者壓力相當大,因為只要稍微動一 下,就必需重新作定位的動作。故,控制不易。 有鑑於此,必需研發出可解決上述習用缺點之技術。 【發明内容】 ,,明之目的,在於提供一種頭部控制游標之裝置及方 次,/、兼具移動臉部即可控制游標、手部不需離開鍵盤相當方 201237674 =手:之優點。特別是,,所欲解決之 不舒服與控制不“。必需保持充足電量、容易造成頭部 置及段係提供-種頭部控制游標之裳 標於其上移$,游標’該晝面係供該游 標、一左包括一中心點座標、一右上頂角座 -影像』:f;、二左下頂角座標及一右下頂角座標; 像擷取裝Ϊ =置-右側影像擷取裝置及-左側影 二與-=== 個定位點:該、,二ί架體間之拍攝距離;該參考架體具有複數 面的座標令,具座標’其可對應於該晝 攝口右„裝置取“ 中心譽傻* —目^像、帛一右眼中心影像及-第-左眼 心上人臉 像,並在即時影像處理後,可與該i:ij對匕, 別件到-第二2D ,,對應’而分 二沁左眼中心座標;’、第一2〇左眼中心座標與-第 心卸係以該兩個2D眉心座標、該兩個2〇右眼中 換,:鋪線性轉 中心座押* U右眼中心座標及一 3D左眼 虛擬:ί平;;平面’紅前述三組座標求得該 -角千面_其三角中心點之法向量,並可求出由該三角 201237674 該法向量直線指向該畫面之交點座標,其為該游標之 動’並汉疋該兩個右眼中心影像之眨 鍵指令;且設定該兩個左眼中鼠右 ^ 〜像之眨眼犄間變化可輸出滑 間變化可__指令;置 本發明之方法部份係包括下列步驟:醫知之瑕置 一·準備步驟; 晝面座標設定步驟; 拍頁取參考架體座標步驟; 四 五 -Λ- /、 七 擷取使用者眉心與兩眼中心座標步驟; 進行3D座標線性轉換步驟; 三角中心點控制游標步驟;及 完成步驟。 本f明之上述目的與優點難從下 細說明與_巾,獲得深人瞭解。 ^胃州(坪 兹以下列實施舰配合圖式詳細本發明於後: 【實施方式】 本發明係為-_部控制游標之裝置及方法,參 、第-B、第二A、第三B及第五圖,其裝置部份包括: 一顯示ϋ 10,係用以顯示一畫面U及一游標12, 11係供該游標12於其上移動,且該晝面1Η系包括__中& 標(/。^^、一右上頂角座標⑷心心^—左上頂角座 石:》: —左下頂角座標(κΛ〇及一右下頂角座^ -影像榻取裝置20,係、包括—右側影軸取裝置2 一 左側影像擷取裝置2GB,該右、該左側影像麻裝置雇 皆先後對位於同-位置上的參考架體91與使用者臉部92、= 201237674 之座標可4_括^ y t f ) =畫面11 一定位點911皆具為24個)定位點91卜該每 與202對應該複數個定 =:影像201 別具有複數個(共24個> 至^,U24)),而分 個(共24個)第二座择^ 庋裇(4队而至為4(\4))與複數 者臉部92拍攝時,;當分別對該使用 包括人臉及其上1 一第右置2〇α取得之右側影像 A .) , 1^;1; 得到一第一 2D 7二 U(m至"2Κ4))對應而分別 (切與一第)、一第-2D右眼中心座標 -第二右眼及ΐ上之-第二眉心影像通、 第四Β圖),並在即第二左眼巾处像923Β(參閱 w r))_ Γ·!象處理後,可與該第二座標(印切至 -控制部3G 二2D左眼W座標⑷,U; 心座標及該兩個2D左^兩m心座標、該_ 2d右眼中 公知技術,非本宰重點\^座^刀別進行3d座標線性轉換( (似)、—贅述〇,制—3D眉心座標 W’ze),其μ成—虛擬三角左,票 队从)可姆求中心點抓其三角中心點座標 可求出由該三角中心(亦即法線312) ’並 法線3⑵指向該畫面u3^量寧直線(亦即 座標位置; 乂點座“(尤上幻,其為該游標12之 201237674 當該使用者之注視方向移動時,該控制部30係透 ^^角私中心點311之法線312與該晝面11之交點,控制該 游軚12於该晝面U上移動’並設定該兩個右眼中心影像之 眼時間變化可輸出滑氣右鍵指令;且設定該 =目=間變化可輸出滑鼠左鍵指令,再設定該兩=、: 心影像之眨眼時晴化可輸出滑鼠中鍵指令;而達到 頭部控制游標之裝置。 遣 貫務上’該24個定位座標係包括以H2)、ρ狀4 Ζ)、 ^3(^3,^ζ3) > P4(x4,r4,z4) . p5(^)75jZ5) , Λ ^ ^ ^ 机认)、w;’z·。)' W|,[,Z|)、抓A y、 W4,„4)、W”yi5)、W6,W、w7,M7) 心 Λ W9’m) ”2°(V2°,Zm)、似、 A(4,&,z22)、P23(x23;,Z23)、ρ24(κ4,Ζ24)。 該右側影像擷取裝置2〇A可為攝影機; 該左側影像擷取裝置2〇b可為攝影機。 該第一、該第二眉心影像龍與921β皆位於 中心點位置(參閱第四Λ及第四Β圖); 離之 连二該第二右眼中心影像922Α * 922Β消失卜2秒再 生,灰化雜’可啟動滑鼠右鍵功能(例如為圏選)。 該$ —、該第二左眼中心影像923Α與923β消失1〜 產生,變化過程,可啟動滑鼠左鍵功能(例如為點擊}。 —Ϊ第Γ、該第二右眼中心影像922Α與92沈,以及該第- 、該第一左眼中心影像923Α與923Β同時消失卜2秒再雇㈣ 變化過程,可啟崎鼠中鍵功能。以作再產生的 本發明之方法部份則包括下列步驟(參閱第八圖): 一.準備步驟51 :準備一顯示器10、一影像摘 及一=(參閱第-Α及第-㈣),該== 游標12,該畫面11係供該游標12於其上^ 動’ _像#貞取裝置20係包括-右侧影像獅^置· 左側影像操取裝置2GB ; 衣直繼及- 201237674 二. 畫面座標設定步驟52 :該畫面η係包枯一 標(^。” -右上頂角座標以石)、^中备 W、-左下頂角座標(彻及一右巧= ^〇J〇,ZD) ; Μ Θ 度知 三. 擷取參考架體座標步驟53 :以該右、該 襄置20Α與20Β先朝-參考架體91拍攝,而^J取 影像201(參閱第二Α圖)與一第二影像202(參第一 由該晝面11之座標可量測該顯示器10與該參考 二 拍攝距離Z ;該參考架體91具有複數個(本發明為、,之 點911,該每一定健911 *具有一定位座標(‘似 A(U24,Z24)),其可對應於該晝面11的座標中,該二^‘ 二影像201與202對應該複數個定位座標^ ),而分別具有複數個(共24個)第—庙1 至)與複數個(共24個)第二座標ι }^至二2 四·擷取使用者眉心及兩目艮中心座標步 臉部92取代該參考架體91,且該右、 以用: 者臉部92拍攝,該右側影Si : 圏第一左眼中心_二=第= Τ,λΟ對應而分別得到一第一 2D眉 、(1至 2D右眼中心座標與一第一左 ☆,?) J二 影像擷取裝置20B取得之左伽马心紅x f ,3玄左側 心影像921Β、一第二右眼中二:H: J j J _(參閱第四B圖),並在即時影心^ (Ud、一第二2D左眼中、、、成神 t弟一』眉〜座才示 心座標队2屯); 中、座“队…與一第二2D左眼中 五進行3D座‘線性轉換步驟55 :該控制部㈨係以該兩 201237674 個2D眉心座標、兩個2D右眼中心座標及兩個肋左眼中心 標分別進行3D座標線性轉換(公知技術,非本案重點,恕 述!),得到一 3D眉心座標(m)、一 3D右眼中 (K’zj及- 3D左眼中心座標,其組成一虛擬 面31,並以前述三組座標求得該虛擬三角平面31經過其三 ^心點311(其三角中心點座標κ,Ζ/〇可相對求出)之法 量Ατ(ΑΛ,ΖΛί)(亦即法線312),並可求出由該三角中心點幻1 經該法向量直線(亦即法線312)指向該畫面u之 點座標(x,r,z),其為該游標12之座標位置; 六·三角中心點控制游標步驟56 :當該使用者之注視方 移動時,該控制部30係透過該三角中心點311201237674 VI. Description of the invention: [Technical field to which the invention pertains] This is a device and method for controlling a cursor and a method for controlling a cursor, which both have a moving face type and do not need to leave the keyboard, which is convenient and advantageous. The handicapped hand makes the hand π [previous technique] 'The more common Microsoft Windows system is to use the mouse 1 to execute the command, and use the computer to work all the time to face the button position. Moving from the keyboard to the mouse also means sliding the keys, which is cumbersome and time consuming. Shihan r, although there is a head-controlled cursor control device on the market, the model M2 leaky number can be assigned to the image tracking device = 疋ί must have two light sources on the head for the camera to capture two lights i The location is converted to the cursor position, such a device generates a problem: Threat [1] 3 must be fully charged. In the first time, the point-by-point control device needs to keep the power of the light source sufficient, and the use of the light source is not enough to cause the light to be weakened or even completely unable to emit light. 2] It is easy to cause head discomfort. Traditional head-controlled devices must control a set of light sources on the head, which can cause head discomfort when used for a long time. [3] Control is not easy. The selection requirements of the traditional head-controlled device, when the reservation is made, that is, the coordinate point is kept _, the button is selected by the system, so that the pressure of the user is quite large, because the movement must be re-positioned as long as it is slightly moved. . Therefore, control is not easy. In view of this, it is necessary to develop a technique that can solve the above disadvantages. SUMMARY OF THE INVENTION The purpose of the present invention is to provide a device and a method for controlling a cursor of a head, and to control a cursor with a moving face and a hand without leaving the keyboard. 201237674 = Hand: The advantage. In particular, the discomfort and control that you want to solve are not “. It is necessary to maintain sufficient power, and it is easy to cause the head and the segment to provide. The head control cursor is moved up to $, and the cursor is 'the face'. For the cursor, one left includes a center point coordinate, a right upper corner seat-image』: f;, two left lower corner coordinates and a lower right corner coordinate; the image picking device = set - right image capturing device And - the left side of the second and -=== positioning points: the distance between the two frames; the reference frame has a plurality of coordinate commands, with coordinates 'which can correspond to the right of the camera" The device takes the "central reputation" - the eye image, the right eye center image, and the - first-left eye face image, and after the instant image processing, can be confronted with the i: ij, the other piece - the second 2D , corresponding to 'and divided into two left eye center coordinates; ', the first 2 〇 left eye center coordinates and - the first heart unloading system with the two 2D eyebrow coordinates, the two 2 〇 right eye change,: linear rotation Center seated * U right eye center coordinates and a 3D left eye virtual: ί 平;; plane 'red of the above three sets of coordinates to obtain the - angle thousand faces _ The normal vector of the triangle center point, and the coordinates of the intersection of the normal vector line pointing to the picture by the triangle 201237674 can be obtained, which is the motion of the cursor and the key command of the two right eye center images; The left eye of the left eye can change the change of the slip between the eyes and the image of the eye can be outputted by the __ command; the method of the present invention partially includes the following steps: the first step of the medical knowledge, the preparation step, the setting of the face coordinate Step; take the reference frame coordinate step; take the steps of the user's eyebrow and the center coordinates of the two eyes; perform the 3D coordinate linear conversion step; the triangle center point control cursor step; and complete the steps. f The above-mentioned purposes and advantages are difficult to explain from the following to the _ towel, to gain a deep understanding. ^ stomach state (Pingz with the following implementation of the ship with the drawings detailed in the present invention: [Embodiment] The present invention is -_ The device and method for controlling the cursor, the reference, the -B, the second A, the third B and the fifth figure, the device part thereof comprises: a display ϋ 10 for displaying a picture U and a cursor 12, 11 series For the cursor 12 in it Move, and the face 1 line includes __中& mark (/.^^, a top right corner coordinate (4) heart ^ - top left corner stone:": - lower left corner coordinate (κΛ〇 and one lower right) The top corner seat ^ - the image reclining device 20, the system includes a right side image capturing device 2 and a left image capturing device 2 GB, and the right and left image hemp devices are respectively applied to the reference frame at the same position. 91 and the user's face 92, = 201237674 coordinates can be 4 _ ^ ytf ) = screen 11 a positioning point 911 has 24) positioning points 91 each of the 202 pairs should be a plurality of fixed =: image 201 There are a plurality of (24 total> to ^, U24), and a total of 24 (a total of 24) second choice ^ 庋裇 (4 teams to 4 (\4)) and multiple faces 92 When, respectively, the right image A.), 1^;1 obtained from the use of the face and the upper right side 2〇α is obtained for the first face 2D 7 (U to "2Κ4) Correspondingly, respectively (cut and one), one -2D right eye center coordinates - second right eye and squat - second eyebrow image pass, fourth map), and at the second left eyelid 923Β(see wr))_ Γ·! After the second coordinate (printed to - control part 3G 2D left eye W coordinate (4), U; heart coordinates and the two 2D left ^ two m center coordinates, the _ 2d right eye is known in the technology, non The main focus of the slaughter is ^3 seat ^ knife to carry out 3d coordinate linear conversion ((like), - 赘 〇 制, system - 3D eyebrow coordinate W'ze), its μ into - virtual triangle left, the ticket team from) Kemu seeking center point Grasping the coordinates of the center point of the triangle can be found from the center of the triangle (ie, the normal line 312) and the normal line 3 (2) points to the picture u3 ^ quantity of the line (that is, the coordinate position; the point seat " (especially the magic, which is The cursor 12 of 201237674, when the user's gaze direction moves, the control unit 30 controls the intersection of the normal line 311 of the private center point 311 and the face 11 to control the playway 12 on the face U. Move up and set the eye time change of the two right eye center images to output the air slide right command; and set the = target = change to output the left mouse button command, and then set the two =,: the blink of the heart image Time-sharing can output the middle button command of the mouse; and the device that controls the cursor of the head. On the continuation task, 'the 24 positioning coordinates include H2), ρ-like 4 Ζ), ^3(^3,^ζ3) > P4(x4,r4,z4) . p5(^)75jZ5) , Λ ^ ^ ^ Machine recognition), w; 'z·. )' W|,[,Z|), grab A y, W4, „4), W”yi5), W6, W, w7, M7) palpitations W9'm) ”2° (V2°, Zm), Like, A (4, &, z22), P23 (x23;, Z23), ρ24 (κ4, Ζ 24). The right image capturing device 2A can be a camera; the left image capturing device 2〇b can The first and second eyebrow image dragons and 921β are located at the center point position (refer to the fourth and fourth maps); the second right eye center image 922 Α * 922 Β disappears for 2 seconds Regeneration, ashing 'can start the right mouse button function (for example, for selection). The $-, the second left-eye center image 923Α and 923β disappear 1~ generated, change process, can start the left mouse button function (for example) For the click}. - Ϊ Γ, the second right eye center image 922 Α and 92 sink, and the first, the first left eye center image 923 Α and 923 Β disappear at the same time, 2 seconds re-employment (four) change process, can Kaiqi The mouse middle key function. The method of the present invention for regenerating includes the following steps (refer to the eighth figure): 1. Preparation step 51: preparing a display 10, an image picking one = (see paragraph - And - (4)), the == cursor 12, the screen 11 is for the cursor 12 on the moving _ image # 装置 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20衣直继和 - 201237674 II. Picture coordinate setting step 52: The picture η is a package with a standard (^.) - the top right corner coordinates are stone), ^ 中中W, - lower left corner coordinates (complete and one right Qiao = ^〇J〇,ZD) ; Μ Θ 知 三 3.. 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取 取(refer to the second figure) and a second image 202 (refer to the coordinates of the first surface 11 to measure the display 10 and the reference two shooting distance Z; the reference frame 91 has a plurality of (the present invention is Point 911, the fixed 911 * has a positioning coordinate ('like A (U24, Z24)), which can correspond to the coordinates of the face 11, the two images 201 and 202 correspond to a plurality of positioning coordinates ^), and respectively have a plurality of (a total of 24) first - temple 1 to) and a plurality of (a total of 24) second coordinates ι } ^ to two 2 4 · draw the user's eyebrows and two eyes艮 center The stepped face portion 92 replaces the reference frame body 91, and the right side is photographed by: the face portion 92, the right side shadow Si: 圏 the first left eye center _ two = the first = Τ, λ Ο corresponds to each of the first A 2D eyebrow, (1 to 2D right eye center coordinate and a first left ☆,?) J two image capturing device 20B obtained left gamma heart red xf, 3 Xuan left heart image 921 Β, a second right eye two: H : J j J _ (see Figure 4B), and in the instant shadow ^ (Ud, a second 2D left eye,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, , "Team... and a second 2D left eye, 5th, 3D seat, linear conversion step 55: The control unit (9) is based on the two 201237674 2D eyebrow coordinates, two 2D right eye center coordinates, and two rib left eye centers Standard 3D coordinate linear conversion (known technology, not the focus of this case, forgive! ), obtaining a 3D eyebrow coordinate (m), a 3D right eye (K'zj and - 3D left eye center coordinates, which constitute a virtual surface 31, and the virtual triangle plane 31 is obtained by the three sets of coordinates ^The heart point 311 (the triangle center point coordinates κ, Ζ / 〇 can be obtained relative to each other) the amount τ (ΑΛ, ΖΛί) (that is, the normal 312), and can be found from the triangle center point 1 The normal vector line (ie, normal 312) points to the point coordinate (x, r, z) of the picture u, which is the coordinate position of the cursor 12; the sixth triangle center point control cursor step 56: when the user's gaze When the square moves, the control unit 30 transmits the triangular center point 311

,畫面η之交點,控制該游標12於該畫面u U 疋该兩個右眼中㈣像之⑽時間變化可輸出滑鼠右鍵指令 ^設定該兩個左眼中心影像之魏時間變化可輸出滑鼠左鍵 和I ’再設定該義右、該_左眼巾^縣之眨眼時間變化 可輸出滑鼠中鍵指令;而達到頭部控制游標之裝置; 七·元成步驟57 :完成頭部控制游標之裝置。 實務上,該24個定位座標係包括似从)my、 户3(q,Z3)、W,M)、糾,y5,Z5)、W};,z6)、他純、 =(«,z8)、w、说义石)、、 从,½)、认,、W,5,n5,zl5)、W6,M6)、W7,W7) PM9,ZI9) > P2〇(Z2〇,720,Z20) > p2i(z2I,721,z2i) ^ 該右側影像擷取裝置20A可為攝影機; 該左側影像擷取裝置20B可為攝影機。 二ϊ(:二 =2:921B皆位於兩眉距離之 產:二====。烟 該第該第一左眼中心影像923A與923B消失1〜2秒再 201237674 產生的,化過私’可啟動滑鼠左鍵功能(例如為點擊)。 、ιί第二,二右眼中心影像922A與922β,以及該第一 923A ^ 9236 隻化過私,可啟動滑鼠中鍵功能。 、左側影像娜裝謹及施 片讀三度空晴時,取影片 此1 要說日㈣是,關於直接(3D座標)線性轉換之原理 L=P!Tar transformation,_DLT),係可利用兩部 的線ΐ關係來計算人體在空間的座標方法,稱為直接線 換ΐ數來ί 轉並依兩部攝影機的轉 ,数术„协空間座標。其計算攝賴常數的方法 個方程式計算求得Η個攝影機常數,計算式如下:The intersection of the screen η, the cursor 12 is controlled on the screen u U 疋 the two right eyes (4) (10) time change can output the right mouse button command ^ set the two left eye center image Wei time change can output the mouse Left button and I 'reset the right right, the left eye towel ^ county blink time change can output the mouse middle key command; and reach the head control cursor device; seven yuan into step 57: complete the head control The device of the cursor. In practice, the 24 positioning coordinates include: from my, household 3 (q, Z3), W, M), correct, y5, Z5), W};, z6), he pure, = («, z8 ), w, say Yi Shi),, from, 1⁄2), recognize, W, 5, n5, zl5), W6, M6), W7, W7) PM9, ZI9) > P2〇 (Z2〇, 720, Z20) > p2i(z2I, 721, z2i) ^ The right image capturing device 20A may be a camera; the left image capturing device 20B may be a camera. Two ϊ (: two = 2: 921B are located in the distance between the two eyebrows: two ====. The smoke of the first left eye center image 923A and 923B disappeared 1~2 seconds and then 201237674 produced, turned over private' The left mouse button function can be activated (for example, click). , ιί second, second right eye center image 922A and 922β, and the first 923A ^ 9236 can only be activated, and the middle mouse function can be activated. When I read the film and read it three times, I will take the film. This is to say that the day (4) is about the principle of direct (3D coordinate) linear conversion L=P!Tar transformation, _DLT), which can use two lines. ΐ 来 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算 计算Camera constant, the calculation formula is as follows:

ττ- ^ + BY + CZ + D ......................[1] v_hx+jy+kz+l ......................... 其中: U、V為影片上的平面座標。 X、Y、Z為實際丈量之立體空間座標。 A、B、C、D、E、F、G、H、I小K、L為線性嶋式之攝 影機常數。 由[1]、[2]式計算式,可解得下列聯立方程式: [AX + BY + CZ + D-UEX-UFY-UGZ = U \-vex-vfy-vgz + hx+jy+kz+l = v * · · * [3] 再將兩部攝影機影片上參考架(即本發明之參考架體⑴ 之所有參考點的兩組平面座標(W)(即複數個第—座標 (4(<,ί;)至名(尤24,}^)及("2名)(即複數個第二座標(尽(尤ο至 201237674 w4))與實際丈量之x、γ、z立體空間座標值代入⑶式 假設本實驗之參考架共有15個參考點 ,可用以下矩陣表示: 其代入[3]式後之運算 r 1 2 3 z z z 71 r2 y3 z' ^ Z3 t/.— %·· 1-2-3Ττ- ^ + BY + CZ + D .....................[1] v_hx+jy+kz+l ......... ................ Where: U, V are the plane coordinates on the film. X, Y, and Z are the three-dimensional space coordinates of actual measurement. A, B, C, D, E, F, G, H, I small K, L are linear camera-type camera constants. The following equations can be solved by the formulas [1] and [2]: [AX + BY + CZ + D-UEX-UFY-UGZ = U \-vex-vfy-vgz + hx+jy+kz+ l = v * · · * [3] Two sets of plane coordinates (W) of all the reference points of the reference frame (1) of the present invention (ie, multiple coordinate coordinates (4) <, ί;) to the name (especially 24,}^) and (" 2) (that is, the plural second coordinates (extra (ou ο to 201237674 w4)) and the actual measurement of x, γ, z stereo space Substituting the coordinate value into equation (3) Assume that the reference frame of this experiment has 15 reference points, which can be represented by the following matrix: The operation after substituting [3] is r 1 2 3 zzz 71 r2 y3 z' ^ Z3 t/.— %·· 1-2-3

KlK2K3ns z 715o o o o o 15 o o o 1 ο ο o ¾¾ ι 2 3 z z z z z z z ......c/.,f^2r3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 y, Z_ 1 Z2 1 Z3 1 0 0 0 〇 ^15 ~F15ris _ 15 ^15 ^,5 Zl5 Γ u. 1 1 u, B u' c 3 D E r r • F = "15 G K H 1 J [k /,5 再將此矩陣如最何方法(ι_square)求得11伛數。並將[3]式整理如下: -ΊΙ〇Λ7 - γ/_ η• [4] 最後將已求得的常數及已知的兩部攝影機影片上參3 之所有參考點的兩組平面座標(⑽及(哪分別代入⑷3 形成下列之[5]式與[6]式’接著以最,卜平方法的矩陣解法 下列之[7]式)求得χ、γ、ζ(即本發明之游標12的交點』 (i,r,z))KlK2K3ns z 715o oooo 15 ooo 1 ο ο o 3⁄43⁄4 ι 2 3 zzzzzzz ......c/.,f^2r3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 y, Z_ 1 Z2 1 Z3 1 0 0 0 〇^15 ~F15ris _ 15 ^15 ^,5 Zl5 Γ u. 1 1 u, B u' c 3 DE rr • F = "15 GKH 1 J [k /,5 then this matrix For example, the most method (ι_square) finds 11 turns. And [3] is organized as follows: -ΊΙ〇Λ7 - γ/_ η• [4] Finally, the obtained constants and the two sets of plane coordinates of all the reference points of the known two camera films ( (10) and (Which substituting (4) 3 respectively form the following [5] and [6] formulas] and then using the matrix solution of the most, the method of sumping the following [7] formula) to obtain χ, γ, ζ (ie, the cursor of the present invention) The intersection of 12" (i, r, z))

L ..[5]L ..[5]

[{A - U2E)X + (B~ U2F)Y + (C - U2G)Z = u2-d { (H - V2E)X + (j _ y^Y + (A: _ v^z = v L ..[6] [7] [7]201237674 'a~EUx B-EU' C-EU,' ~ y~ _u'-D- ff-EV' I-EV' J-EV{ yl V K~k a-eu2 b-eu2 c-eu2 1 7 u2~d β-εν2 i-ev2 j-ev2 La 關於本發明之3D座標線性轉換過程,於開始後可包括下 列步驟: 步驟AK61):設定畫面上之中心點座標(ά,ζ<))、一右上 頂角座、一左上頂角座標(\,4石)、一左下頂角座 標(尤c,4,zc)及一右下頂角座標(j\rD,yfl,Z/j。 步驟A2(62):藉前述座標設定一參考架體與晝面間的拍攝 距離Z,該參考架體具有24個定位點(具有24個定位座標 (以^/^至/^心匕石:^’其可對應於該晝面的座標中。 步驟A3(63):以右側及左側攝影機拍攝參考架體的24個 定位點。 步驟A4(64):右側攝影機得到一第一影像,該第一影像對 應該24個定位座標(肌„)至,❿分別具有24 個第一座標UK,);)至4¾,匕))。 步驟A5(65):左侧攝影機得到一第二影像,該第二影像斜 應該24個定位座標(肌„)至似…仏)),*分別具有μ 個第二座標(4(«)至524(^;))。 步驟Α6(66):將24個定位座標(肌以)至讲Λ,〜)、 24個第一座標至W4,r24))及24個第二座標 至屯(/24;))三筆資料進行最小平方法線性轉換。 12 201237674 步驟A7(67).轉求得最小平方法之雜轉換係數。 v驟A8(68) ·將線性轉換係數儲存於資料庫,最後結束。 於臉雜购_過程,闕始後可包括下列步驟(來 閱第十圖): v驟BK71).讀取線性轉換係數(可從資料庫讀取)。 ' ()右、左側影像操取裝置同步拍攝使用者臉部 影像。 步驟B3(73):臉部影像辨識。 步驟M(74):分棚取右、左側影像操取裝置拍攝之使用 者臉部影像上之2D眉心座標、2D右眼巾㈣標及2])左眼中心 座標。 步驟B5C75):進行3D座標線性轉換。 步驟B6(76):取得3D眉心座標(m)、3D魏中心座 ^(K,ZA)及3D左眼中心座標(m),其組成—虛擬三角平 面。 步驟B7(77):以3D眉心座標(u,ZJ、3D右眼中心座標 (Κλ)及3D左眼中心座標(m)求得虛擬三角平面之二角 中心點(三角中心點座標可相對求出)。 步驟B8(78):計算求得虛擬三角平面經過三角中心點之 法句量 Ν{Χν,Υν,Ζν)。 步驟Β9(79):計算求得經過三角中心點之法向量 #(\,}^〃)’並直線指向晝面之交點座標(x,r z),其為游梗。 13 201237674 步驟B10(80):以臉上的虛擬三角平面取代滑鼠,押制轉 標於晝面上移動。 步驟B1K81):判別是否收到按鍵指令?如判別為“否,, ,則回到步驟B2(72)重新動作,如判別為“是’,,則進彳_下 步驟。 步驟B12(82):判別是否收到左(右)眼閉眼(1〜2秒)指人 ?如判別為“否”,則進行步驟B13(83)動作,如判別為 ”,則進行步驟B14(84)動作。 步驟B13C83):判別是否進行左(右)眼閉眼(1〜2秒)指令 ?如判別為“否’,,則回到步驟B2(72)重新動作,如判別為 “是”,亦進行步驟B14(84)。 步驟B14(84):執行指令工作,並在執行後結束。 亦即,本發明先在用以供該游標12移動的該顯示器之 該晝面11上,設定中心點座標(„石)、右上頂角座標(以❸ 、左上頂角座標(V«,zs)、左下頂角座標氏,rc,Zc)及右下頂角 座標(ΑΛ,ζβ);再以設在該顯示器1〇上的該右、該左側影像 掏取裝置2GA、2GB(參閱第—Α及第—Β圖),魅該顯示器 10前方的該參考架體91(具有可對應於該畫面u之座標的% 個疋位座標/^,Z|)至户从认))進行拍攝,而分別從該畫 面11的右方與左方,取得該參核體91的24個第一座標 “(^)至W4,r24))與24個第二座標(咐而至尽从乂))。 接著以該個麵部92取代够考· 91的位置,並使該右 201237674 、該左側影像掏取裝置2〇A與20B皆朝該使用者臉部 ’而可從該hU的右方,在24個第—賴似 ΛΚ4))上對應出第一 2D眉心座標(Ά)、第一扣右眼中 心座標⑷❿與第一 2D左眼中心座標⑷乂);並從該晝面'1 的左方’在24個第二座標(肌而至队心)上對應出第二 2D眉心座標(κ2)、第二2D右眼中心座標(心心與第 左眼中心座標(H2)。 — 再由該控制部30以兩個2D眉心座標、兩個2D右眼中、、 座標及兩個2D左眼k座標共同進行3D座標線性轉換,而^ 到由3D眉心座標似乂〕、3D右眼中心座標(以,城3d左^ 中心座k(K’zj組成的虛擬三角平面31,並可求得經過其_ 角中心點311(三角中心點座標any可相對求出)之法向 量#(心,心,2〃)(亦即法線312) ’再求出由該三角中心點311經 該法向量直線(亦即法線312)指向該畫面u之交點 座標(x,r,z) ’其為該游標12 ;如此,當該使用者臉部92移動 時(參閱第六圖’例如由右朝左轉頭,座標部份改變成一移動 後3D眉心座標(m,)、一移動後3D右眼中心座標队‘ 、一移動後3D左眼中心座標及一移動後三角中心點座標 ’即透過該二角中心點311 ’控制該游標12於該 畫面11上同步移動(參閱第六圖’例如由原本的一第一位置Ρ1 移動到一第二位置Ρ2,座標變成一移動後交點座標(Χ|;Ζ)), ji設定由雙眼的動作(例如閉住左眼1〜2秒,使該第一、該第[{A - U2E)X + (B~ U2F)Y + (C - U2G)Z = u2-d { (H - V2E)X + (j _ y^Y + (A: _ v^z = v L ..[6] [7] [7]201237674 'a~EUx B-EU' C-EU,' ~ y~ _u'-D- ff-EV' I-EV' J-EV{ yl VK~k a -eu2 b-eu2 c-eu2 1 7 u2~d β-εν2 i-ev2 j-ev2 La With regard to the 3D coordinate linear conversion process of the present invention, the following steps may be included after the start: Step AK61): setting the center on the screen Point coordinates (ά, ζ <)), a top right corner seat, a top left corner coordinate (\, 4 stone), a left lower corner coordinate (eu c, 4, zc) and a lower right corner coordinate (j \rD, yfl, Z/j. Step A2 (62): Set the shooting distance Z between the reference frame and the face by the coordinates, the reference frame has 24 positioning points (with 24 positioning coordinates (to ^ /^至/^心匕石:^' It can correspond to the coordinates of the face. Step A3 (63): Take the 24 positioning points of the reference frame with the right and left cameras. Step A4 (64): Right The camera obtains a first image corresponding to 24 positioning coordinates (muscles) to ❿, respectively having 24 first coordinates UK, );) to 43⁄4, 匕)). Step A5 (65): left Side shot Machine to obtain a second image, the second image 24 should be obliquely positioned coordinates (muscle ") seems to Fo ...)), each having μ * a second coordinate (4 («) to 524 (^;)). Step Α6(66): Set 24 positioning coordinates (muscle) to Speaking, ~), 24 first coordinates to W4, r24)) and 24 second coordinates to 屯 (/24;)) Perform a least squares linear transformation. 12 201237674 Step A7 (67). Transfer the minimum conversion factor of the least squares method. vStep A8 (68) • The linear conversion coefficients are stored in the database and finally ended. In the face purchase process, the following steps can be included after the start (see the tenth figure): v (BK71). Read the linear conversion coefficient (can be read from the database). ' () The right and left image manipulation devices simultaneously capture the user's face image. Step B3 (73): Face image recognition. Step M (74): Take the 2D eyebrow coordinate on the face image of the user's face image taken by the right and left image manipulation device, 2D right eye towel (4) mark and 2]) left eye center coordinate. Step B5C75): Perform 3D coordinate linear conversion. Step B6 (76): Obtain 3D eyebrow coordinate (m), 3D Wei center seat ^ (K, ZA) and 3D left eye center coordinate (m), which constitute a virtual triangle plane. Step B7 (77): Find the center point of the virtual triangle plane with the 3D eyebrow coordinate (u, ZJ, 3D right eye center coordinate (Κλ) and 3D left eye center coordinate (m) (the triangle center point coordinate can be relative Step B8 (78): Calculate the sentence Ν {Χν, Υν, Ζν) of the virtual triangle plane passing through the triangle center point. Step Β9(79): Calculate the normal vector #(\,}^〃)' that passes through the center point of the triangle and point straight to the coordinate of the intersection point (x, r z), which is the stalk. 13 201237674 Step B10 (80): Replace the mouse with the virtual triangle plane on the face and move the index on the face. Step B1K81): Determine whether a button command is received? If the determination is "NO,", the process returns to step B2 (72) to re-operate. If the determination is "Yes", the process proceeds to the next step. Step B12 (82): It is determined whether or not the left (right) eye closed eye (1 to 2 seconds) is received. If the determination is "NO", the operation proceeds to step B13 (83). If the determination is "", then step B14 is performed. (84) Operation Step B13C83): It is determined whether or not the left (right) eye closed eye (1 to 2 seconds) command is issued. If the determination is "NO", the process returns to step B2 (72) to re-act, if it is judged as "Yes" ", also proceed to step B14 (84). Step B14 (84): The instruction operation is executed and ends after execution. That is, the present invention first sets the coordinates of the center point (the stone) and the coordinates of the upper right corner of the top surface 11 of the display for the movement of the cursor 12 (with ❸, upper left corner coordinates (V«, zs). ), the lower left corner coordinate, rc, Zc) and the lower right corner coordinate (ΑΛ, ζβ); and the right and left image capturing device 2GA, 2GB provided on the display 1〇 (see the first Α 第 第 第 , , , , 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器 显示器From the right and left sides of the screen 11, respectively, the 24 first coordinates "(^) to W4, r24)) and the 24 second coordinates (from the top) of the reference body 91 are obtained. . Then, the position of the test panel 91 is replaced by the face portion 92, and the right side 201237674 and the left image capturing device 2A and 20B are both facing the user's face and can be from the right side of the hU. The 24th - 赖 ΛΚ 4)) corresponds to the first 2D eyebrow coordinate (Ά), the first buckle right eye center coordinate (4) ❿ and the first 2D left eye center coordinate (4) ;); and from the left side of the '面 '1 The square' corresponds to the second 2D eyebrow coordinate (κ2) and the second 2D right eye center coordinate (the heart and the left eye center coordinate (H2) on the 24 second coordinates (muscle to the team center). The control unit 30 performs 3D coordinate linear conversion together with two 2D eyebrow coordinates, two 2D right eye, coordinates, and two 2D left eye k coordinates, and the 3D right eye center coordinates are obtained from the 3D eyebrow coordinate coordinates and the 3D right eye center coordinates ( So, the city 3d left ^ center block k (K'zj composed of the virtual triangle plane 31, and can obtain the normal vector through its _ corner center point 311 (the triangle center point coordinates can be relatively calculated) #心,心, 2〃) (ie, normal 312) 'Recalculate the coordinates of the intersection of the triangle center point 311 through the normal vector line (ie, normal 312) pointing to the picture u (x r, z) 'It is the cursor 12; thus, when the user's face 92 moves (see the sixth figure 'for example, turning from right to left, the coordinate portion is changed to a moving 3D eyebrow coordinate (m,)) After moving, the 3D right-eye center coordinate team', a moving 3D left-eye center coordinate, and a moving rear triangular center point coordinate' control the cursor 12 to move synchronously on the screen 11 through the two-corner center point 311' ( Referring to the sixth figure 'for example, moving from a first position Ρ1 to a second position Ρ2, the coordinates become a moving point coordinate (Χ|;Ζ)), ji set the action by both eyes (for example, closing the left eye) 1~2 seconds to make the first, the first

S 15 201237674 二^中心影像923A與923B消失Η秒再出現)輸出滑鼠左 建扎々(參閱第七圖’假設使游標12變成執行狀態的游標似) 對游仏12產生控制指令(此為公知可達成之技術,恕不費述 )° 本發明之優點及功效可歸納如下: ⑴移動臉部即可控制游標。本發明先以兩個攝影機由不 同角度分別對—參考架體棟取兩組(雜24個)定位座標,之 後將參考架體移開,讓個者臉部糊縣參考架體的位置, 再以同角度的兩個攝影機,分別擷取臉部對應在24個定位座 標上的兩組2D眉心座標、兩組2D右眼中心座標及兩組肋左 眼中心座標’即可以這6組座標進行3[)座標線性轉換(公知技 術,恕不贅述);而得到一與使用者臉部相對應的虛擬三角平 面’並可進-步求得其三角巾^點經法向量麟指向該晝面之 父點座軚(亦即游標所在之座標),如此當使用者臉部移動(更 详細的講應是眉心及兩眼中心移動)即可驅動游標在晝面上移 動,並可設定由兩眼動作控制游標執行相關命令(例如圈選或 是點擊)。 [2]手部不需離開鍵盤相當方便。在習慣一般輸入法後( 不管是注音、倉頡、大易·.·),手指習慣性會位於鍵盤的 預疋按鍵上,這樣才能在不看鍵盤(因按鍵的位置已大致背起 來了)的情況下直接打字,而本發明能讓使用者在手部不用離 開鍵盤的情況下,透過移動臉部以及開、閉雙眼,達到控制游 16 201237674 ‘移動及執彳了命令’可省略手部在鍵盤與料間赫的次數與 時間(移動到滑鼠還㈣手指分別再按於左、右及中間鍵上), 相當方便。 [3]利於手部殘缺者使用。部份手指有殘缺或是根本沒有 手部(例域域障献肤事㈣失^部)的人,並沒有辦 法使用滑鼠(除錄義肢或是其解貴_置)控辦標,本發 明只要讓使用麵動臉部並開、暖眼,即可移動ii控制游標 ,利於手部殘缺者使用。 以上僅是藉由較佳實施例詳細說明本發明,對於該實施例 所做的任何簡單修改與變化,皆不脫離本發明之 【圖式簡單制】 图 第A,係本發明之右側影像操取裝置操取參考架體影之 示意圖 ” 第一 B ®係本發明之左側影像擷取裝置擷取參考架 示意圖 /' Α ίΓίΓ月之f側影侧取裝置擷取之參考架體影像 對應於晝面座標之示意圖 > B圖係本發明之左侧影像擷取裝置擷取之 對應於晝面座標之示_ % H Α圖^發明之右懈彡賴轉置娜使財臉部影像 之示意圖 ' B圖,本發明之左側影像掏取裝置擁取使用者臉部影像 之不意圖 第四A _本發明之右侧影像錄裝置娜之使用者 制艮中心影像對應於參考架體之定位座標之示专圖 四B圖係本發明之左側影像掏取裝置擷取之使用者屑心及 第 第 第三 第三 :體影像之 201237674 兩眼中心影像對應於參考架體之定位座桿之干立圖 第五圖係本發明之虛擬三角平面 之賴的示意圖 两^點直線指向晝面 第六,係士發明之使用者臉部移動而驅動游標於書面 的示意圖 一 =七,係本發明之眼中心睛動作控制游標執行命令之示意圖 第八圖係本發明之方法之流程圖 第九圖係本發明之線性轉換係數演算程序之流裎圖 第十圖係本發明之臉部控制游移動及執行命令之 圖 【主要元件符號說明】 11晝面 12A執行狀態的游標 20A右側影像擷取裝置 201第一影像 30控制部 311三角中心點 51準備步驟 1 〇顯示器 12游標 20影像擷取裝置 20B左側影像擷取裝置 202第二影像 31虛擬三角平面 312法線 52晝面座標設定步驟 53擷取參考架體座標夕驟 54擷取使用者眉心及兩眼中心座標步驟 55進行3D座標線性轉換步驟 56三角中心點控制游標步驟57完成步驟 61步驟A1 63步驟A3 65步驟A5 67步驟A7 Ή步驟B1 73步驟Β3 75步驟Β5 Ή步驟Β7 79步驟Β9 62步驟Α2 64步驟Α4 66步驟Α6 68步驟Α8 72步驟Β2 74步驟Β4 76步驟Β6 78步驟Β8 80步驟Β10 201237674 81步驟Bll 83步驟B13 91參考架體 92使用者臉部 922A第一右眼中心影像 921B第二眉心影像 923B第二左眼中心影像 P1第一位置 (HZ。)中心點座標 左上頂角座標 (心Γβ,ζβ)右下頂角座標 82步驟Β12 84步驟Β14 911定位點 921Α第一眉心影像 923Α第一左眼中心影像 922Β第二右眼中心影像 Ζ拍攝距離 Ρ2第二位置 右上頂角座標 (xc,rc,zc)左下頂角座標 Ρ,{Χχ,ΥνΖχ) ' Ρ2{Χ2,Υ2,Ζ2) ' Ρ,(Χ3,Υ3,Ζ,) ' Ρ4(Χ4,Γ4,Ζ4) ' P5(X5J5,ZS) ' Ρ6(Χ6,Υ6,Ζ6) ' Ρ7(Χ7,ΥΊ,ΖΊ) ' ρ^χ,,γ,,ζ,) ' Ρ9(Χ9,Υ9,Ζ9) ' Pl0(Xw,Yw,ZJ 'S 15 201237674 2nd center image 923A and 923B disappears and the second appears again) Output mouse left construction 々 (refer to the seventh figure 'Assumed to make the cursor 12 into the execution state of the cursor) to generate control commands for the cursor 12 (this is Known achievable technologies, without mentioning) The advantages and effects of the present invention can be summarized as follows: (1) The cursor can be controlled by moving the face. The invention firstly uses two cameras to respectively perform different angles from different angles - the reference frame body takes two sets (24 pieces of miscellaneous) positioning coordinates, and then the reference frame body is removed, so that the face paste county refers to the position of the frame body, and then Two sets of 2D eyebrow coordinates corresponding to the 24 positioning coordinates, two sets of 2D right eye center coordinates and two sets of rib left eye center coordinates of the two cameras at the same angle can be used for the six sets of coordinates. 3[) coordinate linear transformation (known in the art, not to be described); and a virtual triangular plane corresponding to the user's face is obtained, and the triangle towel can be obtained step by step. The parent point 軚 (that is, the coordinates of the cursor), so when the user's face moves (more detailed should be the eyebrow and the center of the two eyes move) can drive the cursor to move on the face, and can be set by two The eye movement control cursor executes related commands (such as circle or click). [2] It is quite convenient for the hand to leave the keyboard. After getting used to the general input method (whether it is Zhuyin, Cangjie, Da Yi···), the finger habit will be on the pre-key of the keyboard, so that you can not look at the keyboard (because the position of the button has been roughly back up) In the case of direct typing, the present invention allows the user to control the swim by moving the face and opening and closing the eyes without leaving the keyboard. 201237674 'Moving and executing the command' can omit the hand The number and time between the keyboard and the material (moving to the mouse (4) and then pressing the left, right and middle buttons respectively) is quite convenient. [3] Conducive to the use of handicapped persons. Some of the fingers are defective or there is no hand at all (such as the domain barrier (4) lost), and there is no way to use the mouse (except for recording the prosthetic or its solution). In the invention, as long as the face is moved and the face is warmed, the cursor can be moved ii to facilitate the use of the handicapped hand. The above is only the detailed description of the present invention by way of a preferred embodiment, and any simple modifications and changes made to the embodiment are omitted from the present invention. The first B ® is the left side image capturing device of the present invention, and the reference frame image is taken from the left side image capturing device of the present invention. 参考 Γ Γ Γ 之 侧 侧 侧 侧 侧 侧 侧 侧 侧 侧 参考 参考 参考The schematic diagram of the surface coordinates > B is a schematic diagram of the left side image capturing device of the present invention corresponding to the kneading coordinates _ % H Α 图 ^ The invention of the right lazy 彡 转 置 娜 使 使 使 使 脸 脸 脸 脸'B, the left image capturing device of the present invention does not intend to capture the user's face image. Fourth A_ The right image recording device of the present invention is designed to correspond to the positioning coordinates of the reference frame. The figure 4B is the user's core of the left image capturing device of the present invention and the third and third body image 201237674. The center image of the two eyes corresponds to the positioning of the reference frame. The fifth picture of the photo is The schematic diagram of the virtual triangle plane of the invention is two points of the straight line pointing to the sixth surface of the skull, and the user's face movement of the invention is driven to drive the cursor in a written diagram of the figure = seven, which is performed by the eye center movement control cursor of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 8 is a flow chart of a method of the present invention. FIG. 9 is a flow chart of a linear conversion coefficient calculation program of the present invention. FIG. 10 is a diagram of a face control movement and execution command of the present invention. DESCRIPTION OF REFERENCE NUMERALS 11 cursor 12A execution state cursor 20A right image capturing device 201 first image 30 control unit 311 triangular center point 51 preparation step 1 〇 display 12 cursor 20 image capturing device 20B left image capturing device 202 second Image 31 virtual triangle plane 312 normal line 52 face coordinate setting step 53 capture reference frame coordinate step 54 capture user eyebrow and two eye center coordinates step 55 to perform 3D coordinate linear conversion step 56 triangle center point control cursor step 57 Complete step 61 Step A1 63 Step A3 65 Step A5 67 Step A7 Step B1 73 Step Β 3 75 Step Β 5 Ή Step Β 7 79 Step Β 9 62 Step Α 2 64 steps Α 4 66 steps Α 6 68 steps Α 8 72 steps Β 2 74 steps Β 4 76 steps Β 6 78 steps Β 8 80 steps Β 10 201237674 81 steps B ll 83 steps B13 91 reference frame 92 user face 922A first right eye center image 921B second Eyebrow image 923B Second left eye center image P1 first position (HZ.) Center point coordinate Left upper corner coordinate (heart Γβ, ζβ) Lower right corner coordinate 82 Step 8412 84 Step Β14 911 Positioning point 921ΑFirst eyebrow image 923Α First left eye center image 922 Β second right eye center image Ζ shooting distance Ρ 2 second position upper right apex coordinate (xc, rc, zc) lower left corner coordinate Ρ, {Χχ, ΥνΖχ) ' Ρ 2{Χ2, Υ 2, Ζ 2 ) ' Ρ,(Χ3,Υ3,Ζ,) ' Ρ4(Χ4,Γ4,Ζ4) ' P5(X5J5,ZS) ' Ρ6(Χ6,Υ6,Ζ6) ' Ρ7(Χ7,ΥΊ,ΖΊ) ' ρ^χ ,,γ,,ζ,) ' Ρ9(Χ9,Υ9,Ζ9) ' Pl0(Xw,Yw,ZJ '

Pn(Xn,Yn,Zu) ' Pn(Xn,Yi2,Zi2) ' ^3(^,3^3^.3) ' ^4(^4^4^,4) ' ^5(^,5^5^,5) ("^16 5 ^16' ^16) ' ^7(^17^7^17) ' ' ^9(^19^95^19) ^20 ("^20 >^20 ^20) ' ^21(^21^1^21) ' ^22(^22^2^22) ' ^23(^235^35^23) ' p24(x24,r24,z24)定位座標 冬(/,,]^)〜為4(义24,724)第一座標 5,(1,4)〜524(x24,;r24)第二座標 (AA)第一 2D眉心座標 (‘&,)第一 2D右眼中心座標 (K,)第一 2D左眼中心座標(uj第二2D眉心座標 (xfl2,rfl2)第二2D左眼中心座標(xC2,fC2)第二2D左眼中心座標 (m)3D眉心座標 d,H)移動後3D眉心座標 (K,zA)3D右眼中心座標 (m,)移動後3D右眼中心座標 (m) 3D左眼中心座標 (χεΙ,ΓεΙ,ζΗ)移動後3D左眼中心座標 尸(υ,,,Ζρ)三角中心點座標 移動後三角中心點座標 (x,r,z)交點座標 (ιπ,ζ,)移動後交點座標 #(八,};,4)法向量 19Pn(Xn,Yn,Zu) ' Pn(Xn,Yi2,Zi2) ' ^3(^,3^3^.3) ' ^4(^4^4^,4) ' ^5(^,5^ 5^,5) ("^16 5 ^16' ^16) ' ^7(^17^7^17) ' ' ^9(^19^95^19) ^20 ("^20 >^ 20 ^20) ' ^21(^21^1^21) ' ^22(^22^2^22) ' ^23(^235^35^23) ' p24(x24,r24,z24) locating coordinates winter ( /,,]^)~ is 4 (Yi 24, 724) first coordinate 5, (1, 4) ~ 524 (x24,; r24) second coordinate (AA) first 2D eyebrow coordinate ('&,) first 2D right eye center coordinate (K,) first 2D left eye center coordinate (uj second 2D eyebrow coordinate (xfl2, rfl2) second 2D left eye center coordinate (xC2, fC2) second 2D left eye center coordinate (m) 3D eyebrow coordinate d, H) 3D eyebrow coordinate after movement (K, zA) 3D right eye center coordinate (m,) 3D right eye center coordinate (m) 3D left eye center coordinate (χεΙ, ΓεΙ, ζΗ) after moving 3D left eye center coordinates corpse (υ,,,Ζρ) triangle center point coordinates movement rear triangle center point coordinates (x, r, z) intersection coordinates (ιπ, ζ,) movement after intersection coordinates # (eight,};, 4 ) normal vector 19

Claims (1)

201237674 七申凊專利範圍·· 1 ’ ”制游標之裝置,其包括: 游標移:標,該畫面係供該 頂角座標、-了 =座標、一右上 角座標; 月弘左下頂角座標及一右下頂 影像擷裝t係=一 f側影像掏取裝置及一左側 C上的參考架體與使用者臉部進ifi ίίίϊ =,,相產生一第一影像與⑵像=: n f‘可量測該顯示器與該參考架體間之拍攝距離广; t,;;,定位點,該每-定2 對應該複數個定位座標,=有像 ; #靖姻姻 即時與健¥4目、〜影像及—第—左眼中^影像,並在 ”理後’可與該第一座標對 : t 二=左側衫像掏取裝置取得之左側影像包括人臉及其上 心二二影像、—第二右眼中心影像及-第二左眼中 丨ίί即_像處理後’可與該第二座標對應,而 —第座標、—第二2D左眼巾心座標與 、:控制部’係以該兩個2D屑心座標、該兩個2D右眼 =f及該兩個2D左眼中心座標分別進行3D座標線性 、,得到一 3D眉心座標、一 3D右眼中心座標及一 3D左眼 二„其組成―虛擬三角平面’並以前述三組座標求 亥^擬二角平面經過其三角中心點之法向量,並可求出 该二角令心點經該法向量直線指向該畫面之交點座標, 20 201237674 其為該游標之座標位置; 今藉=當該者之注視方向移動時,該控卿係透過 til心.ί線與該晝面之交點,控制該游標於該畫 輪出巧^^該兩個右眼中心影像之眨眼時間變化可 f ΐ 2ίΐί r且設定該_左眼中心影像之魏時 “鼠鍵指令’再設定該義右、該兩個左 2 頭部控制游標之裝置。“鼠中私令,而達到 其中 •如申請專利範圍第1項所述之頭部控制游標之裝置, 該右側影像擷取裝置係為攝影機; 該左侧影像擷取裝置係為攝影機。 3·如申請專利範圍第i項所述之頭部控制游標之裝置,其令 亥第-、該第二眉心影像皆位於兩眉距離之巾心點 9 ,第-、該第二右眼中心影像消失卜 過程,可啟動滑鼠右鍵功能; 、該第二左眼中心影像消失卜 過程,可啟動滑鼠左鍵舰; ㈣座玍敝化 »亥第一、該第一右眼中心影像以及該一一 目同時消失1〜2秒再產生的變化過程,;“滑 4 . •種,部控制游標之方法,其包括下列步驟: •準=步驟··準備—顯示器、—影像娜裝置及一控制部 伊Stir顯示—畫面及一游標,該畫面係供該游 Γ'i1 ,該影像擷取裝置係包括一右側影像擷取 裝置及一左側影像擷取裝置; 21 1 票設定步驟:該晝面係包括-t心點座標、-右 頂角座標、一左上頂角座標、一左下頂角座標及一右 201237674 下頂角座標; 有複數個第-座標與複數個第二座標; 刀I、 i代該:及:以"'使用者臉部 -4 且該右、省左側衫像擷取裝置皆鉬兮 者臉#拍攝’該右娜像擷取 = J括人臉及其上之一第一眉心影像、一丨影J iiH:中心影像,並在即時影像處理‘,ΐϊ 二眉心座標、-口 ΐ時中心影像及一第二左眼中上S 第Id,處,可與該第二座標對應’而分別得ί-標、-第二2D左眼中心座標與-第二2D 五 '#i,J~3! 肀座輮及一 3D左眼中心座標,其組成一 角中二組座標求得該虛擬三角平面經過2 置直線指向該畫面之交點座標,其為該游標 標步驟:當該使用者之注視方向# 控制。晴過該三角中心點之法線與該晝面:j 22 201237674 點,控制該游標於該晝面上移動,並設定該兩個右眼中 心影像之眨眼時間變化可輸出滑鼠右鍵指令;且設定該 兩個左眼中心影像之眨眼時間變化可輸出滑鼠左鍵指令 ’再設定該兩個右、該兩個左眼中心影像之眨眼時間變 5 一:匕:=骨5中鍵指令;而達到頭部控制游標之裝置; 七疋成步驟.元成頭部控制游標之裝置。 •如申請專概圍第4項所述之頭部控辦標之方法,1 該右側影像擷取裝置係為攝影機; ,、中· 該左側影像擷取裝置係為攝影機。 .如ίϊΐ利5·第4項所述之頭部控制游標之方法,其中: ‘第-、二=2心影像皆位於兩眉轉之中心點位置; 程,可啟“ 影職失1〜2秒再產生的變化過 程 中心影像同時及該第-、 鍵功能。 v再產生的變化過程,可啟動滑鼠中 1啟動嘴消失1〜2秒再產生的變化過 該第二左眼201237674 七申凊 patent scope·· 1 ' ” vernier device, including: vernier shift: standard, the picture is for the apex coordinate, - = = coordinate, a right upper corner coordinate; Yue Hong left lower corner coordinates and A right lower top image armor t system = a f side image capturing device and a reference frame on the left side C and the user's face into ifi ίίίϊ =, the phase produces a first image and (2) image =: nf' The shooting distance between the display and the reference frame can be measured; t,;;, the positioning point, the each of the two pairs should be a plurality of positioning coordinates, = like; #靖婚婚真 and 健¥4目, ~ image and - first - left eye ^ image, and "after" can be associated with the first coordinate: t 2 = left side of the image obtained by the capture device on the left side of the image including the face and the upper heart image, - the second right eye center image and - the second left eye 丨 ίί ie _ after processing ' can correspond to the second coordinate, and - the first coordinate, - the second 2D left eye towel heart coordinates and: control department' 3D coordinates are respectively performed by the two 2D chip coordinates, the two 2D right eyes=f, and the two 2D left eye center coordinates Linear, get a 3D eyebrow coordinate, a 3D right eye center coordinate and a 3D left eye two „the composition of the virtual triangle plane' and use the above three sets of coordinates to find the normal vector of the triangle plane through its triangular center point And obtaining the coordinates of the intersection point of the heart point passing the normal vector line to the picture, 20 201237674 which is the coordinate position of the cursor; now borrowing = when the direction of the person moves, the control system is through Tul heart. ί line and the intersection of the face, control the cursor on the wheel to make ^^ the two right eye center image of the blink time change can f ΐ 2ίΐί r and set the _ left eye center image Wei Shi The "mouse key command" is used to set the right and the left 2 head control cursors. "The mouse is in private order and reaches the device." The device for controlling the head of the cursor as described in claim 1 of the patent scope, the right image capturing device is a camera; the left image capturing device is a camera. The device for controlling the cursor of the head according to item i of the patent application, wherein the image of the second eye and the second eyebrow are located at the point 9 of the distance between the two eyebrows, and the image of the center of the second right eye disappears. In the process, the right mouse button function can be activated; the second left eye center image disappears, and the left mouse can be started; (4) the seat » » 亥 亥 、, the first right eye center image and the one The process of changing the first one at the same time disappears for 1~2 seconds; "sliding 4. The method of controlling the cursor, which includes the following steps: • standard = step · preparation - display, image device and control The syllabic display shows a picture and a cursor, the picture is for the game 'i1, the image capturing device comprises a right image capturing device and a left image capturing device; 21 1 ticket setting step: the face Department includes -t heart Coordinates, - right apex coordinates, one upper left corner coordinate, one lower left corner coordinate and one right 201237674 lower corner coordinate; there are plural number of coordinates - a plurality of second coordinates; knife I, i generation: and: Take "'user's face-4' and the right and left side of the shirt is like a picking device, all of which are molybdenum's face#shooting' the right eye image capture = J's face and one of the first eyebrow images, A shadow J iiH: the center image, and in the instant image processing ', ΐϊ two eyebrow coordinates, - mouth ΐ center image and a second left eye in the upper S, Id, can correspond to the second coordinate' and respectively Ί--, the second 2D left-eye center coordinates and - the second 2D five '#i, J~3! 肀 輮 and a 3D left-eye center coordinates, which form the two sets of coordinates in the corner to obtain the virtual triangle The plane passes through 2 straight lines to point to the intersection coordinates of the picture, which is the cursor labeling step: when the user's gaze direction # is controlled. Clearing the normal line of the center point of the triangle and the face: j 22 201237674 points, controlling the cursor to move on the face, and setting the blink time change of the two right eye center images to output a right mouse button command; Setting the blink time change of the two left-eye center images can output a left mouse button command to re-set the blink time of the two right and the two left-eye center images to change one: 匕:= bone 5 middle key command; And the device that reaches the head control cursor; the seventh step into the step. • If you apply for the method of head control as described in item 4 of the special section, 1 the right image capturing device is a camera; and the middle image capturing device is a camera. The method for controlling the cursor of the head according to the fourth item, wherein: 'the first and second=2 heart images are located at the center point of the two eyebrows; the process can be started. 2 seconds to reproduce the process center image and the first -, key function. v reproduce the change process, can start the mouse 1 start mouth disappears 1~2 seconds and then the change occurs over the second left eye
TW100108134A 2011-03-10 2011-03-10 Device and method for controlling mouse cursor by head TWI480764B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
TW100108134A TWI480764B (en) 2011-03-10 2011-03-10 Device and method for controlling mouse cursor by head

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW100108134A TWI480764B (en) 2011-03-10 2011-03-10 Device and method for controlling mouse cursor by head

Publications (2)

Publication Number Publication Date
TW201237674A true TW201237674A (en) 2012-09-16
TWI480764B TWI480764B (en) 2015-04-11

Family

ID=47223192

Family Applications (1)

Application Number Title Priority Date Filing Date
TW100108134A TWI480764B (en) 2011-03-10 2011-03-10 Device and method for controlling mouse cursor by head

Country Status (1)

Country Link
TW (1) TWI480764B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103412641A (en) * 2013-03-04 2013-11-27 景智电子股份有限公司 System for control by head
CN104199549A (en) * 2014-08-29 2014-12-10 福州瑞芯微电子有限公司 Man-machine interactive type virtual touch device, system and method
CN104199548A (en) * 2014-08-29 2014-12-10 福州瑞芯微电子有限公司 Man-machine interactive type virtual touch device, system and method
CN106547339A (en) * 2015-09-22 2017-03-29 百度在线网络技术(北京)有限公司 The control method and device of computer equipment
TWI580462B (en) * 2015-09-16 2017-05-01 中華學校財團法人中華科技大學 Head and neck training aid device

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWM254669U (en) * 2000-10-24 2005-01-01 Shin Hua Rung Head control device of adjustable image-tracking type
US7197165B2 (en) * 2002-02-04 2007-03-27 Canon Kabushiki Kaisha Eye tracking using image data
TWI240891B (en) * 2003-08-20 2005-10-01 Jyh-Horng Chen Method of controlling the computer mouse by tracking user's head rotation and eyes movement
CN101051349B (en) * 2007-05-18 2010-10-13 北京中科虹霸科技有限公司 Multiple iris collecting device using active vision feedback
TWM364920U (en) * 2009-04-10 2009-09-11 Shen-Jwu Su 3D human face identification device with infrared light source
TWI396930B (en) * 2009-04-24 2013-05-21 A 3D image camera with adjustable parallax

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103412641A (en) * 2013-03-04 2013-11-27 景智电子股份有限公司 System for control by head
TWI492098B (en) * 2013-03-04 2015-07-11 Head control system and method
CN103412641B (en) * 2013-03-04 2016-04-27 达运精密工业股份有限公司 System for control by head
CN104199549A (en) * 2014-08-29 2014-12-10 福州瑞芯微电子有限公司 Man-machine interactive type virtual touch device, system and method
CN104199548A (en) * 2014-08-29 2014-12-10 福州瑞芯微电子有限公司 Man-machine interactive type virtual touch device, system and method
CN104199548B (en) * 2014-08-29 2017-08-25 福州瑞芯微电子股份有限公司 A kind of three-dimensional man-machine interactive operation device, system and method
CN104199549B (en) * 2014-08-29 2017-09-26 福州瑞芯微电子股份有限公司 A kind of virtual mouse action device, system and method
TWI580462B (en) * 2015-09-16 2017-05-01 中華學校財團法人中華科技大學 Head and neck training aid device
CN106547339A (en) * 2015-09-22 2017-03-29 百度在线网络技术(北京)有限公司 The control method and device of computer equipment
CN106547339B (en) * 2015-09-22 2020-03-20 百度在线网络技术(北京)有限公司 Control method and device of computer equipment

Also Published As

Publication number Publication date
TWI480764B (en) 2015-04-11

Similar Documents

Publication Publication Date Title
TW201237674A (en) Device and method for controlling mouse cursor by head
JP6188332B2 (en) Medical image display apparatus and program
JP5547968B2 (en) Feedback device for instructing and supervising physical movement and method of operating
CN102665589B (en) For the patient-side surgeon interface of remote-operated micro-wound surgical operation apparatus
CN102693005B (en) There is provided interactive experience using 3D depth camera and 3D projector
JP6364022B2 (en) System and method for role switching in a multiple reality environment
JP6045139B2 (en) VIDEO GENERATION DEVICE, VIDEO GENERATION METHOD, AND PROGRAM
CN103443742B (en) For staring the system and method with gesture interface
WO2020125261A1 (en) Radiotherapy auxiliary system and method thereof employing three-dimensional reconstruction of infrared and visible light
WO2014115817A1 (en) Movement-information processing device
TW201224829A (en) Methods and systems for generating stereoscopic views from monoscopic endoscope images, and computer program products thereof
TW201123077A (en) Animation generation system and method
Jiang et al. Registration technology of augmented reality in oral medicine: A review
JP2012244196A (en) Image processing apparatus and method
JP2003044873A (en) Method for generating and deforming three-dimensional model of face
KR101961844B1 (en) Augment reality based occupational therapy table device for cognitive training and occupational therapy method
JP2017018519A (en) Display device and computer program
WO2019044124A1 (en) Multiple-viewpoint video image viewing system and camera system
JP2017012264A (en) Rehabilitation assist implement and rehabilitation apparatus
Smit et al. Macroscope: First-person perspective in physical scale models
CN105616003B (en) A kind of soft tissue 3D vision tracking based on radial direction spline interpolation
CN112698724B (en) Implementation method of penetrating screen system based on camera eye movement tracking
JP6398938B2 (en) Projection control apparatus and program
TWM569887U (en) Augmented reality system
JP6964302B2 (en) Animation production method

Legal Events

Date Code Title Description
MM4A Annulment or lapse of patent due to non-payment of fees