TW200816062A - Video-based handwriting input method and apparatus - Google Patents

Video-based handwriting input method and apparatus Download PDF

Info

Publication number
TW200816062A
TW200816062A TW95136186A TW95136186A TW200816062A TW 200816062 A TW200816062 A TW 200816062A TW 95136186 A TW95136186 A TW 95136186A TW 95136186 A TW95136186 A TW 95136186A TW 200816062 A TW200816062 A TW 200816062A
Authority
TW
Taiwan
Prior art keywords
image
light source
stylus
handwriting
small light
Prior art date
Application number
TW95136186A
Other languages
Chinese (zh)
Inventor
Kelvin Shieh
Original Assignee
Ibm
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ibm filed Critical Ibm
Priority to TW95136186A priority Critical patent/TW200816062A/en
Publication of TW200816062A publication Critical patent/TW200816062A/en

Links

Abstract

Disclosed is a method and an apparatus, as well as a computer recording medium for handwriting input. The invention is implemented by capturing an image including effective strokes with a digital camera, when a user writes on a writing surface with a stylus having a light source (e.g. LED), said images is time-stamped; wherein the stylus comprising a micro-switch, when the stylus touches the writing surface, the micro switch is turned on to let the light source ON and generate a "light point", and when the stylus leaves the writing surface, the micro switch is turned off to let the light source OFF; and extracting effective stroke images by comparing the captured images with the light point on the stylus. Moreover, the invention discloses a low cost, wireless method and apparatus for handwriting input which enables an user to write on a paper without a touch sensor provided for a computer system (such as Notebook and PDA etc.) equipped with an embedded digital camera.

Description

200816062 九、發明說明·· 【發明所屬之技術領域】 本發明主要係關於手寫資訊輪入方法與裝置,具體而 言’係關於藉由數位攝影機捕獲之即時手寫筆劃資料供線 上字元辨識裝置辯識的方法與裝置。 【先前技術】 隨著電腦技術的快速發展,目前可以接收手寫輪 入的資訊處理裝置很多。它們通常使用在需輸入文字 至一小型行動(mobile)裝置,或輪入不容易鍵入字 元(例如:中文)至電腦的環境中。使用者使用筆形 衣置將手寫資料及符號輸入電腦中。相對地,也出現 了卉多手寫辨識系統,它們可以對使用者的手寫輸入 進行辨識。 . 在手寫輸入領域中,手寫文字辨識的應用,主要 可分為兩類:一為線上字元辨識(〇n line chamcter r⑽gnition,OLCR),另一類為光學字元辨識(〇ptical chamcter'recognition,OCR)。光學字元辨識是一種離 線(off-line)字元辨識的方式,它利用如掃瞄器等電 腦輸入工具,將現成的文件讀入電腦中轉為影像檔, 經過資料認知(data perceiving )與資訊重整 (data-reconstruction )的動作,再將其結果存成文字 檔,以利編輯或其它使用。相對於線上(〇n_Une)字 4 旧 M/06100TW-092906 TW9-2006-0001TW1(JHW) 6 200816062 =識,鮮字元辨朗麵㈣得 序 ^壓、速度等資訊來當作特徵,故其辨識Kg影 OLCR 轉上(Qn_line)方 _____ (digitizing tablet) ,200816062 IX. OBJECT DESCRIPTION OF THE INVENTION · Technical Fields of the Invention The present invention mainly relates to a handwriting information wheeling method and apparatus, and specifically relates to an instant handwritten stroke data captured by a digital camera for an online character recognition device. Methods and devices. [Prior Art] With the rapid development of computer technology, there are many information processing devices that can receive handwritten rounds. They are usually used when you need to enter text into a small mobile device or wheel into an environment where it is not easy to type characters (for example: Chinese) into your computer. The user enters the handwritten materials and symbols into the computer using the pen-shaped clothes. In contrast, there are also Huiduo handwriting recognition systems that can recognize the user's handwriting input. In the field of handwriting input, the application of handwritten character recognition can be divided into two categories: one is online character recognition (〇n line chamcter r(10)gnition, OLCR), and the other is optical character recognition (〇ptical chamcter'recognition, OCR). Optical character recognition is an off-line character recognition method. It uses a computer input tool such as a scanner to read an existing file into a computer and convert it into an image file. After data perceiving and The action of data-reconstruction, and then save the result as a text file for editing or other use. Relative to the line (〇n_Une) word 4 old M/06100TW-092906 TW9-2006-0001TW1 (JHW) 6 200816062 = knowledge, fresh characters to distinguish the surface (four) the order of pressure, speed and other information as a feature, so its Identify Kg shadow OLCR to turn on (Qn_line) side _____ (digitizing tablet),

再由電的制倾進行軸。㈣ 得知下筆順序、筆壓、速度”訊#作特徵,因此; 其辨識率相當高’而廣為目料寫輸人裝置採用。 目前習知_〇LCR的手寫輪4置通常要求一 感觸式襯塾’不論其是壓力感測器或磁測哭 感知且記錄在該襯墊上接觸的筆劃,〜如數位板 (tablet)。習知數位板通常具有一連線連結至一外部 智慧觸筆(smart stylus )。 IBM公司的Think Scribe是一種整合手寫數位儀 (digitizer )和傳統基於紙上記錄系統之基於筆的輸入 裝置。數位板包括一能接收電磁訊,赛的主動區 (active ) ’、該電磁訊號來自一射頻耦合的特殊觸筆。 該裝置按筆劃(stroke)與相關時序記錄使用者的手寫 輸入,並能按原來時序再現使用者的手寫輸入。使用 者可直接在數位板(tablet)上或數位板表面之紙張上 書寫。數位板產生反應筆劃與相關事件的資料串流, 41BM/06100TW-092906 TW9-2006-0001TW1(JHW) 200816062 « 並將其存在儲存器中。相關事件例如筆落事件 (pen-down)其表示筆與書寫表面接觸,筆起事件 (pen-up)其表示筆離開書寫表面。筆落事件與筆起事 件之間紙上記錄的一系列筆的座標便是、、墨跡// ( ink ) ° 上述輸入裝置皆需求一感觸式襯墊。對現存pDA 或個人電腦等等而言,這些襯墊皆是額外的成本。它 ® —般也占據空間不易攜帶;當它内建於PDA中且運作 時,也將占據螢幕的空間。感觸式襯墊通常也具有連 , 線連接至電腦,也有連線連接至一觸筆。此情形易造 成混亂。 目前,有其它類型的辨識系統,有使用數位攝影 機( digital video camera)輸入提供手寫輸入,然不是 處理過程複雜就疋結果並不理想。例如有讓渡外 φ California Institute of Technology 之美國專利 6,044,165,其揭示利用數位攝影機監視書寫器具 (writing implement)相對一書寫表面之移動,及相關之 處理硬體以處理該數位攝影機之輸出來追蹤該移動。 隨著配備有相當高解度之内建的數位攝影機的電 腦系統(如筆記型電腦、PDA等)愈來愈多進入市場, 使用數位攝影機以提供數位視訊資料將是有利的。因 4IBM/06100TW-092906 TW9-2006-0001TW1 (JHW) 8 200816062 提供—更簡易而成本低廉的解決方案。立 ^ 影機功能’而不需感測器,不索 連、、泉且此在紙上書寫’以解決上述習知之缺點。 【發明内容】 本毛明目的在提供一低成本、不需連線、不需感 測器’且能在紙上書寫之簡㈣手寫輪人方法及裝置。The shaft is then tilted by electric tilting. (4) Knowing the order of the pen, the pen pressure, and the speed of the message, so the recognition rate is quite high, and it is widely used as the input device. At present, the handwriting wheel 4 of the LCR usually requires a touch. Type lining 'whether it is a pressure sensor or a magnetic test crying and records the strokes on the pad, such as a tablet. The conventional tablet usually has a connection to an external wisdom touch. Smart stylus. IBM's Think Scribe is a pen-based input device that integrates a digitizer and a traditional paper-based recording system. The tablet includes an active area that can receive electromagnetic signals. The electromagnetic signal is from a special stylus with radio frequency coupling. The device records the user's handwriting input according to stroke and related time, and can reproduce the user's handwriting input according to the original time. The user can directly on the tablet ( On the paper on the tablet or on the surface of the tablet. The tablet generates a stream of reactive strokes and related events, 41BM/06100TW-092906 TW9-2006-0001TW1(JHW) 200816062 « and There is a memory. A related event such as a pen-down indicates that the pen is in contact with the writing surface, and a pen-up indicates that the pen leaves the writing surface. A paper record between the pen event and the pen event The coordinates of a series of pens are, ink, / ( ink) ° The above input devices all require a touch pad. For existing pDA or personal computers, etc., these pads are an additional cost. It also takes up space and is not easy to carry; when it is built into a PDA and operates, it will also occupy the screen space. The touch pad is usually connected, the cable is connected to the computer, and the connection is connected to a stylus. The situation is easy to cause confusion. At present, there are other types of identification systems, which use digital video camera input to provide handwriting input, but the processing is not complicated, and the result is not ideal. For example, there is a transfer of φ California Institute of Technology US Patent 6,044,165, which discloses the use of a digital camera to monitor the movement of a writing implement relative to a writing surface, and associated processing hardware. Processing the output of the digital camera to track the movement. With the increasing availability of computer systems (such as notebooks, PDAs, etc.) equipped with fairly high resolution built-in digital cameras, digital cameras are used to provide digital Video information will be advantageous as 4IBM/06100TW-092906 TW9-2006-0001TW1 (JHW) 8 200816062 provides an easier and less costly solution. The camera function does not require a sensor, and it does not connect, spring, and write on paper to solve the above-mentioned shortcomings. SUMMARY OF THE INVENTION The present invention aims to provide a simple (four) handwriting wheel method and apparatus that is low cost, does not require wiring, does not require a sensor, and can be written on paper.

本發明係擷取觸筆上光點之累積軌跡(有效筆 劃)’而非墨跡。因此’本發明另一目的在提供一手寫 輸入方法及裝置,其可姻於在紙上書寫的環境,或 非紙之書寫表面上而無墨跡存在的環境。 本發明揭示一手寫輸入方法、裝置及電腦記錄媒 體,當使用者使时—小絲之觸筆(stylus)於書寫表 面上書寫時,捕獲一包含有效筆劃(effective strokes) 之影像,該影像被加以時間戳記(time_stamped),其中 該觸筆包含一微開關,當該觸筆接觸書寫表面時,該 微開關、導通,使該小光源導通而發光呈現一,,光 點,當该觸筆離開書寫表面時,該微開關不導通, 使該小光源不導通;及將捕獲之影像與觸筆上光點 (如:亮度)作比較,而篩選出該有效筆劃影像。 【實城方式】 4IBM/06100TW-092906 TW9-2006-0001TW1 (JHW) 200816062 用語表: f劃~筆(Pen)或觸筆(stylus)移動的 $跡。筆劃包含在紙或書寫表面上或表面附近書寫移 動的執跡。如圖7之實線與虛線部分。 墨跡(ink) -書寫在,,紙,,表面上的文字之軌 跡。例如圖7實線部分代表之有效的筆劃。軌 以下詳細說明本發明實_。揭露的實施例只是 用來解說,因為熟悉此技藝人士應明白其中有許多修 改及變化。參考圖式時,相同號碼代表所有圖式中相 同的部分。 圖1及圖2分別說明了實施本發明之不同資料處 理系統。圖1中PDA裝置100或相似的普及運算裝置 (pensive computing device)具有一内建的數位^影 機110及手寫輸入模組(未顯示)。該數位攝影機 可以疋CCD或〇]^08數位攝影機,也可以是_紅外 線攝影機。 N .. . ... 使用者在書寫表面或紙140上書寫時,該數位攝 影機110捕獲包含即時筆劃之影像,該模組具有能夠 產生與觸筆130移動之即時筆劃影像相對應之具時間 截€ (time stamped)的動態視訊(video)資料輪出的功 4IBM/06100TW-092906 TW9-2006-0001TW1 (J HW) 1c 200816062 料有—侧⑽(如咖 ,有效^==:;及篩; 表置可接收到來自數位攝影機110捕獲之景I及:: =所示之本發明手寫輸人模組篩選出有忒筆The present invention captures the cumulative trajectory (effective stroke) of the spot on the stylus instead of the ink. Accordingly, another object of the present invention is to provide a handwriting input method and apparatus which can be used in an environment where writing on paper, or an environment on a non-paper writing surface without ink. The invention discloses a handwriting input method, a device and a computer recording medium. When a user writes a stylus of a small wire on a writing surface, an image containing effective strokes is captured, and the image is captured. Time stamped (time_stamped), wherein the stylus includes a micro switch. When the stylus contacts the writing surface, the micro switch turns on, causing the small light source to be turned on to emit a light spot, when the stylus leaves When the surface is written, the microswitch does not conduct, so that the small light source does not conduct; and the captured image is compared with the spot on the stylus (eg, brightness), and the effective stroke image is screened out. [Solid City Mode] 4IBM/06100TW-092906 TW9-2006-0001TW1 (JHW) 200816062 Glossary: f strokes ~ pen (pen) or stylus (stylus) movement of the trace. Strokes include the execution of writing movements on or near the surface of a paper or writing surface. Figure 7 is the solid line and the dotted line. Ink - A track of writing on, on, paper, and on the surface. For example, the solid line portion of Figure 7 represents a valid stroke. Tracks The present invention will be described in detail below. The disclosed embodiments are merely illustrative, as those skilled in the art will recognize many modifications and variations. When referring to the drawings, the same numbers represent the same parts in all figures. Figures 1 and 2 illustrate different data processing systems embodying the present invention, respectively. The PDA device 100 or similar pensive computing device of Figure 1 has a built-in digital camera 110 and a handwriting input module (not shown). The digital camera can be a CCD or 〇]^08 digital camera or an _ infrared camera. N.. . . When the user writes on the writing surface or paper 140, the digital camera 110 captures an image containing an instant stroke having a time corresponding to an instant stroke image that the stylus 130 moves. Turned out (time stamped) dynamic video (video) data rounded out 4IBM/06100TW-092906 TW9-2006-0001TW1 (J HW) 1c 200816062 material has - side (10) (such as coffee, effective ^ ==:; and sieve The watch can receive the scene I captured by the digital camera 110 and the handwriting input module of the present invention shown by::=

/ Ή腦糸統200,包含-主機210, 螢幕220及手寫輪入模組(未顯示)。榮幕2加上且 内建的數位攝影機則以捕獲即時筆劃影像而產^ 應之具時間戳記的動態視訊資料。 圖3為本發明使用的觸葦13〇。觸筆13〇包含一 微開關160,一可見光或紅外線等的小光源(如1^]〇) 180及一電池170。當觸筆13〇接觸書寫表面時,微開 關160導通,而由電池][70供電給小光源(LED) 18〇, 使小光源(LED) 180導通而發光呈現一,,光點”。當觸 筆130離開書寫表面時,微開關16〇不導通,電池17〇 不供電给小光源(LED) 180,使小光源(LED) 180不導 通而熄滅變暗,光點消失。 該觸筆130基本上可以是具有小光源(如LED) 180的書寫用筆,或具有小光源的不能書寫之普通觸 筆(如塑料觸筆)。該小光源一般位於筆尖^ 150旁。該 4IBM/06100TW-092906 TW9-2006-0001TW1(JHW) 200816062 小光源呈現之光點係供手寫輪入模組由數位攝 n獲的影像中’篩選出所要的即時有效的筆畫^ 分。貫施本發明時,該小光源並不限定在筆尖旁 的位置上。具有小光源的不能書寫之普通塑料可 直接以該小光源作為筆尖150。 目4說關於本發明之手寫輪人過程之-實施例 的流程圖。圖中A-A,部分是本發明之手寫輸入模 組。首先,於進行手寫輸入之前,使用者先執行一目 標,界定步驟(步驟410),而在書寫表面(14〇)上 界定供手寫輸入的目標區,即數位攝影機(11〇)摘取 影像的有效區。如圖5所示,使用者可透過螢幕22〇 與電腦系統互動,而在書寫表面上界定該目標區。此 外,由於數位攝影機擷取書寫表面上長方形區域之影 像’將呈現梯形,因此界定目標區後可再執行一簡易 的數學轉換將梯形轉換回長方形19〇而顯示在螢幕 上。在獲得筆劃影像後得再對筆劃影像進行數學轉換 的功能(如後述之步驟433)。須知在手寫輪入過程中該 數學气換並非必要,因該轉換可在手寫辨識過程處理: 當使用者使用有小光源180之觸筆於書寫表面之 目標區書寫且接觸書寫表面時,小光源⑽導通而發 光呈現-光點。本發卿用此光點,而對數位攝影機 捕獲之影像進行處理,以獲得與有效筆劃影像相應之 4IBM/06100TW-092906 TW9-2006-0001TW1 (JHW) 12 200816062 視訊資料(步驟430) 〇 獲得有效筆劃影像資料之步驟430,可細分為三 步驟,分別陳述如下: 、、、 步驟431,執行一捕獲影像與加時間瞿戈記(time stamped)之步驟。數位攝影機一般以每秒約%張圖 龜 框(fmme)的速度取像,因此,除由數位攝影機 矚標區捕獲的影像外,郷像也被加上相關的時間戮記 (time stamp)。接著於步驟432,執行一由捕獲的影 像中篩選出有效筆劃影像之步驟,而將取得之&框^ 像與觸筆上光點作比較’滤除不相干影像(如:手和 觸筆的影像),僅留下觸筆上光點之執跡,即,,有效 筆劃”部分。例如,以該觸筆上光點之,,亮度 (intensity)”作為門檻值(threshold),濾除該捕獲的影 像中亮度小於該門檻值的部分,留下的僅剩觸筆上光 _ 點之軌跡,即”有效筆劃”部分。此外,若小光源 是紅外線,則也可以紅外線之,,亮度,,作為門檻值 (threshold),濾除該捕獲的影像中亮度小、於該門檻值的 部分,留下的僅剩觸筆上光點之軌跡,即”有效筆 劃,,部分。 步驟433中,則如前述圖5的相關陳述,執行一 對有效筆劃影像進行數學轉換的步驟,將對含有時間 4 旧 M/06100TW-092906 TW9-2006-0001TW1 (JHW) 13 200816062 換及正規化 以調整數位攝影 必要,因其可在 戳記的有效筆劃進行數學轉 (Transformation and Normalization ), 機取像的偏差。如前述,此步驟並非 手寫辨識過程中處理。 /圖6描H列數位攝影機所拍下之影像圖框 (frame)。因數位攝影機一般係以每秒約3〇張圖框 傻=6僅呈現部分圖框。由於數位攝影機 將捕獲取像% (Vlewfield)内所有影像,因此,若3 以前述之具有小統的書寫用筆於紙上書寫,則g 上圖框中將包含墨跡、卿、—光點及㈣影像。為 Γ便圖:之該光點係以一累積轨跡(___ 部分,其中實線部分為墨跡,其係與觸上 ^ = 軌跡。因此,每以 甘 、 光點的累積轨跡及手的影像, :觸筆上光點的累積執跡,便構成所要之^ ^事實上單-圖框中之影像,不會有如圖6所示 線部分的影像將只呈現—個 蚩而方小光源180而非光點(因觸筆離開 曰馬表面,小光源180炮滅,而光點消失)。藉由此觸 4IBM/06100TW-092906 TW9-2006-0001TW1 (JHW) 14 200816062 筆上之光點之,,亮度,,夹去 墨跡、鱗其餘部分及^影像本容巧軸 點之累積軌跡的視訊資料儲存在存^下觸筆上光 步驟切後,儲存在儲存器故在執行 效筆割j之累積執跡所構成之有 、、、氏上的墨跡與此處之有效筆劃相對應。有 扣右疋以具有小光源180的非書寫用筌 缸+ 寫表面上書寫,則將盔# 、、、 ; 、、、氏之句 中將句人在。因此,事實上圖框 上光點所不之觸筆〜光點及手的影像^觸筆 先』的累積軌跡,才構成所要之有效筆劃部分。 寫 ^ 說明進一步說明實施步驟430之實例。當書 ▲ 日守’於兀成步驟43〇由數位攝影機η〇獲得的 有效筆劃影像係圖7中實線部分,即含有線段〇 1 λ^3及0:5。事實上,虛線及實線部分係分別代表 觸筆離開書寫表面及接觸書寫表面時,觸筆上筆尖150 方小光源180之累積軌跡。其中時段〇2及〇4係表 示觸拳離開書寫表面而不存在光點的時段,其對應筆 劃影像中虛線部分之線段〇2〇4。因此,以該觸筆上 光點之”亮度”作為門檻值(threshold),濾除該取得之 筆劃影像中亮度小於該門檻值的部分(如:手、觸筆的 影像及虛線部分之線段〇2〇4),留下的僅剩觸筆上光 點之轨跡,即”有效筆劃,,部分,如圖7中實線部 4IBM/06100TW-092906 TW9-2006-0001TW1 (JHW) 15 200816062 1的Ύ。同樣地’圖6顯 筆者部八 步驟·處理後的結果 ’經由 6中筆劃之實線部分被儲存。見在圖8中,其令僅有圖 識引tt:驟430錢得的有效筆劃將送至一手寫辨 識引擎中進行辨識(步驟46G)。由 :寫辨 提供手寫輪入的有效筆詞x的在 身。因此該手寫觸^辨識而非此辨識技術本 引擎。4寫觸料可叹任何—般料寫辨識 落H轉日轉體實施例可知,本發日⑽取 ί,ii點之累積軌跡(即有效筆劃),而非墨跡。因 _电明之手寫輪人方法可適用於寫 境,或非紙之書寫表面上而無墨跡存在氏的環;寫叫 上述僅為本發明之較佳具體實 實現於硬體、軟體、或硬體與軟體的組合本 口此存在許多其他的變化方式實施。 注意的是’熟悉此技藝者應該可以了解,本發 月序可用儲存於電腦可讀媒體或其他各種格 ’加以韻,而與實際用以實施分送之承载 ㈣的特__縣關。電腦可軸體的例子包含 以下媒體.可抹除程絲唯讀記憶體(咖價)、唯讀 4ΙΒΜ/06100TW-092906 TW9-2006-0001TW1 (JHW) 200816062 。己f思體(ROM)、磁帶(tape)、紙張、教 在敗今产駚/ 、及浪敦碟、硬碟、隨機 存取妹體(RAM),以及CD_R〇M,還 媒體’像是數位與類比的通訊連結。 、別a 上迷之本發明實施例係用於說明與描述發, 而並非是餘限定本伽。依關示之 ^ =的修正及變化。本發明的範圍並“/ Camphor 200, including - host 210, screen 220 and handwriting wheeled module (not shown). The VW 2 plus built-in digital camera produces a time-stamped dynamic video data capture of the instant stroke image. Figure 3 shows the touch panel 13 used in the present invention. The stylus 13A includes a micro switch 160, a small light source such as visible light or infrared light (e.g., 〇) 180 and a battery 170. When the stylus 13 is in contact with the writing surface, the microswitch 160 is turned on, and the battery [70] supplies power to the small light source (LED) 18 〇, causing the small light source (LED) 180 to be turned on to emit a light, "light spot". When the stylus 130 leaves the writing surface, the microswitch 16 does not conduct, and the battery 17 does not supply power to the small light source (LED) 180, so that the small light source (LED) 180 does not conduct and turns out, and the light spot disappears. The stylus 130 It can basically be a writing pen with a small light source (such as LED) 180, or a conventional stylus (such as a plastic stylus) with a small light source that cannot be written. This small light source is generally located next to the nib ^ 150. The 4IBM/06100TW- 092906 TW9-2006-0001TW1(JHW) 200816062 The light source presented by the small light source is used for the hand-written wheel-in module to screen out the desired instant and effective strokes in the image obtained by digital capture. When applying the invention, The small light source is not limited to the position beside the nib. A common plastic that cannot be written with a small light source can directly use the small light source as the nib 150. Figure 4 is a flow chart of an embodiment of the handwriting wheel process of the present invention. In the figure, AA, part is the handwriting input mode of the present invention. First, before performing handwriting input, the user first performs a target, defines a step (step 410), and defines a target area for handwriting input on the writing surface (14〇), that is, a digital camera (11〇). The effective area of the image. As shown in Figure 5, the user can interact with the computer system through the screen 22, and define the target area on the writing surface. In addition, since the digital camera captures the image of the rectangular area on the writing surface, it will be presented. Trapezoidal, so after defining the target area, a simple mathematical conversion can be performed to convert the trapezoid back to the rectangle and display it on the screen. After the stroke image is obtained, the stroke image is mathematically converted (step 433, which will be described later). It should be noted that the mathematical gas exchange is not necessary during the handwriting rounding process, because the conversion can be processed in the handwriting recognition process: when the user uses the stylus with the small light source 180 to write on the target area of the writing surface and touch the writing surface, small The light source (10) is turned on and the light is emitted - the light spot. The hair spot uses the light spot, and the image captured by the digital camera is processed to obtain and be effective. The corresponding image of 4IBM/06100TW-092906 TW9-2006-0001TW1 (JHW) 12 200816062 Video data (step 430) 步骤 Step 430 of obtaining valid stroke image data can be subdivided into three steps, which are respectively stated as follows: , , , step 431 Performing a step of capturing images and adding time stamped. The digital camera generally takes images at a speed of about 1% of the frame per frame (fmme), so, except for images captured by the digital camera's target area. In addition, the image is also added with the relevant time stamp. Next, in step 432, a step of filtering out the effective stroke image from the captured image is performed, and the obtained & frame image is compared with the stylus light spot to filter out irrelevant images (eg, hand and stylus) Image), leaving only the trace of the light point on the stylus, ie, the effective stroke portion. For example, using the illuminating point of the stylus, the intensity is used as the threshold value, filtering out The portion of the captured image whose brightness is less than the threshold value leaves only the trajectory of the stylus glazing, that is, the "effective stroke" portion. In addition, if the small light source is infrared light, the infrared light, the brightness, and the threshold value may be used as a threshold value to filter out the portion of the captured image that has a small brightness and a threshold value, leaving only the stylus remaining. The trajectory of the light spot, that is, "effective stroke, part." In step 433, the step of performing a mathematical conversion of a pair of effective stroke images as in the related statement of FIG. 5 described above, will contain the time 4 old M/06100TW-092906 TW9 -2006-0001TW1 (JHW) 13 200816062 It is necessary to change the normalization to adjust the digital photography, because it can perform the transformation and normalization in the effective stroke of the stamp, and the deviation of the image is taken. As mentioned above, this step is not handwriting recognition. Processed in the process. / Figure 6 depicts the image frame taken by the H-column digital camera. The factor-bit camera is usually about 3 frames per second, silly = 6 and only part of the frame is presented. Since the digital camera will Capture all images in the image (Vlewfield). Therefore, if 3 is written on paper with the aforementioned writing pen, the upper frame of g will contain ink, qing, light spot and (4) For example, the light point is a cumulative trajectory (the ___ part, where the solid line is the ink, and the line is touched with the ^ = trajectory. Therefore, the cumulative trajectory of each of the sweet and light points and The image of the hand, the cumulative trace of the light point on the stylus, constitutes the image of the desired ^ ^ factual-frame, there will be no image of the line as shown in Figure 6 will be presented only - Small light source 180 instead of light spot (because the stylus leaves the surface of the Hummer, the small light source 180 is extinguished, and the light spot disappears.) By touching 4IBM/06100TW-092906 TW9-2006-0001TW1 (JHW) 14 200816062 The light spot, the brightness, the clipped ink, the rest of the scale, and the video data of the cumulative track of the image point are stored in the storage stylus and then stored in the memory. The accumulated traces of the pen cut j are composed of the ink on the , and the ink on the mark corresponds to the effective stroke here. If there is a non-writing cylinder with a small light source 180 written on the surface, the writing will be Helmets #, , , ; , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , The cumulative trajectory of the stylus ~ light spot and the hand image ^ stylus first constituts the desired stroke portion. Write ^ Description to further illustrate the example of the implementation step 430. When the book ▲ 日守' Yu Yucheng step 43 〇 by the digital The effective stroke image obtained by the camera η〇 is the solid line part of Fig. 7, which contains the line segments 〇1 λ^3 and 0: 5. In fact, the dotted line and the solid line portion respectively represent the stylus leaving the writing surface and contacting the writing surface. , the cumulative trajectory of the 150-small light source 180 on the pen tip. The time periods 〇2 and 〇4 indicate the time period in which the touch box leaves the writing surface without the light spot, which corresponds to the line segment 〇2〇4 of the broken line portion in the stroke image. Therefore, using the "brightness" of the spot on the stylus as a threshold, filtering out the portion of the obtained stroke image whose brightness is less than the threshold (eg, the line of the hand, the stylus, and the dotted line) 2〇4), leaving only the trajectory of the stylus on the light spot, that is, "effective stroke, part, as shown in Figure 7 solid line 4IBM/06100TW-092906 TW9-2006-0001TW1 (JHW) 15 200816062 1 Similarly, 'the result of the eight steps of the pen and the processed result' is stored through the solid line portion of the stroke in the six strokes. See in Fig. 8, which makes it only the tt: 430 money. The valid stroke will be sent to a handwriting recognition engine for identification (step 46G). By: writing and providing the handwritten rounded valid word x is in the body. Therefore, the handwriting touches the recognition instead of the identification technology engine. The material can be sighed with any kind of sighing. The example of the singularity of the singularity of the singularity of the singularity of the singularity of the singularity (ie, the effective stroke), rather than the ink. Can be applied to the writing environment, or the non-paper writing surface without the presence of ink ring; write the above It is only a preferred embodiment of the present invention that is implemented in hardware, software, or a combination of hardware and software. There are many other variations in the implementation of this invention. It is noted that 'satisfy with this skill should be understood, It can be stored in computer readable media or other various styles, and it is specially used to implement the distribution of the carrier (4). The computer can be used as an example of the shaft. The following media can be erased. Memory (coffee price), only reading 4ΙΒΜ/06100TW-092906 TW9-2006-0001TW1 (JHW) 200816062. Have a body (ROM), tape (tape), paper, teach in the loss of production / /, and London Disc, hard disc, random access girl (RAM), and CD_R〇M, also media 'like digital and analog communication links. The other embodiments of the present invention are used for illustration and description, and It is not the limit of the original gamma. According to the correction and change of ^ =. The scope of the invention and "

^來加⑽定’而是以其後所附之申請範圍來決 疋。上述的說明、範例、及資料提供了一完整製造及 使用本發明的組合。因其他很多的實施例可在未脫離 本發,所揭示之精神下完成等效改變或修飾,本發明 申睛範圍歸屬於申請專利範圍中所述之範圍。 【圖式簡單說明】 為使本發明及優點能更明瞭,本發明說明將伴隨下述 相關圖式來說明: 圖1及圖2分別說明了實施本發明之不同資料處理系 統。 圖3為本發明使用的觸筆130。 圖4說明關於本發明之手寫輸入過程之一實施例的 流程圖。 圖5說明使用者可透過螢幕220與電腦系統互動,而 在書寫表面上界定該目標區。 圖6描繪一系列數位攝影機所拍下之影像圖框 4IBM/06100TW-092906 TW9-2006-0001TW1 (JHW) 200816062 (frame)。圖7說明實施步驟430之實例。 圖8說明圖6顯示的有效筆劃部分,經由步驟430處理後 的結果。 【主要元件符號說明】 10Θ PDA裝置 110 數位攝影機110 12Θ 麥克風120 130觸筆 140書寫表面或紙 150筆尖 160微開關 170電池 180小光源(如LED) 190長方形 200電腦系統 210主機 220\螢幕 410目標區界定步驟 430獲得有效筆劃影像資料之步驟 431執行一捕獲影像與加時間戳記之步驟 432執行一由捕獲的影像中篩選出有效筆劃影像之步 驟 4IBM/06100TW-092906 18 TW9-2006-0001TW1 (JHW) 200816062 433執行一對有效筆劃影像進行數學轉換的步驟 460手寫辨識引擎中進行辨識之步驟 700有效筆劃影像^Additional (10) is determined by the scope of the application attached to it. The above description, examples, and materials provide a complete fabrication and use of the combinations of the present invention. Equivalent changes or modifications may be made without departing from the spirit of the invention, and the scope of the invention is intended to be within the scope of the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS In order to make the present invention and advantages clear, the description of the present invention will be described with reference to the accompanying drawings: Fig. 1 and Fig. 2 respectively illustrate different data processing systems embodying the present invention. Figure 3 shows the stylus 130 used in the present invention. Figure 4 illustrates a flow chart of one embodiment of a handwriting input process in accordance with the present invention. Figure 5 illustrates that a user can interact with a computer system via screen 220 to define the target area on the writing surface. Figure 6 depicts an image frame taken by a series of digital cameras 4IBM/06100TW-092906 TW9-2006-0001TW1 (JHW) 200816062 (frame). FIG. 7 illustrates an example of implementing step 430. Figure 8 illustrates the results of the effective stroke portion shown in Figure 6 after processing via step 430. [Main component symbol description] 10Θ PDA device 110 Digital camera 110 12Θ Microphone 120 130 stylus 140 writing surface or paper 150 nib 160 micro switch 170 battery 180 small light source (such as LED) 190 rectangle 200 computer system 210 host 220 \ screen 410 target The step 431 of obtaining the effective stroke image data by the region defining step 430 performs a step of capturing a video and time stamping step 432 to perform a step of filtering out the effective stroke image from the captured image. 4 IBM/06100TW-092906 18 TW9-2006-0001TW1 (JHW 200816062 433 Steps for performing a mathematical conversion of a pair of valid stroke images 460 Steps for identifying in the handwriting recognition engine 700 Effective stroke images

4IBM/06100TW-092906 TW9-2006-0001TW1(JHW)4IBM/06100TW-092906 TW9-2006-0001TW1 (JHW)

Claims (1)

200816062 、申請專利範圍: 1· 一種基於視訊(video-based)的手寫輪入方法,包 當使用者使用有-小光源之觸筆(stylus)於 書寫表面上書寫時,捕獲-包含有敦筆劃 strokes)之影像,該影像被加以時間戳纪 其中該觸筆包含〜微開關,當該 f表面時,該微開關導通,使該小光源 光壬現光點,,’當該觸筆離開蚩寫表 面時’該微開關不導通’使該小光源不導通及 將該捕獲之影像與觸筆上光點作比較而 選出該有效筆劃影像。 ’師 2. 如^請專利範圍第1項之方法,其包含1伊區心 餘行步驟’而在書寫表面上衫供手寫二^ 3. j請專利範圍第阳項之方法,其中媒 或⑽數位攝影機或一紅外 4. 如申^魏㈣印奴方法,其㈣麻 肩:以:’具有一小光源的書寫用乘或:有2基本 源的不能書寫之普通觸筆,如塑料觸筆、。〜小光 4IBM/06100TW-092906 TW9-2006-0001TW1 (JHW) 20 200816062 。如中請專利範圍第4項之方法,其中談小光源係— LED。 6·如申請專利範圍第1或2或5項之方法,其中該一小光 源係在筆尖旁。 7·如申請專利範圍第2項之方法,其中該目標區界定之 先行步驟,包含一數學轉換步驟,將目標區之影像 ,轉換回長方形。 8β如申請專利範圍第7項之方法,進一步包含: 對原始筆劃影像進行數學轉換的步驟,將對含有時 間戳記的有效筆劃進行數學轉換及正規化 (Transformation and Normalization),以調整取像的 偏差。 、 馨 9·如申請專利範圍第丨或匕戈“員之方法,進一步包含·· 將該有效肇劃送至一手寫辨識引擎中進行辨識之步 10·如申請專利範圍第〗或2或5項之方法,其中該篩選 出該有效料f彡像之轉’細該光狀亮度作為 -門檻’滤除該捕獲之影像中亮度小於該門檀值的 部分’而篩選出該有效筆劃影像。 41BM/06100TW-092906 TW9-2006-0001TW1 (JHW) 200816062 11.如申請專利範圍第3項之方法,其中該小光源可以 是一紅外線光源。 12·如申請專利範圍第u之方法,其中該篩選出該有 效筆劃影像之步驟,係以該紅外線之亮度作為門檻 值(threshold),濾除該捕獲的影像中亮度小於該門 檻值的部分,而篩選出該有效筆劃影像。 13· —種基於視訊的手寫輸入裝置,包含: 一觸筆(stylus),其包含一微開關及一小光 源’當該觸筆接觸書寫表面時,該微開關導通,使 該小光源導通而發光呈現一,,光點”;當該觸筆離 開書寫表面時,該微開關不導通,使該小光源不導 通; 數位影像捕獲褒置’以捕獲一包含有效筆劃 (effective strokes)之影像,該影像被加& 記;及 篩選有效筆劃影像裝置,將捕獲之影像與觸筆 上光點作比較,而篩選出該有效筆劃影像。 14。如申請專利範圍第13項之手寫輸入裝置,其進一步 包含-目標H界定裝置,*在書寫表面上界定供$ 寫輸入的目標區。 八 4IBM/06100TW-092906 TW9-2006-0001TW1(JHW) 22 200816062 15·如申請專利範圍第13或14項之手寫輪入裝置,复 中該數位影像捕獲裝置,係一數位攝影機,如 或CMOS數位攝影機或一紅外線攝影機。 16·如申請專利範圍第13或14項之手寫輪入裝置,豆 中遠觸筆是具有一小光源的書寫用筆或具有一 光源的不能書寫之普通觸筆,如塑料觸筆'。’、 17·如申請專利範圍第16項之手寫輪入裝置,复 小光源係一 LED。 "" 18β如申請專利範圍第13或14或17項之手寫輸入妒 置’其中該一小光源係在筆尖旁。 」才 19. 如申請專利範圍第14項之手寫輪入裝置,其中該 目標區界絲置’包含—數學轉換裝置,將’目伊 之影像轉換回長方形。^ 20. =申請專利範圍第19項之手寫輪入裝置,進一步包 對原始筆劃影像進行數學轉換的裝置,將對含有 時間戳記的原始筆劃進行數學轉換及正規化,以々 整取像的偏差。 4 旧 M/06100TW-092906 TW9-2006-0001TW1 (J HW) 23 21.200816062 如申請專利範圍第13或Η或 置,進一步包含·· 一手寫辨識引擎,將該有效筆劃 擎中進行辨識。200816062, the scope of patent application: 1. A video-based handwriting wheeling method, when the user writes on the writing surface using a stylus with a small light source, the capture - contains the strokes Strokes), the image is time stamped, wherein the stylus contains a ~microswitch, when the f surface, the microswitch is turned on, causing the small light source to illuminate the spot, 'When the stylus leaves 蚩When the surface is written, 'the microswitch is not turned on' causes the small light source to be non-conducting and the captured image is compared with the spot on the stylus to select the effective stroke image. '师2. For example, please refer to the method of the first item of the patent scope, which includes the steps of the 1st district, and the handwriting on the surface of the writing is for the handwriting. ^^ Please ask the method of the patent range of the positive term, where the medium or (10) Digital camera or an infrared 4. Such as Shen Wei (four) printed slave method, its (four) hemp shoulder: to: 'have a small light source for writing or: there are 2 basic sources of non-writing ordinary stylus, such as plastic touch pen,. ~ Xiaoguang 4IBM/06100TW-092906 TW9-2006-0001TW1 (JHW) 20 200816062. For example, please refer to the method of item 4 of the patent scope, in which the small light source system is used. 6. The method of claim 1 or 2 or 5, wherein the small light source is next to the nib. 7. The method of claim 2, wherein the first step of defining the target zone comprises a mathematical conversion step of converting the image of the target zone back into a rectangle. 8β The method of claim 7 further includes: performing a mathematical conversion step on the original stroke image, and performing mathematical transformation and normalization on the effective stroke with time stamp to adjust the deviation of the image. . , Xin 9 · If you apply for the patent scope of the third or the "personal method", further include · · Send the valid 肇 to a handwriting recognition engine for identification step 10 · If you apply for patent scope 〗 〖 or 2 or 5 The method of the present invention, wherein the screening of the effective material f 彡 该 该 该 该 光 光 光 光 光 光 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 The method of claim 3, wherein the small light source can be an infrared light source. 12. The method of claim u, wherein the screening is as described in claim 5, wherein the screening The step of outputting the effective stroke image is to use the brightness of the infrared light as a threshold to filter out the portion of the captured image whose brightness is less than the threshold value, and to filter out the effective stroke image. 13·-based video The handwriting input device comprises: a stylus comprising a micro switch and a small light source. When the stylus contacts the writing surface, the micro switch is turned on, so that the small light source is turned on. The light presents a light spot; when the stylus leaves the writing surface, the microswitch does not conduct, so that the small light source does not conduct; the digital image capture device ' captures an image containing effective strokes, The image is added & and the effective stroke image device is screened, and the captured image is compared with the spot on the stylus to screen out the effective stroke image. 14. The handwriting input device of claim 13, further comprising - a target H defining means, * defining a target area for writing input on the writing surface.八4IBM/06100TW-092906 TW9-2006-0001TW1(JHW) 22 200816062 15·If the handwriting wheeling device of claim 13 or 14 is applied, the digital image capturing device is a digital camera, such as a CMOS digital device. Camera or an infrared camera. 16. The hand-held wheel-in device of claim 13 or 14, wherein the far-talking pen is a writing pen having a small light source or a non-writing ordinary stylus having a light source, such as a plastic stylus. '17'. For the hand-written wheel-in device of claim 16 of the patent application, the small light source is an LED. "" 18β, as in the handwritten input device of claim 13 or 14 or 17, wherein the small light source is next to the nib. 19. In the handwritten wheel-in device of claim 14, wherein the target boundary line includes a mathematical conversion device that converts the image of the object back into a rectangle. ^ 20. = Handwritten wheeling device of claim 19, further comprising means for mathematically converting the original stroke image, mathematically converting and normalizing the original stroke with time stamp to adjust the deviation of the image . 4 Old M/06100TW-092906 TW9-2006-0001TW1 (J HW) 23 21.200816062 If the patent scope is 13 or ,, it further includes a handwriting recognition engine that recognizes the effective stroke engine. 22β如申請專利範圍第13或14或17項之手寫輸入裝 置’其中該篩選有效筆劃影像裝置,係以該光點之 亮度作為一門檻,濾除該捕獲之影像中亮度小於 該門檻值的部分,而篩選出該有效筆劃影像。 23·如申請專利範圍第15項之手寫輸入裝置,其中該 小光源可以是一紅外線光源。22β, as in the handwriting input device of claim 13 or 14 or 17, wherein the screening effective image processing device uses the brightness of the light spot as a threshold to filter out the portion of the captured image whose brightness is less than the threshold value. And the effective stroke image is screened out. 23. The handwriting input device of claim 15, wherein the small light source is an infrared light source. 口項之手寫輸入裝 送至一手寫辨識引 24·如申請專利範圍第23之手寫輸入裝置,其中該篩 選出遠有效筆劃影像之步驟,係以遠紅外線之冗度 作為門檻值(threshold),濾除該捕獲的影像中亮度 小於該門檻值的部分,而篩選出該有效筆劃影像。 25。 一種資料處理系統,包含: 如申請專利範圍第13至24中任一項所述之手寫輸 入裝置。 26. —種普及運算裝置,包含: 4IBM/06100TW-092906 2A TW9-2006-0001TW1 (JHW) 200816062 如申请專利範圍第13至24項中任一項所述之手寫 輸入裝置。 27· —種具有數位影像捕獲裝置之資料處理系統,包 含一手寫輸入模組以執行如申請專利範圍第丨至]^ 項中任一項所述之手寫輸入方法。 28· —種電腦記錄媒體,其内儲存一程式,該程勹 ,碼,使一資料處理系統執行如申請專;= 第1至12項中任一項所述之手寫輪入方法轭圍The handwriting input of the mouth item is sent to a handwriting recognition guide. 24. The handwriting input device of the 23rd patent application scope, wherein the step of filtering out the far effective stroke image is based on the redundancy of the far infrared ray as a threshold value. The effective stroke image is screened out except for the portion of the captured image whose brightness is less than the threshold value. 25. A data processing system comprising: the handwriting input device according to any one of claims 13 to 24. 26. A pervasive computing device, comprising: 4 IBM/06100 TW-092906 2A TW9-2006-0001 TW1 (JHW) 200816062 A handwriting input device according to any one of claims 13 to 24. A data processing system having a digital image capture device, comprising a handwriting input module for performing the handwriting input method of any one of the claims. 28. A computer recording medium in which a program is stored, the program, code, and a data processing system are executed as an application; = handwriting wheel method yoke according to any one of items 1 to 12. 4IBM/06100TW-092906 TW9-2006-0001TW1(JHW) 254IBM/06100TW-092906 TW9-2006-0001TW1(JHW) 25
TW95136186A 2006-09-29 2006-09-29 Video-based handwriting input method and apparatus TW200816062A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
TW95136186A TW200816062A (en) 2006-09-29 2006-09-29 Video-based handwriting input method and apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW95136186A TW200816062A (en) 2006-09-29 2006-09-29 Video-based handwriting input method and apparatus

Publications (1)

Publication Number Publication Date
TW200816062A true TW200816062A (en) 2008-04-01

Family

ID=44769003

Family Applications (1)

Application Number Title Priority Date Filing Date
TW95136186A TW200816062A (en) 2006-09-29 2006-09-29 Video-based handwriting input method and apparatus

Country Status (1)

Country Link
TW (1) TW200816062A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8629854B2 (en) 2009-05-18 2014-01-14 Pixart Imaging Inc. Controlling method applied to a sensing system
TWI464627B (en) * 2009-04-30 2014-12-11 Hon Hai Prec Ind Co Ltd Handwriting input apparatus
CN105302298A (en) * 2015-09-17 2016-02-03 深圳市国华识别科技开发有限公司 Air writing pen-stopping system and method

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI464627B (en) * 2009-04-30 2014-12-11 Hon Hai Prec Ind Co Ltd Handwriting input apparatus
US8629854B2 (en) 2009-05-18 2014-01-14 Pixart Imaging Inc. Controlling method applied to a sensing system
TWI452488B (en) * 2009-05-18 2014-09-11 Pixart Imaging Inc Controlling method applied to a sensing system
CN105302298A (en) * 2015-09-17 2016-02-03 深圳市国华识别科技开发有限公司 Air writing pen-stopping system and method
US10725552B2 (en) 2015-09-17 2020-07-28 Shenzhen Prtek Co., Ltd. Text input method and device based on gesture recognition, and storage medium

Similar Documents

Publication Publication Date Title
US7889928B2 (en) Video-based handwriting input
TWI336854B (en) Video-based biometric signature data collecting method and apparatus
Yang et al. Magic finger: always-available input through finger instrumentation
US20150234518A1 (en) Dynamic Input At A Touch-Based Interface Based On Pressure
Taylor et al. Type-hover-swipe in 96 bytes: A motion sensing mechanical keyboard
JP5470051B2 (en) Note capture device
KR20020087993A (en) Information input device
CA2501118A1 (en) Method of combining data entry of handwritten symbols with displayed character data
US20140152543A1 (en) System, data providing method and electronic apparatus
CN105608438A (en) Information recognition method
US9195697B2 (en) Correlation of written notes to digital content
JP5925957B2 (en) Electronic device and handwritten data processing method
CN102103462A (en) Multifunctional electronic pen, character writing and sampling method and track data storage method
US20130033460A1 (en) Method of notetaking using optically imaging pen with source document referencing
US7922099B1 (en) System and method for associating content with an image bearing surface
CN109871117A (en) Information processing unit, display device and information processing system
TW200816062A (en) Video-based handwriting input method and apparatus
US20130033429A1 (en) Method of notetaking with source document referencing
CN103336633A (en) Electronic white board
CN1667561A (en) Intelligent pen
CN205405472U (en) Electronic handwriting pad
JP2009146273A (en) Terminal device and program thereof
CN109375799A (en) A kind of extension Writing method, apparatus and system
TW201823939A (en) Touch pen and operation method thereof
Ransiri et al. SmartFinger: connecting devices, objects and people seamlessly