New! View global litigation for patent families

US8150162B2 - Method and system for three-dimensional handwriting recognition - Google Patents

Method and system for three-dimensional handwriting recognition Download PDF

Info

Publication number
US8150162B2
US8150162B2 US10528938 US52893805A US8150162B2 US 8150162 B2 US8150162 B2 US 8150162B2 US 10528938 US10528938 US 10528938 US 52893805 A US52893805 A US 52893805A US 8150162 B2 US8150162 B2 US 8150162B2
Authority
US
Grant status
Grant
Patent type
Prior art keywords
recognition
motion
device
handwriting
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US10528938
Other versions
US20060149737A1 (en )
Inventor
Yonggang Du
Jiawen Tu
Lei Feng
Xiaoling Shao
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koninklijke Philips NV
Original Assignee
Koninklijke Philips NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Grant date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/20Image acquisition
    • G06K9/22Image acquisition using hand-held instruments
    • G06K9/222Image acquisition using hand-held instruments the instrument generating sequences of position coordinates corresponding to handwriting; preprocessing or recognising digital ink
    • G06K9/224Image acquisition using hand-held instruments the instrument generating sequences of position coordinates corresponding to handwriting; preprocessing or recognising digital ink in three dimensions
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/20Image acquisition
    • G06K9/22Image acquisition using hand-held instruments
    • G06K9/222Image acquisition using hand-held instruments the instrument generating sequences of position coordinates corresponding to handwriting; preprocessing or recognising digital ink

Abstract

The present invention provides a 3D handwriting recognition system that allows users to freely write words or characters in a 3D space in a touchless manner, without requiring any physical medium such as a pad or a tablet. The users' handwriting input in a 3D space will be tracked by an input device of the system that generates corresponding 3D motion data and wirelessly transfers the 3D motion data to a recognition device of the system. The 3D motion data will be converted and then mapped onto a 2D plane to generate corresponding 2D images for handwriting recognition. In this way, the users' inputting will never be limited to any screen, pad or plane, and the users will have more flexibility and enjoyable writing experience.

Description

BACKGROUND OF THE INVENTION

The invention relates generally to handwriting recognition technologies, and more particularly to three-dimensional (3D) handwriting recognition methods and systems.

Handwriting recognition is a technique by which an intelligence system can recognize characters and other symbols written by hand. This technique has been very popular since it frees users from the keyboard, allowing users to write and draw in a more natural way. With the increasing demands from users, more and more devices now have incorporated handwriting recognition system to give users natural input experience. Handwriting recognition is particularly popular in various handheld devices that provides handwriting recognition of words with complex structures, e.g., Chinese characters, input of which using a conventional keyboard is very time consuming.

A handwriting input device provides a user with a friendly way of inputting information. At present, the minimum requirement for an inputting device is a mouse. To write with the mouse, the user needs to press and hold down the primary mouse button, and then move the mouse pointer to form strokes of a word or character for generating the final word or character. Popular handwriting input devices, such as a pen stylus and tablet, are used on conventional handheld devices such as PDAs, or are connected to a computer through a serial or USB port. Handheld devices often use a pen stylus and a touch screen as a pad to allow users to perform recognition functions. Most handheld devices, such as PDAs, are equipped with this kind of input device.

Another kind of handwriting input device includes a pen that allows users to transfer data into a receiving unit, such as a cellular phone, a PDA or a PC by simply writing or drawing in their own natural handwriting on the comfort and space of a regular piece of paper.

At present, all conventional handwriting input devices adopt a two-dimensional input method. Users have to write on a physical medium, such as a tablet, a touch screen, or a paper pad. This limits the choices available for users. For example, if a user wants to write some comments during a presentation or a lecture, he or she would have to first find a physical medium, e.g., a table, a paper pad, etc. This can cause much inconvenience for the user while standing in the room, giving the presentation or lecture. Also, in a mobile environment, such as in a car, a bus or a subway, it would be very inconvenient to “write” on a physical medium using a stylus.

Therefore, there is a need to provide an improved handwriting recognition system that gives users more flexibility and convenience and frees the users from the physical medium required for two-dimensional handwriting recognition.

SUMMARY OF THE INVENTION

The present invention gives users more flexibility and enjoyable writing experience by allowing users to freely write words or characters in a 3D space in a touchless way, without requiring any physical medium such as a pad or a tablet.

According to the invention, there is provided a handwriting recognition system that comprises an input device and a recognition device in communication with the input device. The input device includes a three-dimensional (3D) motion detection sensor configured to generate 3D motion data in response to a 3D motion. In one embodiment, the motion detection sensor measures acceleration of the 3D motion in X, Y and Z axial directions to generate the 3D motion data. The recognition device is configured to receive (e.g., by wireless means) the 3D motion data from the input device and derive corresponding two-dimensional (2D) images for handwriting recognition, based on the 3D motion data.

According to one embodiment of the invention, the recognition device calculates corresponding 3D coordinates based on the 3D motion data, constructs corresponding 3D tracks based the 3D coordinates, and derives the corresponding 2D images from the 3D tracks by mapping the 3D tracks onto a 2D plane for handwriting recognition.

Other objects and attainments together with a fuller understanding of the invention will become apparent and appreciated by referring to the following description and claims taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is explained in further detail, and by way of example, with reference to the accompanying drawings wherein:

FIG. 1 shows a three-dimensional handwriting recognition system according to one embodiment of the invention;

FIG. 2 is a flow chart diagram illustrating a recognition process according to one embodiment of the invention;

FIG. 3A shows a 2D image of a Chinese character derived by mapping 3D tracks onto a 2D projection plane;

FIG. 3B shows a final result of handwriting recognition process based on the 2D image in FIG. 3A;

FIG. 4 shows an external design of a 3D handwriting input device according to one embodiment of the invention; and

FIG. 5 illustrates how the input device may be mounted.

Throughout the drawings, the same reference numerals indicate similar or corresponding features or functions.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 shows a three-dimensional handwriting recognition system 10 according to one embodiment of the invention. As illustrated, system 10 comprises a handwriting input device 20, a recognition device 30 and an output device 40. Input device 20 includes a 3D motion detection sensor 22, a control circuit 26 and a communication interface 28. Recognition device 30 includes a processor 32, a memory 34, a storage device 36, and a communication interface 38. For simplicity, other conventional elements are not shown in FIG. 1.

In operation, a user moves input device 20 to freely write words or characters in a 3D space, e.g., in the air. Motion detection sensor 22 detects the 3D motion and communicates the 3D motion data and a sampling rate to recognition device 30 for handwriting recognition via a communication interface 28, such as Bluetooth, Zigbee, IEEE 802.11, infrared, or a USB port. The sampling rate may be a predetermined value set by an end user or a manufacturer based on factors such as the processing capability of the system. Alternatively, the sampling rate may be dynamically determined and adjusted based on, for example, the speed of the motion. This can be done, for example, by first determining the speed of the initial motion associated with the handwriting. Then the recognition device can dynamically adjust the sampling rate based on the speed of the motion at the last sampling point. The higher the speed is, the higher the sampling rate will be, and vice versa. By dynamically adjusting the sampling rate, the accuracy of the handwriting recognition can be improved since only the optimal number of sampling points will be used for constructing the word or character. Further, lower power consumption is needed.

Based on the motion data and the sampling rate received from input device 20, processor 32 calculates the corresponding 3D coordinates on the X, Y and Z axes and stores these coordinates in storage device 36. Using the calculated coordinates, processor 32 constructs the corresponding 3D tracks. The 3D tracks will then be projected onto a 2D plane to form 2D images which will be recognized using conventional handwriting recognition software. The final result is displayed on output device 40.

Since 3D writing is a continuous process, control circuit 26 of input device 20 provides a control signal to recognition device 30 via interface 28 to indicate separation of individual words or characters upon receiving a user provided external input. For example, the user may press a control button to cause control circuit 26 to generate the control signal after completion of writing a word or a character.

According to a specific embodiment of the invention, motion detection sensor 22 detects the 3D motion by measuring the acceleration of the movement along the X, Y and Z axes. As an example, the piezoresistive-type tri-axial accelerating sensor commercially available from Hitachi Metals, Ltd., Tokyo, Japan, may be used as motion detection sensor 22. This accelerating sensor in the form of an IC chip has the ability to simultaneously detect acceleration in the three axial directions (X, Y and Z). The sensor is highly sensitive and shock resistant and is a very small and thin semiconductor type 3 axial accelerating sensor.

FIG. 2 is a flow chart diagram illustrating a recognition process 100 performed by recognition device 30, according to one embodiment of the invention. In FIG. 2, recognition device 30 receives the 3D motion data (e.g., the acceleration data of the movement in the X, Y and Z directions) and the sampling rate from input device 20 (step 102). Based on the information received, processor 32 calculates the corresponding 3D coordinates on the X, Y and Z axes for each sampling point using the starting point of the movement as the origin (step 106). Each sampling point is also used as a reference point for calculating the coordinates of the following sampling point.

Calculation of the 3D coordinates is continuously performed based on the incoming 3D motion data until processor 32 detects receipt of a control signal (step 112). The control signal indicates completion of writing a word or a character. The corresponding 3D tracks are constructed using the 3D coordinates (step 116) and are then mapped onto a 2D plane (step 122). Thereafter, conventional 2D handwriting recognition is performed (step 126).

In the above recognition process 100, to map the 3D tracks onto a 2D plane at step 122, it is necessary to first find a proper 2D projection plane. In a preferred embodiment of the invention, a proper 2D projection plane is separately derived for each word or character.

According to the geometry principles, a proper 2D projection plane is a plane, to which the sum of the distance square of each sampling point is minimal. Assume the coordinates of n sampling points are known as follows: (x1, y1, z1), (x2, y2, z2) . . . (xn, yn, zn), and the equation of the plane is Ax+By+Cz+D=0, where (A2+B2+C20). Now it is necessary to determine the values of A, B, C, D. The distance from one point (x1, y1, z1) to the plane is given as

d 1 = Ax 1 + By 1 + Cz 1 + D A 2 + B 2 + C 2 ,
and the sum

i = 1 n d i 2
represented by F(A,B,C,D) is expressed as:

F ( A , B , C , D ) = i = 1 n d i 2 = ( Ax 1 + By 1 + Cz 1 + D ) 2 + ( Ax 2 + By 2 + Cz 2 + D ) 2 + + ( Ax n + By n + Cz n + D ) 2 ( A 2 + B 2 + C 2 )

The values of A, B, C, D can be determined using the Lagrange multiplication method as follows, which is described in Mathematics Analysis by Ouyang Guangzhong, published by Fudan University Press in 1999 in China, which is hereby incorporated by reference. Thus,
F(A,B,C,D)=F′(A,B,C,D)=(Ax 1 +By 1 +Cz 1 +D)2+(Ax 2 +By 2 +Cz 2 +D)2+ . . . +(Ax n +By n +Cz n +D)2
under the constraint A2+B2+C2=1. From this equation, the following equation is derived:
G(A,B,C,D)=F′(A,B,C,D)+λ(A 2 +B 2 +C 2−1),
where λ is a Lagrange multiplier, which is a constant. The partial differential functions are now performed on G(A, B, C, D) with respect to A, B, C and D as follows:

G ( A , B , C , D ) A = 0 G ( A , B , C , D ) B = 0 G ( A , B , C , D ) C = 0 G ( A , B , C , D ) D = 0

From the above four equations, the following equations are obtained:

A ( i = 1 n ( x i * x i ) + λ ) + B i = 1 n ( x i * y i ) + C i = 1 n ( x i * z i ) + D i = 1 n x i = 0 ( 1 ) A i = 1 n ( x i * y i ) + B ( i = 1 n ( y i * y i ) + λ ) + C i = 1 n ( y i * z i ) + D i = 1 n y i = 0 ( 2 ) A i = 1 n ( x i * z i ) + B i = 1 n ( z i * y i ) + C ( i = 1 n ( z i * z i ) + λ ) + D i = 1 n z i = 0 ( 3 ) A i = 1 n x i + B i = 1 n y i + C i = 1 n z i + nD = 0 ( 4 ) A 2 + B 2 + C 2 = 1 ( 5 )
where equation (4) can be rewritten as:

D = - 1 n ( A i = 1 n x i + B i = 1 n y i + C i = 1 n z i ) ( 6 )

By incorporating equation (6) into equations (1), (2), and (3), the following is obtained:

[ i = 1 n ( x i * x i ) - 1 n i = 1 n ( x i * x i ) i = 1 n ( x i * y i ) - 1 n i = 1 n ( x i * y i ) i = 1 n ( x i * z i ) - 1 n i = 1 n ( x i * z i ) i = 1 n ( x i * y i ) - 1 n i = 1 n ( x i * y i ) i = 1 n ( y i * y i ) - 1 n i = 1 n ( y i * y i ) i = 1 n ( z i * y i ) - 1 n i = 1 n ( z i * y i ) i = 1 n ( x i * z i ) - 1 n i = 1 n ( x i * z i ) i = 1 n ( z i * y i ) - 1 n i = 1 n ( z i * y i ) i = 1 n ( z i * z i ) - 1 n i = 1 n ( z i * z i ) ] * [ A B C ] = - λ [ A B C ] ( 7 )
Thus, from the above equations, the values of A, B, C and D can be obtained.

As an example, the following 3D coordinates for a total of 13 sampling points are obtained for a Chinese character:

{(0,0,0),(0.49,−1,0.02),(1,0,0.03),(0,0,0.02),(2,−0.13, 0.01),(3.5,−0.14,0),(3.76,−0.31,−0.01),(2.74,−1.26, 0.01),(1.38,−2.24,0),(2.5,−2,0.01),(1.746,1,0.02),(2,−1,5,0.03),(1.876,−4,5,0.02),}.

By using the Lagrange multiplication method described above, the following are obtained:

[ 53.407 - 23.6725 0.1911 - 23.6725 36.2195 - 0.2084 0.1911 - 0.2084 0.0035 ] * [ A B C ] = - λ [ A B C ] ,
A 2 +B 2 +C 2=1, and

D = - 1 13 ( A x i + B y i + C z i ) .
From the above equations, the values of A, B, C and D are determined as follows:
A=0.0045, B=0.0023, C=0.9999, D=−0.01777.

Thus, the equation of the 2D projection plane is 0.0045x+0.0023y+0.9999z−0.01777=0. From the equation of the projection plane, Ax+By+Cz+D=0, and the equation of a line that is vertical to the projection plane,

x - x i A = y - y i B = z - z i C ,
the following equations are derived:

x = ( B 2 + C 2 ) x i - A ( By i + Cz i + D ) A 2 + B 2 + C 2 y = ( A 2 + C 2 ) y i - B ( Ax i + Cz i + D ) A 2 + B 2 + C 2
which can be used to obtain the corresponding 2D coordinates for each 3D sampling point. In this example, the following corresponding 2D coordinates are obtained: {(0.0001,0),(0.49,−1),(1,−2.0001),(0,0),(2,−0.13),(3.5001,−0.14),(3.7601,−0.31),(2.7401,−1.26),(1.3801, 2.24),(2.5001,−2),(1.746,1),(2,−1.5001),(1.876,−4.5001)}.
Based on these 2D coordinates, the corresponding 2D image is projected onto the 2D projection plane. The result of this Chinese character is shown in FIG. 3A. The final result is generated by performing a conventional 2D handwriting recognition process, as shown in FIG. 3B.

In the above, the Lagrange multiplication method is used to obtain the values of A, B, C and D. Other methods such as linear regression method may also be used.

FIG. 4 shows an external design of a 3D handwriting input device 200 according to one embodiment of the invention. As shown in FIG. 4, input device 200 includes a housing 210 that contains the electronics parts of the device (such as a 3D motion detection sensor IC chip), a control button 212 for allowing a user to input a control signal to indicate completion of writing a word or character, and a band 220 for mounting input device 200 on the user's finger.

FIG. 5 illustrates how input device 200 may be mounted. In FIG. 5, device 200 is mounted on a finger 232 of a user's hand 230. By mounting it on the finger, the user can simply move the finger to write any word or character in a 3D space. The 3D motion data will be wirelessly transmitted to a recognition device for handwriting recognition.

According to the invention, the input device and the recognition device can be integrated together as a single unit (e.g., a pen) that operates in the same manner as described above. The final recognition results will be transmitted to an intelligent apparatus such as a PDA, a laptop computer, a PC, etc.

The invention can also be used on a 2D plane if the user so chooses. In such a case, the coordinates calculated will be for 2D plane and the step of mapping the 3D tracks onto a 2D plane is omitted.

While the invention has been described in conjunction with specific embodiments, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art in light of the foregoing description. Accordingly, it is intended to embrace all such alternatives, modifications and variations as fall within the spirit and scope of the appended claims.

Claims (29)

The invention claimed is:
1. A handwriting recognition system, comprising:
an input device including a three-dimensional (3D) motion detection sensor that is configured to generate 3D motion data in response to a 3D motion; and
a recognition device, in communication with the input device, that is configured to receive the 3D motion data and derive corresponding two-dimensional (2D) images for handwriting recognition wherein a proper 2D projection plane is derived, using the 3D motion data, for each word or character; said proper 2D projection plane is where a sum of the distance square of each 3D sampling point is minimal, wherein the sum of the distance square is determined using a predetermined algorithm and a predetermined number of 3D sampling points and their respective distances to the 2D projection plane; and wherein the recognition device is further configured to dynamically adjust the sampling rate of the 3D motion data based on the speed of the 3D motion.
2. The system of claim 1, wherein the recognition device includes means for performing 2D handwriting recognition based on the 2D images.
3. The system of claim 1, wherein the recognition device includes:
means for calculating corresponding 3D coordinates based on the 3D motion data;
means for constructing corresponding 3D tracks based the 3D coordinates; and
means for deriving the corresponding 2D images from the 3D tracks.
4. The system of claim 3, wherein the deriving means includes means for mapping the 3D tracks onto a 2D plane for deriving the 2D images for handwriting recognition.
5. The system of claim 3, wherein the recognition device includes means for performing 2D handwriting recognition based on the 2D images.
6. The system of claim 4, wherein the calculating means calculates the corresponding 3D coordinates of each sampling point based on the 3D motion data and a selected sampling rate.
7. The system of claim 3, wherein the input device further includes a control circuit, responsive to a user's command, that is configured to generate a control signal for transmitting to the recognition device to indicate completion of writing a word or a character.
8. The system of claim 3, wherein the motion detection sensor measures acceleration of the 3D motion in X, Y and Z axial directions to generate the 3D motion data.
9. The system of claim 5, further comprising an output device for displaying final results of the handwriting recognition.
10. The system of claim 1, wherein the input device further includes a control circuit, responsive to a user's command, that is configured to generate a control signal for transmitting to the recognition device to indicate completion of writing a word or a character.
11. The system of claim 1, wherein the motion detection sensor measures acceleration of the 3D motion in X, Y and Z axial directions to generate the 3D motion data.
12. The system of claim 1, wherein the input device wirelessly transmits the 3D motion data to the recognition device.
13. The system of claim 1, wherein the predetermined algorithm is
F ( A , B , C , D ) = i = 1 n d i 2 = ( ( Ax 1 + By 1 + Cz 1 + D ) 2 + ( Ax 2 + By 2 + Cz 2 + D ) 2 + + ( Ax n + By n + Cz n + D ) 2 ) ( A 2 + B 2 + C 2 )
where (x1, y1, z1), (x2, y2, z2) . . . , (xn, yn, zn) are the predetermined sampling points, and A, B, C and D are determined using a linear regression method.
14. The system of claim 13, wherein the derived corresponding two-dimensional (2D) images for handwriting recognition is derived from the predetermined algorithm to be
x = ( B 2 + C 2 ) x i - A ( By i + Cz i + D ) A 2 + B 2 + C 2 y = ( A 2 + C 2 ) y i - B ( Ax i + Cz i + D ) A 2 + B 2 + C 2
where x and y are 2D coordinants of the two-dimensional (2D) images on said proper 2D projection plane.
15. A computing system, comprising:
a memory;
an input device including a three-dimensional (3D) motion detection sensor that is configured to generate 3D motion data in response to a 3D motion; and
a recognition device, operably coupled to the memory and in communication with the input device, that is configured to receive the 3D motion data and derive corresponding two-dimensional (2D) images for handwriting recognition wherein a proper 2D projection plane is derived, using the 3D motion data, for each word or character; said proper 2D projection plane is where a sum of the distance square of each 3D sampling point is minimal, wherein the sum of the distance square is determined using a predetermined algorithm and a predetermined number of 3D sampling points and their respective distances to the 2D projection plane; and wherein the recognition device is further configured to dynamically adjust the sampling rate of the 3D motion data based on the speed of the 3D motion.
16. The system of claim 15, wherein the recognition device includes means for performing 2D handwriting recognition based on the 2D images.
17. The system of claim 15, wherein the recognition device includes:
means for calculating corresponding 3D coordinates based on the 3D motion data;
means for constructing corresponding 3D tracks based the 3D coordinates; and
means for deriving the corresponding 2D images from the 3D tracks.
18. The system of claim 17, wherein the deriving means includes means for mapping the 3D tracks onto a 2D plane for deriving the 2D images for handwriting recognition.
19. A handwriting recognition method using a recognition system having a three-dimensional (3D) motion detection sensor and a recognition device, the method comprising the steps of:
generating 3D motion data in response to a 3D motion, using the three-dimensional (3D) motion detection sensor;
deriving corresponding two-dimensional (2D) images for handwriting recognition wherein a proper 2D projection plane is derived, using the 3D motion data, for each word or character; said proper 2D projection plane is where a sum of the distance square of each 3D sampling point is minimal, wherein the sum of the distance square is determined using a predetermined algorithm and a predetermined number of 3D sampling points and their respective distances to the 2D projection plane; and
dynamically adjusting the sampling rate of the 3D motion based on the speed of the 3D motion.
20. The method of claim 19, further comprising the step of performing 2D handwriting recognition based on the 2D images.
21. The method of claim 19, further comprising the steps of:
calculating corresponding 3D coordinates based on the 3D motion data;
constructing corresponding 3D tracks based the 3D coordinates; and
deriving the corresponding 2D images from the 3D tracks.
22. The method of claim 21, wherein the step of deriving the corresponding 2D images from the 3D tracks includes mapping the 3D tracks onto a 2D plane for deriving the 2D images for handwriting recognition.
23. The method of claim 21, further comprising a step of performing a 2D handwriting recognition based on the 2D images.
24. The method of claim 22, wherein the corresponding 3D coordinates of each sampling point are calculated based on the 3D motion data and a selected sampling rate.
25. The method of claim 21, further comprising the step of generating an indication, based on a user's command, to indicate completion of writing a word or a character.
26. The method of claim 21, further comprising the step of measuring acceleration of the 3D motion in X, Y and Z axial directions and wherein the 3D motion data are generated based on the acceleration of the 3D motion in the X, Y and Z axial directions.
27. The method of claim 23, further comprising a step of displaying final results of the handwriting recognition.
28. The method of claim 19, further comprising the steps of wirelessly transmitting the 3D motion data and wirelessly receiving the 3D motion data for calculating the 3D coordinates.
29. The method of claim 19, further comprising the step of measuring acceleration of the 3D motion in X, Y and Z axial directions and wherein the 3D motion data are generated based on the acceleration of the 3D motion in the X, Y and Z axial directions.
US10528938 2002-09-28 2003-09-18 Method and system for three-dimensional handwriting recognition Active 2026-11-21 US8150162B2 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
CN02144248 2002-09-28
CN 02144248 CN100377043C (en) 2002-09-28 2002-09-28 Three-dimensional hand-written identification process and system thereof
EP0229406.2 2002-09-30
PCT/IB2003/004102 WO2004029866A1 (en) 2002-09-28 2003-09-18 Method and system for three-dimensional handwriting recognition

Publications (2)

Publication Number Publication Date
US20060149737A1 true US20060149737A1 (en) 2006-07-06
US8150162B2 true US8150162B2 (en) 2012-04-03

Family

ID=32034737

Family Applications (1)

Application Number Title Priority Date Filing Date
US10528938 Active 2026-11-21 US8150162B2 (en) 2002-09-28 2003-09-18 Method and system for three-dimensional handwriting recognition

Country Status (5)

Country Link
US (1) US8150162B2 (en)
EP (1) EP1546993B1 (en)
JP (1) JP2006500680A (en)
CN (1) CN100377043C (en)
WO (1) WO2004029866A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090245645A1 (en) * 2008-03-28 2009-10-01 Smart Technologies Inc. Method and tool for recognizing a hand-drawn table
US20120020566A1 (en) * 2010-07-26 2012-01-26 Casio Computer Co., Ltd. Character recognition device and recording medium
US20130035890A1 (en) * 2011-08-04 2013-02-07 Wang Jeen-Shing Moving trajectory calibration method and moving trajectory generation method
WO2014011009A1 (en) * 2012-07-13 2014-01-16 Samsung Electronics Co., Ltd. Portable terminal using touch pen and handwriting input method using the same

Families Citing this family (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1512298A (en) * 2002-12-26 2004-07-14 皇家飞利浦电子股份有限公司 Method for three dimension hand writing identification and its system
KR100465241B1 (en) * 2003-03-17 2005-01-13 삼성전자주식회사 Motion recognition system using a imaginary writing plane and method thereof
US8036465B2 (en) 2003-09-26 2011-10-11 Khomo Malome T Method of text interaction using chirographic techniques
US7317450B2 (en) 2003-09-26 2008-01-08 Khomo Malome T Spatial chirographic sign reader
US20050069203A1 (en) * 2003-09-26 2005-03-31 Khomo Malome T. Spatial character recognition technique and chirographic text character reader
DE102004008253A1 (en) * 2004-02-19 2005-09-15 Siemens Ag A method for estimating a virtual writing plane
JP2007079943A (en) 2005-09-14 2007-03-29 Toshiba Corp Character reading program, character reading method and character reader
US20090115744A1 (en) 2007-11-06 2009-05-07 Innovative Material Solutions, Inc. Electronic freeboard writing system
US8166421B2 (en) * 2008-01-14 2012-04-24 Primesense Ltd. Three-dimensional user interface
US9035876B2 (en) 2008-01-14 2015-05-19 Apple Inc. Three-dimensional user interface session control
US20090300671A1 (en) * 2008-05-30 2009-12-03 At&T Knowledge Ventures, L.P. Controlling Access to Multimedia Content
CN101751200B (en) 2008-12-09 2012-01-11 三星电子株式会社 Space input method for mobile terminal and implementation device thereof
US8378967B2 (en) * 2009-02-27 2013-02-19 Denso Corporation Wearable electrical apparatus
JP4683148B2 (en) * 2009-02-27 2011-05-11 株式会社デンソー Electrical equipment
CN102163119A (en) * 2010-02-23 2011-08-24 中兴通讯股份有限公司 Single-hand inputting method and device
US8787663B2 (en) * 2010-03-01 2014-07-22 Primesense Ltd. Tracking body parts by combined color image and depth processing
CN102959616B (en) 2010-07-20 2015-06-10 苹果公司 Interactive reality augmentation for natural interaction
US9201501B2 (en) 2010-07-20 2015-12-01 Apple Inc. Adaptive projector
JP4877621B1 (en) * 2010-07-30 2012-02-15 カシオ計算機株式会社 Character recognition device and program
US8959013B2 (en) 2010-09-27 2015-02-17 Apple Inc. Virtual keyboard for a non-tactile three dimensional user interface
CN102004623B (en) * 2010-11-29 2013-02-27 深圳市九洲电器有限公司 Three-dimensional image display device and method
US8872762B2 (en) 2010-12-08 2014-10-28 Primesense Ltd. Three dimensional user interface cursor control
US8933876B2 (en) 2010-12-13 2015-01-13 Apple Inc. Three dimensional user interface session control
US9285874B2 (en) 2011-02-09 2016-03-15 Apple Inc. Gaze detection in a 3D mapping environment
CN102650905A (en) * 2011-02-23 2012-08-29 西安龙飞软件有限公司 Method utilizing gesture operation in three-dimensional space to realize word input of mobile phone
CN102135821B (en) * 2011-03-08 2013-03-06 中国科学技术大学 Handwriting pen and graphic restoration system
JP2012208602A (en) * 2011-03-29 2012-10-25 Kyocera Corp Information recognition device
US9377865B2 (en) 2011-07-05 2016-06-28 Apple Inc. Zoom-based gesture user interface
US8881051B2 (en) 2011-07-05 2014-11-04 Primesense Ltd Zoom-based gesture user interface
US9459758B2 (en) 2011-07-05 2016-10-04 Apple Inc. Gesture-based interface with enhanced features
US9030498B2 (en) 2011-08-15 2015-05-12 Apple Inc. Combining explicit select gestures and timeclick in a non-tactile three dimensional user interface
US9218063B2 (en) 2011-08-24 2015-12-22 Apple Inc. Sessionless pointing user interface
US9122311B2 (en) 2011-08-24 2015-09-01 Apple Inc. Visual feedback for tactile and non-tactile user interfaces
CN102262732A (en) * 2011-08-26 2011-11-30 信源通科技(深圳)有限公司 A character recognition method and system
CN102306060B (en) * 2011-08-26 2014-07-09 深圳市优利麦克科技开发有限公司 Input method and system of mobile equipment
CN102253718B (en) * 2011-08-31 2016-07-06 由田信息技术(上海)有限公司 Handwriting input method of three-dimensional space
CN103076896B (en) * 2011-10-26 2016-08-17 联想(北京)有限公司 An apparatus and a method and a terminal and a system input
US9229534B2 (en) 2012-02-28 2016-01-05 Apple Inc. Asymmetric mapping for tactile and non-tactile user interfaces
CN102662465A (en) * 2012-03-26 2012-09-12 北京国铁华晨通信信息技术有限公司 Method and system for inputting visual character based on dynamic track
US9377863B2 (en) 2012-03-26 2016-06-28 Apple Inc. Gaze-enhanced virtual touchscreen
CN102810008B (en) * 2012-05-16 2016-01-13 北京捷通华声语音技术有限公司 An air-input system, method and air input acquisition device
CN104571603B (en) * 2013-10-10 2018-02-27 北京壹人壹本信息科技有限公司 Air kinds of handwriting system and stylus
CN104714650B (en) * 2015-04-02 2017-11-24 三星电子(中国)研发中心 An information input method and apparatus

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1995021436A1 (en) 1994-02-04 1995-08-10 Baron Motion Communications, Inc. Improved information input apparatus
US5568565A (en) * 1992-03-05 1996-10-22 International Business Machines Corporation Handwriting input method and apparatus
US6212296B1 (en) * 1997-12-23 2001-04-03 Ricoh Company, Ltd. Method and apparatus for transforming sensor signals into graphical images
US6226404B1 (en) * 1997-06-09 2001-05-01 Nec Corporation On-line character recognition system
US6233351B1 (en) * 1994-08-12 2001-05-15 Dassault Systemes Of America Corp. Method and apparatus for processing a freehand sketch
US20010004254A1 (en) * 1998-08-10 2001-06-21 Tohru Okahara Terminal operation apparatus
US6628847B1 (en) * 1998-02-27 2003-09-30 Carnegie Mellon University Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US20030215140A1 (en) * 2002-05-14 2003-11-20 Microsoft Corporation Interfacing with ink
US20040003071A1 (en) * 2002-06-28 2004-01-01 Microsoft Corporation Parental controls customization and notification
US6737591B1 (en) * 1999-05-25 2004-05-18 Silverbrook Research Pty Ltd Orientation sensing device

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0628096B2 (en) 1984-07-02 1994-04-13 東レ株式会社 Polyester film for a magnetic recording medium
JP3450355B2 (en) * 1992-07-07 2003-09-22 株式会社東芝 Space handwriting graphical input device and method
JP3456735B2 (en) * 1994-02-21 2003-10-14 沖電気工業株式会社 Information input device and a presentation system using the same
JPH08320756A (en) * 1995-05-24 1996-12-03 Sharp Corp Electronic equipment with data input means
US6081261A (en) * 1995-11-01 2000-06-27 Ricoh Corporation Manual entry interactive paper and electronic document handling and processing system

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5568565A (en) * 1992-03-05 1996-10-22 International Business Machines Corporation Handwriting input method and apparatus
WO1995021436A1 (en) 1994-02-04 1995-08-10 Baron Motion Communications, Inc. Improved information input apparatus
US6233351B1 (en) * 1994-08-12 2001-05-15 Dassault Systemes Of America Corp. Method and apparatus for processing a freehand sketch
US6226404B1 (en) * 1997-06-09 2001-05-01 Nec Corporation On-line character recognition system
US6212296B1 (en) * 1997-12-23 2001-04-03 Ricoh Company, Ltd. Method and apparatus for transforming sensor signals into graphical images
US6628847B1 (en) * 1998-02-27 2003-09-30 Carnegie Mellon University Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US20010004254A1 (en) * 1998-08-10 2001-06-21 Tohru Okahara Terminal operation apparatus
US6737591B1 (en) * 1999-05-25 2004-05-18 Silverbrook Research Pty Ltd Orientation sensing device
US20030215140A1 (en) * 2002-05-14 2003-11-20 Microsoft Corporation Interfacing with ink
US20040003071A1 (en) * 2002-06-28 2004-01-01 Microsoft Corporation Parental controls customization and notification

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090245645A1 (en) * 2008-03-28 2009-10-01 Smart Technologies Inc. Method and tool for recognizing a hand-drawn table
US8634645B2 (en) * 2008-03-28 2014-01-21 Smart Technologies Ulc Method and tool for recognizing a hand-drawn table
US8538156B2 (en) * 2010-07-26 2013-09-17 Casio Computer Co., Ltd. Character recognition device and recording medium
US8571321B2 (en) * 2010-07-26 2013-10-29 Casio Computer Co., Ltd. Character recognition device and recording medium
US20120020566A1 (en) * 2010-07-26 2012-01-26 Casio Computer Co., Ltd. Character recognition device and recording medium
US20130035890A1 (en) * 2011-08-04 2013-02-07 Wang Jeen-Shing Moving trajectory calibration method and moving trajectory generation method
US20130069917A1 (en) * 2011-08-04 2013-03-21 Jeen-Shing WANG Moving trajectory generation method
US9098123B2 (en) * 2011-08-04 2015-08-04 National Cheng Kung University Moving trajectory generation method
WO2014011009A1 (en) * 2012-07-13 2014-01-16 Samsung Electronics Co., Ltd. Portable terminal using touch pen and handwriting input method using the same
US9898186B2 (en) 2012-07-13 2018-02-20 Samsung Electronics Co., Ltd. Portable terminal using touch pen and handwriting input method using the same

Also Published As

Publication number Publication date Type
JP2006500680A (en) 2006-01-05 application
CN1485711A (en) 2004-03-31 application
EP1546993A1 (en) 2005-06-29 application
EP1546993B1 (en) 2015-07-15 grant
CN100377043C (en) 2008-03-26 grant
WO2004029866A1 (en) 2004-04-08 application
US20060149737A1 (en) 2006-07-06 application

Similar Documents

Publication Publication Date Title
US6340957B1 (en) Dynamically relocatable tileable displays
US6297805B1 (en) Multiple interacting computers interfaceable through a physical manipulatory grammar
US6512838B1 (en) Methods for enhancing performance and data acquired from three-dimensional image systems
US6243074B1 (en) Handedness detection for a physical manipulatory grammar
US7082578B1 (en) Computer user interface using a physical manipulatory grammar
US6891527B1 (en) Processing signals to determine spatial positions
US8570273B1 (en) Input device configured to control a computing device
US6292179B1 (en) Software keyboard system using trace of stylus on a touch screen and method for recognizing key code using the same
US20080005703A1 (en) Apparatus, Methods and computer program products providing finger-based and hand-based gesture commands for portable electronic device applications
US20120019488A1 (en) Stylus for a touchscreen display
US20060119588A1 (en) Apparatus and method of processing information input using a touchpad
US20050168437A1 (en) Processing pose data derived from the pose of an elongate object
US20090183929A1 (en) Writing system with camera
US20060279549A1 (en) Writing system
US5581484A (en) Finger mounted computer input device
US20080134784A1 (en) Inertial input apparatus with six-axial detection ability and the operating method thereof
US5347477A (en) Pen-based form computer
US20070130547A1 (en) Method and system for touchless user interface control
US20120256835A1 (en) Motion control used as controlling device
Hinckley et al. Input technologies and techniques
US6614422B1 (en) Method and apparatus for entering data using a virtual input device
US6856327B2 (en) Apparatus for moving display screen of mobile computer device
US20060028457A1 (en) Stylus-Based Computer Input System
WO2005103863A2 (en) Distinguishing tilt and translation motion components in handheld devices
US5247137A (en) Autonomous computer input device and marking instrument

Legal Events

Date Code Title Description
AS Assignment

Owner name: KONINKLIJKE PHILIPS ELECTRONICS, N.V., NETHERLANDS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:DU, YONGGANG;TU, JAIWAN;FENG, LEI;AND OTHERS;REEL/FRAME:016668/0448

Effective date: 20031205

AS Assignment

Owner name: KONINKLIJKE PHILIPS ELECTRONICS, N.V., NETHERLANDS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YONGGANG, DU;JIAWEN, TU;LEI, FANG;AND OTHERS;REEL/FRAME:017302/0026

Effective date: 20031205

FPAY Fee payment

Year of fee payment: 4