WO2012024829A1 - Method and apparatus for segmenting strokes of overlapped handwriting into one or more groups - Google Patents

Method and apparatus for segmenting strokes of overlapped handwriting into one or more groups Download PDF

Info

Publication number
WO2012024829A1
WO2012024829A1 PCT/CN2010/076285 CN2010076285W WO2012024829A1 WO 2012024829 A1 WO2012024829 A1 WO 2012024829A1 CN 2010076285 W CN2010076285 W CN 2010076285W WO 2012024829 A1 WO2012024829 A1 WO 2012024829A1
Authority
WO
WIPO (PCT)
Prior art keywords
strokes
series
stroke
features
program code
Prior art date
Application number
PCT/CN2010/076285
Other languages
French (fr)
Inventor
Yanming Zou
Yinzi Chen
Changsong Liu
Xiang Wan
Original Assignee
Nokia Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nokia Corporation filed Critical Nokia Corporation
Priority to JP2013525107A priority Critical patent/JP5581448B2/en
Priority to KR1020137007240A priority patent/KR101486174B1/en
Priority to PCT/CN2010/076285 priority patent/WO2012024829A1/en
Priority to CN201080068735.8A priority patent/CN103080878B/en
Publication of WO2012024829A1 publication Critical patent/WO2012024829A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition
    • G06V30/32Digital ink
    • G06V30/333Preprocessing; Feature extraction
    • G06V30/347Sampling; Contour coding; Stroke extraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/10Image acquisition
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition
    • G06V30/22Character recognition characterised by the type of writing
    • G06V30/226Character recognition characterised by the type of writing of cursive writing
    • G06V30/2268Character recognition characterised by the type of writing of cursive writing using stroke segmentation

Definitions

  • An example embodiment of the present invention relates to processing of overlapped handwriting and, more particularly, to segmenting a series of strokes that comprise the overlapped handwriting into one or more groups of strokes.
  • a variety of devices capture or otherwise receive handwriting input via a touch screen or other input device.
  • a number of computers, tablets, mobile telephones, personal digital assistants (PDAs) and other types of electronic devices include touch screens which permit a user to provide tactile input, such as in the form of handwriting.
  • the handwritten input may be captured by the electronic device and processed in an effort to recognize the handwritten characters in order to inform future actions of the electronic device including, for example, storage or transmission of a representation of the handwritten input.
  • handwritten input is generally processed so as to segment the handwritten input into different characters, such as letters, numerals or other characters.
  • segmenting handwritten input There are several different techniques for segmenting handwritten input.
  • One technique makes use of a time out between the input of each character.
  • the user interface such as a touch screen, need only include a single writing area. A user may write a single character within the writing area. A time out is then taken in order to receive the character and to clear the writing area in order to prepare the writing area for receipt of the next character. Thus, the time out taken between entry of each character provides for segmentation of the handwriting.
  • Another technique employs a user interface having two or more writing areas, such as two or more touch screens.
  • a user can write a single character in one of the writing areas and may then switch to the other writing area in order to write the next character.
  • the character that was previously written in the other writing area may be received and processed and the other writing area may then be cleared so as to prepare the other writing area for receipt of the next character.
  • the alteration between the different writing areas provides for segmentation of the handwriting.
  • a further technique employs a single writing area that is relatively large. Within a writing area, a user can write several characters on a continuous basis one after another in a comparable fashion to the manner in which characters are written on a piece of paper. This technique utilizes the spatial intervals that are positioned between the written characters as well as underlying intelligence in order to appropriately segment the handwritten input.
  • the user interface provides a single writing area in which a user can write several characters on a continuous basis with the characters being overlapped with one another without a spatial interval between the characters.
  • the overlapped characters may be segmented so as to separate one character from the others.
  • the procedure is much more complicated than the above techniques with a recognition process usually being employed.
  • the recognition process can segment a series of strokes into characters with errors.
  • the recognition process is very computing intensive, and is very challenging for real-time operation, especially for small devices.
  • overlapped continuous handwriting permit electronic devices having relatively small user interfaces to receive a continuous handwritten input using the fingers of the user or other stylus such that the handwritten input may be provided in a relatively rapid fashion in comparison to the input of a single character at a time.
  • the reliance upon overlapped continuous handwriting may pose challenges regarding segmentation and user interaction.
  • the overlapped continuous handwriting may increase the difficulties associated with efficiently segmenting the overlapped handwriting in order to separate each character from the others, particularly in comparison to more common continuous handwriting having spatial intervals between the written characters.
  • the overlapping of multiple characters may create difficulties for the user in that the user may not be able to clearly view what the user is currently writing as a result of the noisy and complicated background created by the other overlapped characters.
  • a method, apparatus and computer program product are provided in accordance with an example embodiment in order to segment a series of strokes that comprise overlapped handwriting into one or more groups of strokes.
  • This segmentation may be performed prior to any segmentation into one or more characters and, indeed, each group of strokes may be either a character or a portion of a character, but, in any event, no group of strokes includes strokes from more than one character.
  • the subsequent segmentation of the series of strokes into one or more characters is made more efficient.
  • one or more prior groups of strokes may be displayed in a less visibly distinct manner, while continuing to receive the overlapped handwriting in order to permit the user to more clearly view the most recent strokes.
  • a method receives a series of strokes that comprise overlapped handwriting and, for each of the plurality of strokes, determines, with a processor, a plurality of features associated with the current stroke based upon geometrical properties of the series of strokes.
  • the method of this embodiment also segments a series of strokes into one or more groups of strokes based upon the features associated with the strokes.
  • each group of strokes is one of a character or a portion of a character, but no group of strokes includes strokes from more than one character.
  • the method of one embodiment may also segment the series of strokes into one or more characters based upon the one or more groups of strokes into which the series of strokes has been segmented, thereby increasing the efficiency with which a series of strokes may be segmented into characters.
  • the method may determine a plurality of features associated with the current stroke by determining the plurality of features associated with the current stroke based only on the geometrical properties of the series of strokes.
  • the method may also cause a display of at least some of the groups such that at least one group is caused to be displayed in a visibly distinct manner from at least one other group.
  • the displayed image of the overlapped handwriting may be simplified to facilitate the user's view of, for example, the most recent strokes.
  • the method also normalizes the plurality of features associated with the current stroke.
  • the plurality of features may be normalized based upon an overall dimension of a series of strokes.
  • the plurality of features may be selected from the group consisting of an endpoint of the current stroke, a geometric center of the current stroke, a beginning point of the next stroke, a geometric center of the next stroke, a smallest rectangle containing the current stroke, a smallest rectangle containing the next stroke and a smallest rectangle containing an immediately prior stroke.
  • the method may be performed in an incremental mode or a batch mode.
  • an incremental mode a determination of the plurality of features and the segmentation of the series of strokes are repeated following receipt of each successive stroke.
  • the batch mode the determination of the plurality of features and the segmentation of the series of strokes are repeated following receipt of a plurality of strokes.
  • an apparatus in another embodiment, includes at least one processor and at least one memory including computer program code.
  • the at least one memory and computer program code of the apparatus of this embodiment are configured, with the at least one processor, to cause the apparatus to at least receive a series of strokes that comprise overlapped handwriting and, for each of the plurality of strokes, determine a plurality of features associated with the current stroke based upon geometrical properties of the series of strokes.
  • the at least one memory and computer program code of the apparatus of this embodiment are also configured, with the at least one processor, to cause the apparatus to segment a series of strokes into one or more groups of strokes based upon the features associated with the strokes.
  • each group of strokes is one of a character or a portion of a character, but no group of strokes includes strokes from more than one character.
  • the at least one memory and computer program code of the apparatus of one embodiment may also be configured, with the at least one processor, to cause the apparatus to segment the series of strokes into one or more characters based upon the one or more groups of strokes into which the series of strokes has been segmented, thereby increasing the efficiency with which a series of strokes may be segmented into characters.
  • the at least one memory and computer program code of the apparatus of one example embodiment may also be configured, with the at least one processor, to cause the apparatus to determine a plurality of features associated with the current stroke by determining the plurality of features associated with the current stroke based only on the geometrical properties of the series of strokes.
  • the at least one memory and computer program code of the apparatus may also be configured, with the at least one processor, to cause the apparatus to cause a display of at least some of the groups such that at least one group is caused to be displayed in a visibly distinct manner from at least one other group.
  • the displayed image of the overlapped handwriting may be simplified to facilitate the user's view of, for example, the most recent strokes.
  • the at least one memory and computer program code of the apparatus may also be configured, with the at least one processor, to cause the apparatus to normalize the plurality of features associated with the current stroke.
  • the plurality of features may be normalized based upon an overall dimension of a series of strokes.
  • the plurality of features may be selected from the group consisting of an endpoint of the current stroke, a geometric center of the current stroke, a beginning point of the next stroke, a geometric center of the next stroke, a smallest rectangle containing the current stroke, a smallest rectangle containing the next stroke and a smallest rectangle containing an immediately prior stroke.
  • the analysis of the strokes may be performed in an incremental mode or a batch mode.
  • an incremental mode a determination of the plurality of features and the segmentation of the series of strokes are repeated following receipt of each successive stroke.
  • the batch mode the determination of the plurality of features and the segmentation of the series of strokes are repeated following receipt of a plurality of strokes.
  • an apparatus in a further embodiment, includes means for receiving a series of strokes that comprise overlapped handwriting and, for each of the plurality of strokes, determining a plurality of features associated with the current stroke based upon geometrical properties of the series of strokes.
  • the apparatus of this embodiment also includes means for segmenting a series of strokes into one or more groups of strokes based upon the features associated with the strokes. As noted above, each group of strokes is one of a character or a portion of a character, but no group of strokes includes strokes from more than one character.
  • the apparatus of one embodiment may also include means for segmenting the series of strokes into one or more characters based upon the one or more groups of strokes into which the series of strokes has been segmented, thereby increasing the efficiency with which a series of strokes may be segmented into characters.
  • the means for determining a plurality of features associated with the current stroke includes means for determining the plurality of features associated with the current stroke based only on the geometrical properties of the series of strokes.
  • the apparatus may also include means for causing a display of at least some of the groups such that at least one group is caused to be displayed in a visibly distinct manner from at least one other group.
  • the displayed image of the overlapped handwriting may be simplified to facilitate the user's view of, for example, the most recent strokes.
  • the apparatus also includes means for normalizing the plurality of features associated with the current stroke.
  • the plurality of features may be normalized based upon an overall dimension of a series of strokes.
  • the plurality of features may be selected from the group consisting of an endpoint of the current stroke, a geometric center of the current stroke, a beginning point of the next stroke, a geometric center of the next stroke, a smallest rectangle containing the current stroke, a smallest rectangle containing the next stroke and a smallest rectangle containing an immediately prior stroke.
  • the apparatus may analyze the strokes in either an incremental mode or a batch mode.
  • an incremental mode a determination of the plurality of features and the segmentation of the series of strokes are repeated following receipt of each successive stroke.
  • the batch mode the determination of the plurality of features and the segmentation of the series of strokes are repeated following receipt of a plurality of strokes.
  • a computer program product includes at least one computer-readable memory having computer-executable code portions stored therein.
  • the computer-executable code portions include program code instructions for receiving a series of strokes that comprise overlapped handwriting and program code instructions for determining, for each of the plurality of strokes, a plurality of features associated with the current stroke based upon geometrical properties of the series of strokes.
  • the computer-executable code portions of this embodiment also include program code instructions for segmenting a series of strokes into one or more groups of strokes based upon the features associated with the strokes. As noted above, each group of strokes is one of a character or a portion of a character, but no group of strokes includes strokes from more than one character.
  • the computer-executable code portions of one embodiment may also include program code instructions for segmenting the series of strokes into one or more characters based upon the one or more groups of strokes into which the series of strokes has been segmented, thereby increasing the efficiency with which a series of strokes may be segmented into characters.
  • the computer-executable code portions of one example embodiment may also include program code instructions for determining a plurality of features associated with the current stroke by determining the plurality of features associated with the current stroke based only on the geometrical properties of the series of strokes.
  • the computer-executable code portions may also include program code instructions for causing a display of at least some of the groups such that at least one group is caused to be displayed in a visibly distinct manner from at least one other group.
  • the displayed image of the overlapped handwriting may be simplified to facilitate the user's view of, for example, the most recent strokes.
  • the computer-executable code portions also include program code instructions for normalizing the plurality of features associated with the current stroke.
  • the plurality of features may be normalized based upon an overall dimension of a series of strokes.
  • the plurality of features may be selected from the group consisting of an endpoint of the current stroke, a geometric center of the current stroke, a beginning point of the next stroke, a geometric center of the next stroke, a smallest rectangle containing the current stroke, a smallest rectangle containing the next stroke and a smallest rectangle containing an immediately prior stroke.
  • the computer program product may analyze the strokes in either an incremental mode or a batch mode.
  • an incremental mode a determination of the plurality of features and the segmentation of the series of strokes are repeated following receipt of each successive stroke.
  • the batch mode the determination of the plurality of features and the segmentation of the series of strokes are repeated following receipt of a plurality of strokes.
  • Figure 1 is an illustration of overlapped handwriting in which four characters are overlapped
  • Figure 2 is a block diagram of an apparatus in accordance with an example embodiment of the present invention.
  • Figure 3 is a functional block diagram of operations performed in accordance with an example embodiment of the present invention.
  • Figure 4 is a flow chart illustrating operations performed in accordance with an example embodiment of the present invention.
  • Figure 5 is an illustration of an overlapped handwriting in which features of three successive strokes are identified in accordance with an example embodiment of the present invention
  • Figure 6 is an illustration of a manner in which the overlapped handwriting of Figure 1 may be processed in order to recognize the four characters in accordance with an example embodiment of the present invention
  • Figure 7 is a flowchart illustrating operations performed in accordance with an incremental mode of an example embodiment of the present invention.
  • Figure 8 is a flowchart illustrating the operations performed in accordance with a batch mode of another example embodiment of the present invention.
  • circuitry refers to (a) hardware-only circuit implementations (e.g., implementations in analog circuitry and/or digital circuitry); (b) combinations of circuits and computer program product(s) comprising software and/or firmware instructions stored on one or more computer readable memories that work together to cause an apparatus to perform one or more functions described herein; and (c) circuits, such as, for example, a microprocessor(s) or a portion of a
  • the term 'circuitry' also includes an implementation comprising one or more processors and/or portion(s) thereof and accompanying software and/or firmware.
  • the term 'circuitry' as used herein also includes, for example, a baseband integrated circuit or applications processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network device, other network device, and/or other computing device.
  • handwriting may be input via a user interface, such as a touch screen, in an overlapped fashion in which characters are continuously overlapped without spatial or temporal separation between the characters.
  • a user interface such as a touch screen
  • display 102 illustrates the entry of the first character
  • display 104 illustrates the overlapped entry of the first and second characters
  • display 106 illustrates the overlapped entry of the first, second and third characters
  • display 108 illustrates the overlapped entry of all four characters.
  • Overlapped handwriting allows a user to make full use of the user interface, which is particularly useful for users who utilize their fingers to provide handwriting input, such as on a relatively small and low resolution touch screen. Additionally, the use of overlapped handwriting provides a relatively natural and quick technique for receiving handwritten input. As will be recognized from reference to the increasingly crowded display of Figure 1 as additional characters are overlapped with prior characters, the reliance upon overlapped handwriting to provide handwritten input poses a challenge in regards to the segmentation of the different characters. Additionally, overlapped handwriting may also make it relatively difficult for a user to view the character that the user is currently writing or an immediately preceding character as a result of the concurrent display of a number of other overlapped characters. Accordingly, a technique for pre-segmenting overlapped handwriting is provided in accordance with an example embodiment of the present invention.
  • the overlapped handwriting that may be pre-segmented may be received via any of a wide variety of input devices, such as user interfaces, e.g., touch screens or the like. Not only may the overlapped handwriting be received via a wide variety of different input devices, but the input devices may be proved by and may comprise a portion of various different types of electronic devices.
  • Figure 2 illustrates a block diagram of a mobile terminal 10 that may embody an example embodiment of the present invention.
  • the mobile terminal 10 as illustrated and hereinafter described is merely illustrative of one type of device that may benefit from an example embodiment of the present invention and numerous types of mobile terminals, such as portable digital assistants (PDAs), mobile telephones, pagers, mobile televisions, gaming devices, laptop computers, cameras, video recorders, audio/video players, radios, positioning devices, e.g., global positioning system (GPS) devices, or any combination of the aforementioned, and other types of voice and text communications systems, may readily employ an example embodiment of the present invention.
  • PDAs portable digital assistants
  • mobile telephones pagers
  • mobile televisions gaming devices
  • laptop computers cameras
  • video recorders audio/video players
  • radios radios
  • positioning devices e.g., global positioning system (GPS) devices, or any combination of the aforementioned, and other types of voice and text communications systems
  • the mobile terminal 10 may include an antenna 12, or multiple antennas, in operable communication with a transmitter 14 and a receiver 16.
  • the mobile terminal 10 may further include an apparatus, such as a processor 20, which provides signals to and receives signals from the transmitter 14 and receiver 16, respectively.
  • the signals include signaling information in accordance with the air interface standard of the applicable cellular system, and also user speech, received data and/or user generated data,
  • the mobile terminal 10 is capable of operating with one or more air interface standards, communication protocols, modulation types, and access types.
  • the mobile terminal 10 is capable of operating in accordance with any of a number of first, second, third and/or fourth-generation communication protocols or the like.
  • the mobile terminal 10 may be capable of operating in accordance with second-generation (2G) wireless communication protocols IS-136, time division multiple access (TDMA), global system for mobile communication (GSM), and IS-95 code division multiple access (CDMA), or with third-generation (3G) wireless communication protocols, such as Universal Mobile Telecommunications System
  • 2G second-generation wireless communication protocols
  • TDMA time division multiple access
  • GSM global system for mobile communication
  • CDMA IS-95 code division multiple access
  • 3G wireless communication protocols such as Universal Mobile Telecommunications System
  • the mobile terminal 10 may be capable of operating in accordance with non-cellular communication mechanisms.
  • the mobile terminal 10 may be capable of communication in a wireless local area network (WLAN) or other communication networks.
  • WLAN wireless local area network
  • the processor 20 may include circuitry desirable for implementing audio and logic functions of the mobile terminal 10.
  • the processor 20 comprises one or more digital signal processors and/or one or more microprocessors.
  • the processor may further comprise one or more analog to digital converters, one or more digital to analog converters and/or other support circuits. Control and signal processing functions of the mobile terminal 10 are allocated between these devices according to their respective capabilities.
  • the processor 20 thus may also include the functionality to convolutionaily encode and interleave message and data prior to modulation and transmission.
  • the processor 20 may additionally include an internal voice coder, and may include an internal data modem. Further, the processor 20 may include functionality to operate one or more software programs, which may be stored in memory.
  • the processor 20 may be capable of operating a connectivity program, such as a conventional Web browser.
  • the connectivity program may then allow the mobile terminal 10 to transmit and receive Web content, such as location-based content and/or other web page content, according to a Wireless Application Protocol (WAP), Hypertext Transfer Protocol (HTTP) and/or the like, for example.
  • WAP Wireless Application Protocol
  • HTTP Hypertext Transfer Protocol
  • the mobile terminal 10 may also comprise a user interface including an output device such as a conventional earphone or speaker 24, a ringer 22, a microphone 26, a display 28, and an input device, such as a user input interface, all of which are coupled to the processor 20.
  • the user input interface which allows the mobile terminal 10 to receive data, may include any of a number of devices allowing the mobile terminal 10 to receive data, such as a keypad 30, a touch screen display, such as represented by display 28, or other input device.
  • the keypad may include the conventional numeric (0-9) and related keys (#, *), and other hard and soft keys used for operating the mobile terminal 10.
  • the keypad 30 may include a conventional QWERTY keypad arrangement.
  • the keypad 30 may also include various soft keys with associated functions.
  • the mobile terminal 10 may include an interface device such as a joystick or other user input interface.
  • the mobile terminal 10 may further include a battery 34, such as a vibrating battery pack, for powering various circuits that are required to operate the mobile terminal 10, as well as optionally providing mechanical vibration as a detectable output.
  • the user input interface may include a touch screen display 28, which may be embodied as any known touch screen display.
  • the touch screen display 28 may be configured to enable touch recognition by any suitable technique, such as resistive, capacitive, infrared, strain gauge, surface wave, optical imaging, dispersive signal technology, acoustic pulse recognition, etc. techniques.
  • the touch screen display 28 may be configured to receive indications of user inputs and to provide a representation of the user inputs to the processor 20.
  • the mobile terminal 10 may further include a user identity module (UIM) 38.
  • the UIM 38 is typically a memory device having a processor built in.
  • the UIM 38 may include, for example, a subscriber identity module (SIM), a universal integrated circuit card (UICC), a universal subscriber identity module (USIM), a removable user identity module (R-UIM), etc.
  • SIM subscriber identity module
  • UICC universal integrated circuit card
  • USIM universal subscriber identity module
  • R-UIM removable user identity module
  • the UIM 38 typically stores information elements related to a mobile subscriber.
  • the mobile terminal 10 may be equipped with memory.
  • the mobile terminal 10 may include volatile memory 40, such as volatile Random Access Memory (RAM) including a cache area for the temporary storage of data.
  • RAM volatile Random Access Memory
  • the mobile terminal 10 may also include other non-volatile memory 42, which may be embedded and/or may be removable.
  • the memories may store any of a number of pieces of information, and data, used by the mobile terminal 10 to implement the functions of the mobile terminal 10.
  • the memories may include an identifier, such as an international mobile equipment identification (IMEI) code, capable of uniquely identifying the mobile terminal 10.
  • IMEI international mobile equipment identification
  • the memories may store instructions for determining cell id information.
  • the memories may store an application program for execution by the processor 20, which determines an identity of the current cell, e.g., cell id identity or cell id information, with which the mobile terminal 10 is in communication.
  • the overlapped handwriting input may be analyzed in accordance with an example embodiment as shown in Figure 3.
  • overlapped handwriting may be received via a touch screen 110, such as the touch screen display 28 of Figure 2 or the like.
  • Each stroke of the handwritten input may then be subjected to feature extraction 1 12 in which one or more features associated with the stroke are determined as described below.
  • a determination may then be made, such as by a classifier 1 14, as to whether the current stroke is a part of the same stroke group as the preceding stroke or whether the current stroke is a part of another stroke group.
  • a common statistical classifier such as a support vector machine or an artificial neural network, may be employed.
  • the classifier 1 14 analyzes the current stroke by utilization of segmentation rules 1 16.
  • the segmentation rules 1 16 may be determined in various manners. In the illustrated embodiment, however, a handwriting database 1 18 is provided that includes a plurality of different series of handwritten strokes. Each series of strokes of the handwriting database 1 18 may then be subjected to feature extraction 120 in which each series of strokes of the handwriting database is analyzed in order to identify the features associated with the series of strokes and the individual strokes that comprise the series. In one embodiment, the features that have been extracted and are associated with the current strokes of the handwriting database 1 18 are subjected to training 122 in order to refine the segmentation rules.
  • the handwriting database 1 18 may contain many samples of different words and/or sentences written by different people.
  • the words and/or sentences may be separated into characters manually. Based upon this separation into characters, each stroke may be classified as either the last stroke of a character or a stroke other than the last stroke of a character.
  • the training 122 may therefore learn from the strokes in the handwriting database 1 18 in order to establish segmentation rules that define the parameters to be considered in determining whether a stroke is or is not the last stroke of a character.
  • a stroke may be represented by a feature vector based upon various parameters of the stroke with the value of the feature vector then being determined by the classifier 1 14 based upon the segmentation rules 1 16.
  • the classifier may then compare the value of the feature vector representative of the stroke to a predefined threshold to determine if the stroke is or is not the last stroke of a character.
  • the training 122 may serve to determine the set of parameters that provides the greatest rate of success in terms of properly classifying strokes for all of the various strokes in the handwriting database 1 18.
  • the segmentation rules 1 16 are defined which, in turn, defines the set of parameters that may be utilized by the classifier 1 14 to classify strokes based upon the different series of strokes stored by the handwriting database 1 18, such as in terms of the features associated with each series of strokes and/or the individual strokes that comprise each series.
  • the classifier 1 14 may therefore determine that the current stroke is a portion of the same stroke group as the preceding stroke or that the current stroke begins a different stroke group.
  • the predefined threshold may be adjusted in one embodiment, such as by increasing the threshold, in order to reduce the likelihood that the segmented group will include strokes from more than one character.
  • those operations above the dashed line comprise a training phase 126. As such, these operations may be performed in advance and need not be repeated as overlapped handwritten input is received by the touch screen 1 10. However, the operations below the dashed line comprise an implementation phase 124 and are performed as or following the receipt of overlapped handwritten input, such as following the entry of one or multiple strokes upon the touch screen 1 10.
  • FIG. 4 illustrates the operations performed by an apparatus in accordance with an example embodiment to the present invention.
  • the apparatus may be employed, for example, on the mobile terminal 10.
  • the apparatus may alternatively be embodied at a variety of other devices, both mobile and fixed, such as, for example, any of the devices listed above.
  • the apparatus may include means, such as the processor 20, a user input interface, e.g., touch screen display 28, or the like, for receiving a series of strokes. See operation 130 of Figure 4.
  • the plurality of strokes that are received comprise overlapped handwriting in which a plurality of characters are continuously written one upon the other. For each stroke, a plurality of features may be determined. See operation 132.
  • an apparatus may include means, such as the processor 20, for determining the plurality of features associated with each stroke.
  • Various features may be determined in accordance with example embodiments of the present invention.
  • the plurality of features that are determined for a current stroke include features associated with the current stroke as well as features associated with the previous and the following stroke.
  • Figure 5 illustrates a plurality of overlapped handwritten characters including a current stroke 200, the previous stroke 210 and the next stroke 220.
  • the features that are determined for a current stroke may include the end point of the current stroke, the geometric center of the current stroke, the beginning point of the next stroke, the geometric center of the next stroke, the smallest rectangle containing the current stroke, the smallest rectangle containing the next stroke, and the smallest rectangle containing the prior stroke.
  • the feature is generally defined in terms of a pair of coordinates, such as x, y coordinates.
  • each shape, such as each rectangle may be defined in terms of four features, such as the coordinates of the left and right side of the shape and the coordinates of the top and bottom of the shape.
  • the features that are extracted from the current stroke in accordance with one example embodiment may include the x, y coordinates of the end point of the current stroke (endX, endY) designated 204, the geometric center of the current stroke (currentGCX, currentGCY) designated 206, the beginning point of the next stroke (startX, startY) designated 222, the geometric center of the next stroke (nextGCX, nextGCY) designated 226 and the smallest rectangles containing the current stroke, the next stroke, and the prior stroke.
  • each rectangle may be defined by four features, namely, the coordinates associated with the left, right, top and bottom.
  • the left, right, top and bottom coordinates for the smallest rectangles associated with the current stroke, the next stroke and the previous stroke are designated by the prefixes current, next and previous, respectively, in Figure 5.
  • the features associated with the smallest rectangle for the current stroke are designated in Figure 5 as currentLeft, currentRight, currentTop and currentBottom.
  • Other points that are designated in Figure 5 for purposes of orientation, even though these other points are not extracted as features in this example embodiment, include the beginning point of the
  • the geometric center of a stroke may be defined in various manners, the geometric center of a stroke in accordance with one example embodiment is defined as the average point for all points in a stroke.
  • a stroke containing end points (xj, y , i 0...n-l , has a geometric center (GCX, GCY) defined as follows:
  • the plurality of overlapped characters and the corresponding strokes that comprise the characters may have different sizes in that the characters may be written smaller or larger while still being intended to convey the same meaning.
  • the apparatus may include means, such as the processor 20, for normalizing the features that are extracted from the current strokes to take into account, such as by removing, any effect of differences in size of the same strokes. See operation 134 of Figure 4.
  • the features are normalized based upon an overall dimension of a series of strokes.
  • the smallest rectangle containing each of the handwritten strokes may be defined as shown, for example, in Figure 5.
  • the smallest rectangle that contains all of the strokes may be defined by the coordinates associated with the top, bottom, left and right of the overlapped handwritten characters.
  • the smallest rectangle that contains all of the series of strokes is positioned so as to have one corner at the origin ; of the coordinate system and to have sides that extend along the coordinate axes
  • the smallest rectangle that contains all of the strokes may be represented, in simplified form by its width, e.g., totalWidth, and its height, e.g., totalHeight.
  • CurrentStrokeEndX endX/totalWidth*2 -1
  • CurrentGCX currentGCX/totalWidth*2 -1
  • CurrentStrokeEndY endY/totalHeight !
  • CurrentGCY currentGCY/totalHeight*2 -1
  • NextStrokeStartX startX/totalWidth*2 -
  • NextGCX nextGCX/totalWidth*2 -1 ;
  • NextStrokeStartY start Y/totalWidth*2 -1 ;
  • NextGCY nextGCY/totalHeight*2 -1;
  • PreLeft previousLefi/totalWidth*2-l
  • Current ight currentRight/tota]Width*2-l ;
  • PreRight previousRight totalWidth*2-l
  • CurrentTop currentTop/totaIHeight*2-l ;
  • PreTop previousTop/totalHeight*2-l ;
  • CurrentBottom currentBottom/totalHeight*2- 1 ;
  • NextStrokeStartY are the normalized x and y coordinates for the beginning point of the next stroke.
  • CurrentGCX and CurrentGCY are the normalized x and y coordinates for the geometric center of the current stroke.
  • NextGCX and NextGCY are the normalized x and y coordinates for the geometric center of the next stroke.
  • the plurality, of features that are extracted from a current stroke may then be combined, such as by being combined by the processor 20 into a feature vector.
  • the features associated with the prior stroke may be set to a predefined value, such as -1.
  • the series of strokes may be segmented into one or more groups of strokes based upon features associated with the strokes. See operation 136 of Figure 4.
  • the apparatus may include means, such as the processor 20, for segmenting the series of strokes into one or more groups of strokes based upon the features.
  • the strokes may be segmented into one or more groups of strokes by a classifier 114 and in accordance with a technique as described in more detail below.
  • the apparatus may also include means, such as the processor 20, for segmenting the series of strokes into one or more characters based upon the one or more groups of strokes into which the series of strokes has been segmented. See operation 138.
  • the segmentation into characters based upon the groups may be performed in various manners including by utilization of segmentation rules 1 16 or the like which define the parameters to be utilized by the classifier 1 14 in order to properly evaluate the feature vector of a stroke.
  • the processor 20 of one example embodiment may include or otherwise implement the feature extraction 112, the classifier 114 and the segmentation rules 116, as described above in conjunction with Figure 3.
  • the segmentation of the strokes into characters may be performed in a more computationally efficient manner than in instances in which a plurality of overlapped characters are segmented without an intermediate grouping of the strokes.
  • a plurality of overlapping characters are shown at 230.
  • a plurality of groups may be defined as shown at 232.
  • the series of strokes may be segmented into a plurality of characters based upon the groups as shown at 234.
  • the resulting characters may then be recognized, such as by pattern recognition or the like, as shown at 236, such that the plurality of overlapped handwritten characters may be efficiently and effectively segmented and recognized as an individual series of characters.
  • the segmentation of the strokes into characters based upon the groups identifies a plurality of possible combinations of the groups, such as all possible combinations of the groups, as shown at 234 and 236.
  • the characters represented by the different combinations of the groups may be recognized by a handwriting recognition engine that may be embodied by the processor 20 or by another computing device that is in communication with the processor.
  • the handwriting recognition engine may also determine the similarity between each combination of the groups and the current set of characters.
  • Each set of characters that is represented by a combination of the groups may then be analyzed, such as by a language model embodied by the processor 20 or by another computing device that is in communication with the processor, in order to determine how meaningful each set of characters is.
  • the apparatus may then assign a score to each possible set of characters that is based upon a combination of the measure of similarity that has been determined by the handwriting recognition engine, the measure of how meaningful the current set of characters is that has been determined by the language model and, in some embodiments, a measure of predefined geometric properties of groups.
  • each group may be analyzed, such as by the processor 20, to determine if the group satisfies one or more predefined geometric properties.
  • one geometric property may relate to the size of a group with a group that is less than a predefined threshold being considered too small to be a whole character.
  • a group that is located along an edge of the writing area, such as along the leftmost or rightmost portion of the touch screen display 28, may not be considered to be a whole character.
  • the set of characters that is determined, such as by the processor 20, to have the highest score based upon, for example, one or more of the similarity, meaningfulness and geometric properties, may be identified as the set of characters that best represents the plurality of overlapping characters, as shown at 238.
  • one or more restrictions may be placed upon the process of identifying the plurality of possible combinations of the groups in order to increase the efficiency with which the combinations are identified and thereafter processed.
  • the possible combinations may be restricted by limiting any potential combination or character to a maximum of four groups.
  • the apparatus may also include, means, such as the processor 20, the display 28 or the like, for causing a display of at least some of the groups such that at least one group is caused to be displayed in a visibly distinct manner from at least one other group. See operation 140 of Figure 4. For example, only a subset of the groups may be displayed, such that at least one group is not caused to be displayed. In this regard, a relatively small number of the most recent group may be displayed with all preceding groups removed from the display 28.
  • the resulting display 28 is less cluttered and the user may more readily view the stroke that is currently being inputted as well as the immediately preceding strokes.
  • the groups of strokes may be displayed in such a manner that a visible distinction is drawn between different groups of strokes.
  • the groups of strokes may be displayed with different colors and/or brightness levels, such as different colors or brightness levels that vary based upon the sequence in which the strokes were received.
  • the most recent stroke group may be displayed with the darkest color (and/or brightness)
  • the immediately preceding stroke group may be displayed with a slightly lighter color (and/or brightness) and so on until the initial stroke group that is displayed with the lightest color (and/or brightness).
  • the different stroke groups may be represented by different types of lines, such as the most recent stroke group being represented by a solid line, the immediately preceding stroke group being represented by a dotted and dashed line and so on.
  • the stroke groups are displayed such that the different stroke groups are visibly distinct and, in some embodiments, the more recent stroke groups are more readily apparent.
  • the determination of the features associated with a stroke and the segmentation of the stroke into a current group may be performed following the input of each stroke as shown, for example, in Figure 7.
  • the current stroke and the next stroke are designated as stroke k-1 and stroke k, respectively.
  • a determination is made as to whether the stroke is the initial stroke, that is, a determination is made as to whether k 0.
  • the real writing area may be initialized. See operation 256.
  • the real writing area may be initialized to be the smallest rectangle that encompasses the initial stroke and may defined in terms of a total width and a total height for a rectangle oriented and positioned relative to a coordinate system as shown in Figure 5 or in terms of the top, bottom, left and right coordinates for rectangles positioned in other orientations and positions. Thereafter, the counter k may be incremented prior to awaiting the input of the next stroke. See operation 258. Following entry of the next stroke, the real writing area may be recalculated such that the recalculated real writing area represents the smallest rectangle that encompasses each stroke. See operation 260. Thereafter, the features for the current stroke k -1 may be determined. See operation 262.
  • a classifier 1 14 may provide a value S pre based upon the features of stroke k-1.
  • the classifier 1 14 may provide a value S pre based a feature vector of stroke k-1 and the segmentation rules 1 16, which define the parameters to be utilized by the classifier to evaluate the feature vector of a stroke. See operation 264 of Figure 7.
  • the S pre value provided by the classifier 1 14 may then be compared to a predefined threshold for classification T pre . See operation 266. If the value S pre provided by the classifier 1 14 exceeds the threshold T pre for classification, stroke k-1 and stroke k are considered to be in different groups. See operation 268.
  • stroke k -1 and stroke k are considered to be in the same group. See operation 270.
  • This process as described above and as shown in Figure 7 may be repeated incrementally for each stroke that is input in order to appropriately segment this series of strokes into groups.
  • a plurality of strokes may be analyzed in a batch process as shown, for example, in Figure 8.
  • the real writing area such as the smallest rectangle that encompasses a plurality of strokes 0, 1, ... M-l, may be determined. See operation 280.
  • the features for stroke k -1 are determined and the value S pre for stroke k -1 may then be determined, such as by a classifier 1 14, based upon the features for stroke k -1. See operations 286 and 288.
  • a comparison of the value S pre to the threshold T pre for classification may performed with stroke k -1 and stroke k being placed in different groups or in the same group depending on whether the value S pre is greater than the threshold T pre for classification or not, respectfully.
  • This process may be repeated for each individual stroke of the batch as indicated by the incrementing of the counter k in operation 296 until each stroke has been considered and appropriately sorted into groups.
  • FIGS 4, 7 and 8 are flowcharts of a method and program product according to example embodiments of the invention. It will be understood that each block of the flowcharts, and combinations of blocks in the flowcharts, may be implemented by various means, such as hardware, firmware, processor, circuitry and/or other devices associated with execution of software including one or more computer program instructions. For example, one or more of the procedures described above may be embodied by computer program instructions. In this regard, the computer program instructions which embody the procedures described above may be stored by a memory device of the mobile terminal 10 and executed by a processor 20 in the mobile terminal.
  • any such computer program instructions may be loaded onto a computer or other programmable apparatus, e.g., hardware, to produce a machine, such that the instructions which execute on the computer or other programmable apparatus create means for implementing the functions specified in the flowcharts block(s).
  • These computer program instructions may also be stored in a computer-readable memory that may direct a computer or other programmable apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowcharts block(s).
  • the computer program instructions may also be loaded onto a computer or other programmable apparatus to cause a series of operations to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus implement the functions specified in the flowcharts block(s).
  • blocks of the flowcharts support combinations of means for performing the specified functions, combinations of operations for performing the specified functions and program instructions for performing the specified functions. It will also be understood that one or more blocks of the flowcharts, and combinations of blocks in the flowcharts, can be implemented by special purpose hardware-based computer systems which perform the specified functions, or combinations of special purpose hardware and computer instructions.

Abstract

A method, apparatus and computer program product are provided in order to determine a plurality of features associated with the current stroke based upon geometrical properties of the series of strokes and to then segment a series of strokes that comprise overlapped handwriting into one or more groups of strokes based on the features associated with the strokes. This segmentation may be performed prior to any segmentation into one or more characters and, indeed, each group of strokes may be either a character or a portion of a character, but, in any event, no group of strokes includes strokes from more than one character.

Description

METHOD AND APPARATUS FOR SEGMENTING STROKES OF OVERLAPPED HANDWRITING INTO ONE OR MORE GROUPS
TECH OLOGIICAL FIELD
[0001] An example embodiment of the present invention relates to processing of overlapped handwriting and, more particularly, to segmenting a series of strokes that comprise the overlapped handwriting into one or more groups of strokes.
BACKGROUND
[0002] A variety of devices capture or otherwise receive handwriting input via a touch screen or other input device. For example, a number of computers, tablets, mobile telephones, personal digital assistants (PDAs) and other types of electronic devices include touch screens which permit a user to provide tactile input, such as in the form of handwriting. The handwritten input may be captured by the electronic device and processed in an effort to recognize the handwritten characters in order to inform future actions of the electronic device including, for example, storage or transmission of a representation of the handwritten input.
[0003] In order to facilitate character recognition, handwritten input is generally processed so as to segment the handwritten input into different characters, such as letters, numerals or other characters. There are several different techniques for segmenting handwritten input. One technique makes use of a time out between the input of each character. In accordance with this technique, the user interface, such as a touch screen, need only include a single writing area. A user may write a single character within the writing area. A time out is then taken in order to receive the character and to clear the writing area in order to prepare the writing area for receipt of the next character. Thus, the time out taken between entry of each character provides for segmentation of the handwriting.
[0004] Another technique employs a user interface having two or more writing areas, such as two or more touch screens. A user can write a single character in one of the writing areas and may then switch to the other writing area in order to write the next character. While a user is writing a character in one of the writing areas, the character that was previously written in the other writing area may be received and processed and the other writing area may then be cleared so as to prepare the other writing area for receipt of the next character. Thus, the alteration between the different writing areas provides for segmentation of the handwriting.
[0005] A further technique employs a single writing area that is relatively large. Within a writing area, a user can write several characters on a continuous basis one after another in a comparable fashion to the manner in which characters are written on a piece of paper. This technique utilizes the spatial intervals that are positioned between the written characters as well as underlying intelligence in order to appropriately segment the handwritten input.
[0006] In yet another technique, the user interface provides a single writing area in which a user can write several characters on a continuous basis with the characters being overlapped with one another without a spatial interval between the characters. Utilizing underlying intelligence, the overlapped characters may be segmented so as to separate one character from the others. However, the procedure is much more complicated than the above techniques with a recognition process usually being employed. Under the maximum likelihood estimation, the recognition process can segment a series of strokes into characters with errors. The recognition process is very computing intensive, and is very challenging for real-time operation, especially for small devices.
[0007] Techniques that utilize overlapped continuous handwriting permit electronic devices having relatively small user interfaces to receive a continuous handwritten input using the fingers of the user or other stylus such that the handwritten input may be provided in a relatively rapid fashion in comparison to the input of a single character at a time. However, the reliance upon overlapped continuous handwriting may pose challenges regarding segmentation and user interaction. In this regard, the overlapped continuous handwriting may increase the difficulties associated with efficiently segmenting the overlapped handwriting in order to separate each character from the others, particularly in comparison to more common continuous handwriting having spatial intervals between the written characters. Additionally, the overlapping of multiple characters may create difficulties for the user in that the user may not be able to clearly view what the user is currently writing as a result of the noisy and complicated background created by the other overlapped characters.
BRIEF SUMMARY
[0008] A method, apparatus and computer program product are provided in accordance with an example embodiment in order to segment a series of strokes that comprise overlapped handwriting into one or more groups of strokes. This segmentation may be performed prior to any segmentation into one or more characters and, indeed, each group of strokes may be either a character or a portion of a character, but, in any event, no group of strokes includes strokes from more than one character. By segmenting the series of strokes into one or more groups of strokes, the subsequent segmentation of the series of strokes into one or more characters is made more efficient. Additionally, one or more prior groups of strokes may be displayed in a less visibly distinct manner, while continuing to receive the overlapped handwriting in order to permit the user to more clearly view the most recent strokes.
[0009] In one embodiment, a method is provided that receives a series of strokes that comprise overlapped handwriting and, for each of the plurality of strokes, determines, with a processor, a plurality of features associated with the current stroke based upon geometrical properties of the series of strokes. The method of this embodiment also segments a series of strokes into one or more groups of strokes based upon the features associated with the strokes. As noted above, each group of strokes is one of a character or a portion of a character, but no group of strokes includes strokes from more than one character.
[0010] The method of one embodiment may also segment the series of strokes into one or more characters based upon the one or more groups of strokes into which the series of strokes has been segmented, thereby increasing the efficiency with which a series of strokes may be segmented into characters. In accordance with an example embodiment, the method may determine a plurality of features associated with the current stroke by determining the plurality of features associated with the current stroke based only on the geometrical properties of the series of strokes. In one embodiment, the method may also cause a display of at least some of the groups such that at least one group is caused to be displayed in a visibly distinct manner from at least one other group. Thus, the displayed image of the overlapped handwriting may be simplified to facilitate the user's view of, for example, the most recent strokes.
[0011] In one embodiment, the method also normalizes the plurality of features associated with the current stroke. In this regard, the plurality of features may be normalized based upon an overall dimension of a series of strokes. In determining the plurality of features associated with the current stroke, the plurality of features may be selected from the group consisting of an endpoint of the current stroke, a geometric center of the current stroke, a beginning point of the next stroke, a geometric center of the next stroke, a smallest rectangle containing the current stroke, a smallest rectangle containing the next stroke and a smallest rectangle containing an immediately prior stroke.
[0012] The method may be performed in an incremental mode or a batch mode. In an incremental mode, a determination of the plurality of features and the segmentation of the series of strokes are repeated following receipt of each successive stroke. In the batch mode, the determination of the plurality of features and the segmentation of the series of strokes are repeated following receipt of a plurality of strokes.
[0013] In another embodiment, an apparatus is provided that includes at least one processor and at least one memory including computer program code. The at least one memory and computer program code of the apparatus of this embodiment are configured, with the at least one processor, to cause the apparatus to at least receive a series of strokes that comprise overlapped handwriting and, for each of the plurality of strokes, determine a plurality of features associated with the current stroke based upon geometrical properties of the series of strokes. The at least one memory and computer program code of the apparatus of this embodiment are also configured, with the at least one processor, to cause the apparatus to segment a series of strokes into one or more groups of strokes based upon the features associated with the strokes. As noted above, each group of strokes is one of a character or a portion of a character, but no group of strokes includes strokes from more than one character.
[0014] The at least one memory and computer program code of the apparatus of one embodiment may also be configured, with the at least one processor, to cause the apparatus to segment the series of strokes into one or more characters based upon the one or more groups of strokes into which the series of strokes has been segmented, thereby increasing the efficiency with which a series of strokes may be segmented into characters. The at least one memory and computer program code of the apparatus of one example embodiment may also be configured, with the at least one processor, to cause the apparatus to determine a plurality of features associated with the current stroke by determining the plurality of features associated with the current stroke based only on the geometrical properties of the series of strokes. In one embodiment, the at least one memory and computer program code of the apparatus may also be configured, with the at least one processor, to cause the apparatus to cause a display of at least some of the groups such that at least one group is caused to be displayed in a visibly distinct manner from at least one other group. Thus, the displayed image of the overlapped handwriting may be simplified to facilitate the user's view of, for example, the most recent strokes.
[0015] In one embodiment, the at least one memory and computer program code of the apparatus may also be configured, with the at least one processor, to cause the apparatus to normalize the plurality of features associated with the current stroke. In this regard, the plurality of features may be normalized based upon an overall dimension of a series of strokes. In determining the plurality of features associated with the current stroke, the plurality of features may be selected from the group consisting of an endpoint of the current stroke, a geometric center of the current stroke, a beginning point of the next stroke, a geometric center of the next stroke, a smallest rectangle containing the current stroke, a smallest rectangle containing the next stroke and a smallest rectangle containing an immediately prior stroke.
[0016] The analysis of the strokes may be performed in an incremental mode or a batch mode. In an incremental mode, a determination of the plurality of features and the segmentation of the series of strokes are repeated following receipt of each successive stroke. In the batch mode, the determination of the plurality of features and the segmentation of the series of strokes are repeated following receipt of a plurality of strokes.
[0017] In a further embodiment, an apparatus is provided that includes means for receiving a series of strokes that comprise overlapped handwriting and, for each of the plurality of strokes, determining a plurality of features associated with the current stroke based upon geometrical properties of the series of strokes. The apparatus of this embodiment also includes means for segmenting a series of strokes into one or more groups of strokes based upon the features associated with the strokes. As noted above, each group of strokes is one of a character or a portion of a character, but no group of strokes includes strokes from more than one character.
[0018] The apparatus of one embodiment may also include means for segmenting the series of strokes into one or more characters based upon the one or more groups of strokes into which the series of strokes has been segmented, thereby increasing the efficiency with which a series of strokes may be segmented into characters. In one example embodiment, the means for determining a plurality of features associated with the current stroke includes means for determining the plurality of features associated with the current stroke based only on the geometrical properties of the series of strokes. In one embodiment, the apparatus may also include means for causing a display of at least some of the groups such that at least one group is caused to be displayed in a visibly distinct manner from at least one other group. Thus, the displayed image of the overlapped handwriting may be simplified to facilitate the user's view of, for example, the most recent strokes.
[0019] In one embodiment, the apparatus also includes means for normalizing the plurality of features associated with the current stroke. In this regard, the plurality of features may be normalized based upon an overall dimension of a series of strokes. In determining the plurality of features associated with the current stroke, the plurality of features may be selected from the group consisting of an endpoint of the current stroke, a geometric center of the current stroke, a beginning point of the next stroke, a geometric center of the next stroke, a smallest rectangle containing the current stroke, a smallest rectangle containing the next stroke and a smallest rectangle containing an immediately prior stroke.
[0020] The apparatus may analyze the strokes in either an incremental mode or a batch mode. In an incremental mode, a determination of the plurality of features and the segmentation of the series of strokes are repeated following receipt of each successive stroke. In the batch mode, the determination of the plurality of features and the segmentation of the series of strokes are repeated following receipt of a plurality of strokes. [0021] In yet another embodiment, a computer program product is provided that includes at least one computer-readable memory having computer-executable code portions stored therein. The computer-executable code portions include program code instructions for receiving a series of strokes that comprise overlapped handwriting and program code instructions for determining, for each of the plurality of strokes, a plurality of features associated with the current stroke based upon geometrical properties of the series of strokes. The computer-executable code portions of this embodiment also include program code instructions for segmenting a series of strokes into one or more groups of strokes based upon the features associated with the strokes. As noted above, each group of strokes is one of a character or a portion of a character, but no group of strokes includes strokes from more than one character.
[0022] The computer-executable code portions of one embodiment may also include program code instructions for segmenting the series of strokes into one or more characters based upon the one or more groups of strokes into which the series of strokes has been segmented, thereby increasing the efficiency with which a series of strokes may be segmented into characters. The computer-executable code portions of one example embodiment may also include program code instructions for determining a plurality of features associated with the current stroke by determining the plurality of features associated with the current stroke based only on the geometrical properties of the series of strokes. In one embodiment, the computer-executable code portions may also include program code instructions for causing a display of at least some of the groups such that at least one group is caused to be displayed in a visibly distinct manner from at least one other group. Thus, the displayed image of the overlapped handwriting may be simplified to facilitate the user's view of, for example, the most recent strokes.
[0023] In one embodiment, the computer-executable code portions also include program code instructions for normalizing the plurality of features associated with the current stroke. In this regard, the plurality of features may be normalized based upon an overall dimension of a series of strokes. In determining the plurality of features associated with a current stroke, the plurality of features may be selected from the group consisting of an endpoint of the current stroke, a geometric center of the current stroke, a beginning point of the next stroke, a geometric center of the next stroke, a smallest rectangle containing the current stroke, a smallest rectangle containing the next stroke and a smallest rectangle containing an immediately prior stroke.
[0024] The computer program product may analyze the strokes in either an incremental mode or a batch mode. In an incremental mode, a determination of the plurality of features and the segmentation of the series of strokes are repeated following receipt of each successive stroke. In the batch mode, the determination of the plurality of features and the segmentation of the series of strokes are repeated following receipt of a plurality of strokes.
BRIEF DESCRIPTION OF THE SEVERAL OF THE DRAWINGS
[0025] Having thus described certain example embodiments of the present disclosure in general terms, reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and wherein:
[0026] Figure 1 is an illustration of overlapped handwriting in which four characters are overlapped;
[0027] Figure 2 is a block diagram of an apparatus in accordance with an example embodiment of the present invention;
[0028] Figure 3 is a functional block diagram of operations performed in accordance with an example embodiment of the present invention;
[0029] Figure 4 is a flow chart illustrating operations performed in accordance with an example embodiment of the present invention;
[0030] Figure 5 is an illustration of an overlapped handwriting in which features of three successive strokes are identified in accordance with an example embodiment of the present invention;
[0031] Figure 6 is an illustration of a manner in which the overlapped handwriting of Figure 1 may be processed in order to recognize the four characters in accordance with an example embodiment of the present invention;
[0032] Figure 7 is a flowchart illustrating operations performed in accordance with an incremental mode of an example embodiment of the present invention; and
[0033] Figure 8 is a flowchart illustrating the operations performed in accordance with a batch mode of another example embodiment of the present invention. DETAILED DESCRIPTION
[0034] Some embodiments of the present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the invention are shown. Indeed, various embodiments of the invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like reference numerals refer to like elements throughout. As used herein, the terms "data," "content," "information" and similar terms may be used interchangeably to refer to data capable of being transmitted, received and/or stored in accordance with embodiments of the present invention. Thus, use of any such terms should not be taken to limit the spirit and scope of embodiments of the present invention.
[0035] Additionally, as used herein, the term 'circuitry' refers to (a) hardware-only circuit implementations (e.g., implementations in analog circuitry and/or digital circuitry); (b) combinations of circuits and computer program product(s) comprising software and/or firmware instructions stored on one or more computer readable memories that work together to cause an apparatus to perform one or more functions described herein; and (c) circuits, such as, for example, a microprocessor(s) or a portion of a
microprocessor(s), that require software or firmware for operation even if the software or firmware is not physically present. This definition of 'circuitry' applies to all uses of this term herein, including in any claims. As a further example, as used herein, the term 'circuitry' also includes an implementation comprising one or more processors and/or portion(s) thereof and accompanying software and/or firmware. As another example, the term 'circuitry' as used herein also includes, for example, a baseband integrated circuit or applications processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network device, other network device, and/or other computing device.
[0036] As defined herein a "computer-readable storage medium," which refers to a non- transitory, physical storage medium (e.g., volatile or non-volatile memory device), can be differentiated from a "computer-readable transmission medium," which refers to an electromagnetic signal. [0037] As shown in Figure I , handwriting may be input via a user interface, such as a touch screen, in an overlapped fashion in which characters are continuously overlapped without spatial or temporal separation between the characters. In this regard, the example of Figure 1 illustrates the sequential overlapping of the four characters that are individually set forth in the lower portion of the figure and designated 100. In this regard, display 102 illustrates the entry of the first character, display 104 illustrates the overlapped entry of the first and second characters, display 106 illustrates the overlapped entry of the first, second and third characters and display 108 illustrates the overlapped entry of all four characters.
10038] Overlapped handwriting allows a user to make full use of the user interface, which is particularly useful for users who utilize their fingers to provide handwriting input, such as on a relatively small and low resolution touch screen. Additionally, the use of overlapped handwriting provides a relatively natural and quick technique for receiving handwritten input. As will be recognized from reference to the increasingly crowded display of Figure 1 as additional characters are overlapped with prior characters, the reliance upon overlapped handwriting to provide handwritten input poses a challenge in regards to the segmentation of the different characters. Additionally, overlapped handwriting may also make it relatively difficult for a user to view the character that the user is currently writing or an immediately preceding character as a result of the concurrent display of a number of other overlapped characters. Accordingly, a technique for pre-segmenting overlapped handwriting is provided in accordance with an example embodiment of the present invention.
[0039] The overlapped handwriting that may be pre-segmented may be received via any of a wide variety of input devices, such as user interfaces, e.g., touch screens or the like. Not only may the overlapped handwriting be received via a wide variety of different input devices, but the input devices may be proved by and may comprise a portion of various different types of electronic devices. For example, Figure 2 illustrates a block diagram of a mobile terminal 10 that may embody an example embodiment of the present invention. It should be understood, however, that the mobile terminal 10 as illustrated and hereinafter described is merely illustrative of one type of device that may benefit from an example embodiment of the present invention and numerous types of mobile terminals, such as portable digital assistants (PDAs), mobile telephones, pagers, mobile televisions, gaming devices, laptop computers, cameras, video recorders, audio/video players, radios, positioning devices, e.g., global positioning system (GPS) devices, or any combination of the aforementioned, and other types of voice and text communications systems, may readily employ an example embodiment of the present invention.
[0040] The mobile terminal 10 may include an antenna 12, or multiple antennas, in operable communication with a transmitter 14 and a receiver 16. The mobile terminal 10 may further include an apparatus, such as a processor 20, which provides signals to and receives signals from the transmitter 14 and receiver 16, respectively. The signals include signaling information in accordance with the air interface standard of the applicable cellular system, and also user speech, received data and/or user generated data, In this regard, the mobile terminal 10 is capable of operating with one or more air interface standards, communication protocols, modulation types, and access types. By way of illustration, the mobile terminal 10 is capable of operating in accordance with any of a number of first, second, third and/or fourth-generation communication protocols or the like. For example, the mobile terminal 10 may be capable of operating in accordance with second-generation (2G) wireless communication protocols IS-136, time division multiple access (TDMA), global system for mobile communication (GSM), and IS-95 code division multiple access (CDMA), or with third-generation (3G) wireless communication protocols, such as Universal Mobile Telecommunications System
(UMTS), CDMA2000, wideband CDMA (WCDMA) and time division-synchronous CDMA (TD-SCDMA), with 3.9G wireless communication protocol such as evolved UMTS terrestrial radio access network (E-UTRAN), with fourth-generation (4G) wireless communication protocols or the like. As an alternative, or additionally, the mobile terminal 10 may be capable of operating in accordance with non-cellular communication mechanisms. For example, the mobile terminal 10 may be capable of communication in a wireless local area network (WLAN) or other communication networks.
[0041] In some embodiments, the processor 20 may include circuitry desirable for implementing audio and logic functions of the mobile terminal 10. For example, the processor 20 comprises one or more digital signal processors and/or one or more microprocessors. The processor may further comprise one or more analog to digital converters, one or more digital to analog converters and/or other support circuits. Control and signal processing functions of the mobile terminal 10 are allocated between these devices according to their respective capabilities. The processor 20 thus may also include the functionality to convolutionaily encode and interleave message and data prior to modulation and transmission. The processor 20 may additionally include an internal voice coder, and may include an internal data modem. Further, the processor 20 may include functionality to operate one or more software programs, which may be stored in memory. For example, the processor 20 may be capable of operating a connectivity program, such as a conventional Web browser. The connectivity program may then allow the mobile terminal 10 to transmit and receive Web content, such as location-based content and/or other web page content, according to a Wireless Application Protocol (WAP), Hypertext Transfer Protocol (HTTP) and/or the like, for example.
[0042] The mobile terminal 10 may also comprise a user interface including an output device such as a conventional earphone or speaker 24, a ringer 22, a microphone 26, a display 28, and an input device, such as a user input interface, all of which are coupled to the processor 20. The user input interface, which allows the mobile terminal 10 to receive data, may include any of a number of devices allowing the mobile terminal 10 to receive data, such as a keypad 30, a touch screen display, such as represented by display 28, or other input device. In embodiments including the keypad 30, the keypad may include the conventional numeric (0-9) and related keys (#, *), and other hard and soft keys used for operating the mobile terminal 10. Alternatively, the keypad 30 may include a conventional QWERTY keypad arrangement. The keypad 30 may also include various soft keys with associated functions. In addition, or alternatively, the mobile terminal 10 may include an interface device such as a joystick or other user input interface. The mobile terminal 10 may further include a battery 34, such as a vibrating battery pack, for powering various circuits that are required to operate the mobile terminal 10, as well as optionally providing mechanical vibration as a detectable output.
[0043] As noted above, the user input interface may include a touch screen display 28, which may be embodied as any known touch screen display. Thus, for example, the touch screen display 28 may be configured to enable touch recognition by any suitable technique, such as resistive, capacitive, infrared, strain gauge, surface wave, optical imaging, dispersive signal technology, acoustic pulse recognition, etc. techniques. The touch screen display 28 may be configured to receive indications of user inputs and to provide a representation of the user inputs to the processor 20.
[0044] The mobile terminal 10 may further include a user identity module (UIM) 38. The UIM 38 is typically a memory device having a processor built in. The UIM 38 may include, for example, a subscriber identity module (SIM), a universal integrated circuit card (UICC), a universal subscriber identity module (USIM), a removable user identity module (R-UIM), etc. The UIM 38 typically stores information elements related to a mobile subscriber. In addition to the UIM 38, the mobile terminal 10 may be equipped with memory. For example, the mobile terminal 10 may include volatile memory 40, such as volatile Random Access Memory (RAM) including a cache area for the temporary storage of data. The mobile terminal 10 may also include other non-volatile memory 42, which may be embedded and/or may be removable. The memories may store any of a number of pieces of information, and data, used by the mobile terminal 10 to implement the functions of the mobile terminal 10. For example, the memories may include an identifier, such as an international mobile equipment identification (IMEI) code, capable of uniquely identifying the mobile terminal 10. Furthermore, the memories may store instructions for determining cell id information. Specifically, the memories may store an application program for execution by the processor 20, which determines an identity of the current cell, e.g., cell id identity or cell id information, with which the mobile terminal 10 is in communication.
[0045] Regardless of the type of input device and the type of electronic device that includes the input device, the overlapped handwriting input may be analyzed in accordance with an example embodiment as shown in Figure 3. In this embodiment, overlapped handwriting may be received via a touch screen 110, such as the touch screen display 28 of Figure 2 or the like. Each stroke of the handwritten input may then be subjected to feature extraction 1 12 in which one or more features associated with the stroke are determined as described below. A determination may then be made, such as by a classifier 1 14, as to whether the current stroke is a part of the same stroke group as the preceding stroke or whether the current stroke is a part of another stroke group. Although various classifiers 1 14 may be employed for analyzing the features associated with the current stroke in order to determine the group to which the current stroke belongs, a common statistical classifier, such as a support vector machine or an artificial neural network, may be employed.
[0046] As shown in Figure 3, the classifier 1 14 analyzes the current stroke by utilization of segmentation rules 1 16. The segmentation rules 1 16 may be determined in various manners. In the illustrated embodiment, however, a handwriting database 1 18 is provided that includes a plurality of different series of handwritten strokes. Each series of strokes of the handwriting database 1 18 may then be subjected to feature extraction 120 in which each series of strokes of the handwriting database is analyzed in order to identify the features associated with the series of strokes and the individual strokes that comprise the series. In one embodiment, the features that have been extracted and are associated with the current strokes of the handwriting database 1 18 are subjected to training 122 in order to refine the segmentation rules. In this regard, the handwriting database 1 18 may contain many samples of different words and/or sentences written by different people. The words and/or sentences may be separated into characters manually. Based upon this separation into characters, each stroke may be classified as either the last stroke of a character or a stroke other than the last stroke of a character. The training 122 may therefore learn from the strokes in the handwriting database 1 18 in order to establish segmentation rules that define the parameters to be considered in determining whether a stroke is or is not the last stroke of a character. As described below, a stroke may be represented by a feature vector based upon various parameters of the stroke with the value of the feature vector then being determined by the classifier 1 14 based upon the segmentation rules 1 16. The classifier may then compare the value of the feature vector representative of the stroke to a predefined threshold to determine if the stroke is or is not the last stroke of a character. Thus, the training 122 may serve to determine the set of parameters that provides the greatest rate of success in terms of properly classifying strokes for all of the various strokes in the handwriting database 1 18. Based upon the training 122, the segmentation rules 1 16 are defined which, in turn, defines the set of parameters that may be utilized by the classifier 1 14 to classify strokes based upon the different series of strokes stored by the handwriting database 1 18, such as in terms of the features associated with each series of strokes and/or the individual strokes that comprise each series. By comparing the value of the feature vector based upon the features that have been extracted from the current stroke to a predefined threshold, the classifier 1 14 may therefore determine that the current stroke is a portion of the same stroke group as the preceding stroke or that the current stroke begins a different stroke group.
[0047] Since an objective is to segment the strokes into entire characters, errors may occur if the segmented group includes only a portion of a character or if the segmented group includes strokes from more than one character. As the method and apparatus of one example embodiment may more readily accommodate and correct for any errors relating to a segmented group including only a portion of a character since the segmented groups may be combined following pre-segmentation, the predefined threshold may be adjusted in one embodiment, such as by increasing the threshold, in order to reduce the likelihood that the segmented group will include strokes from more than one character.
[0048] In the example embodiment of Figure 3, those operations above the dashed line comprise a training phase 126. As such, these operations may be performed in advance and need not be repeated as overlapped handwritten input is received by the touch screen 1 10. However, the operations below the dashed line comprise an implementation phase 124 and are performed as or following the receipt of overlapped handwritten input, such as following the entry of one or multiple strokes upon the touch screen 1 10.
[0049] By way of further explanation, reference is now made to Figure 4, which illustrates the operations performed by an apparatus in accordance with an example embodiment to the present invention. The apparatus may be employed, for example, on the mobile terminal 10. However, the apparatus may alternatively be embodied at a variety of other devices, both mobile and fixed, such as, for example, any of the devices listed above. The apparatus may include means, such as the processor 20, a user input interface, e.g., touch screen display 28, or the like, for receiving a series of strokes. See operation 130 of Figure 4. As noted above, the plurality of strokes that are received comprise overlapped handwriting in which a plurality of characters are continuously written one upon the other. For each stroke, a plurality of features may be determined. See operation 132. In this regard, an apparatus may include means, such as the processor 20, for determining the plurality of features associated with each stroke. Various features may be determined in accordance with example embodiments of the present invention. In one embodiment, however, the plurality of features that are determined for a current stroke include features associated with the current stroke as well as features associated with the previous and the following stroke. In this regard, Figure 5 illustrates a plurality of overlapped handwritten characters including a current stroke 200, the previous stroke 210 and the next stroke 220.
[0050] By way of example, the features that are determined for a current stroke may include the end point of the current stroke, the geometric center of the current stroke, the beginning point of the next stroke, the geometric center of the next stroke, the smallest rectangle containing the current stroke, the smallest rectangle containing the next stroke, and the smallest rectangle containing the prior stroke. In regards to those features that define a particular point or location, the feature is generally defined in terms of a pair of coordinates, such as x, y coordinates. Similarly, for those features that are defined in terms of a rectangle or other two-dimensional shape, each shape, such as each rectangle, may be defined in terms of four features, such as the coordinates of the left and right side of the shape and the coordinates of the top and bottom of the shape. With reference to Figure 5, the features that are extracted from the current stroke in accordance with one example embodiment may include the x, y coordinates of the end point of the current stroke (endX, endY) designated 204, the geometric center of the current stroke (currentGCX, currentGCY) designated 206, the beginning point of the next stroke (startX, startY) designated 222, the geometric center of the next stroke (nextGCX, nextGCY) designated 226 and the smallest rectangles containing the current stroke, the next stroke, and the prior stroke. With reference to Figure 5, each rectangle may be defined by four features, namely, the coordinates associated with the left, right, top and bottom. In this regard, the left, right, top and bottom coordinates for the smallest rectangles associated with the current stroke, the next stroke and the previous stroke are designated by the prefixes current, next and previous, respectively, in Figure 5. By way of example, the features associated with the smallest rectangle for the current stroke are designated in Figure 5 as currentLeft, currentRight, currentTop and currentBottom. Other points that are designated in Figure 5 for purposes of orientation, even though these other points are not extracted as features in this example embodiment, include the beginning point of the
111 ' i nn 1 εηηπτ current stroke designated202, the end point for the next stroke 224 and the beginning and end points for the previous stroke designated 212 and 214, respectively.
[0051] Although the geometric center of a stroke may be defined in various manners, the geometric center of a stroke in accordance with one example embodiment is defined as the average point for all points in a stroke. By way of example, a stroke containing end points (xj, y , i=0...n-l , has a geometric center (GCX, GCY) defined as follows:
GCX = sum (xj),/n, GCY=suni (yi)/n; i=0 ... n-1
[0052] The plurality of overlapped characters and the corresponding strokes that comprise the characters may have different sizes in that the characters may be written smaller or larger while still being intended to convey the same meaning. As such, the apparatus may include means, such as the processor 20, for normalizing the features that are extracted from the current strokes to take into account, such as by removing, any effect of differences in size of the same strokes. See operation 134 of Figure 4. In one embodiment, the features are normalized based upon an overall dimension of a series of strokes. In this regard, the smallest rectangle containing each of the handwritten strokes may be defined as shown, for example, in Figure 5. In this regard, the smallest rectangle that contains all of the strokes may be defined by the coordinates associated with the top, bottom, left and right of the overlapped handwritten characters. In the example embodiment in which the smallest rectangle that contains all of the series of strokes is positioned so as to have one corner at the origin ;of the coordinate system and to have sides that extend along the coordinate axes, the smallest rectangle that contains all of the strokes may be represented, in simplified form by its width, e.g., totalWidth, and its height, e.g., totalHeight.
[0053] In one example, the features that have been described above may be normalized as shown below in which a prefix of Current, Next and Pre refer to features associated with the current stroke, the next stroke, and the previous stroke, respectively. CurrentStrokeEndX = endX/totalWidth*2 -1; CurrentGCX = currentGCX/totalWidth*2 -1; CurrentStrokeEndY = endY/totalHeight!|!2 -1 ; CurrentGCY = currentGCY/totalHeight*2 -1 ; NextStrokeStartX = startX/totalWidth*2 - ; NextGCX = nextGCX/totalWidth*2 -1 ;
NextStrokeStartY = start Y/totalWidth*2 -1 ; NextGCY = nextGCY/totalHeight*2 -1;
CurrentLeft = currentLeft/totalWidth*2-I
PreLeft = previousLefi/totalWidth*2-l Current ight = currentRight/tota]Width*2-l ;
PreRight = previousRight totalWidth*2-l ; CurrentTop = currentTop/totaIHeight*2-l ;
PreTop = previousTop/totalHeight*2-l ; CurrentBottom=currentBottom/totalHeight*2- 1 ;
PreBottom= previousBottom/totalHeight*2-l
NextLeft = nextLeft/totalWidth*2-l
NextRight = nextRight/totalWidth*2-l;
NextTop = nextTop/totalHeight*2-l;
NextBottom=nextBottom/totalHeight*2- 1 ;
[0054] As such, CurrentStrokeEndX and CurrentStrokeEndY are the normalized x and y coordinates for the end point of the current stroke. NextStrokeStartX and
NextStrokeStartY are the normalized x and y coordinates for the beginning point of the next stroke. CurrentGCX and CurrentGCY are the normalized x and y coordinates for the geometric center of the current stroke. NextGCX and NextGCY are the normalized x and y coordinates for the geometric center of the next stroke. CurrentLeft, CurrentRight,
CurrentTop and CurrentBottom are the coordinates for the left, right, top and bottom, respectively, of the current stroke. NextLeft, NextRight, NextTop and NextBottom are the coordinates for the left, right, top and bottom, respectively, of the next stroke. Finally, PreLeft, PreRight, PreTop and PreBottom are the coordinates for the left, right, top and bottom, respectively, of the previous stroke.
[0055] In one embodiment, the plurality, of features that are extracted from a current stroke may then be combined, such as by being combined by the processor 20 into a feature vector. In the instances in which the current stroke that is being analyzed is the initial stroke, there will be no prior stroke. As such, the features associated with the prior stroke may be set to a predefined value, such as -1.
i s [0056] Once a plurality of features have been determined for a current stroke and, in one embodiment, the plurality of features have been normalized, the series of strokes may be segmented into one or more groups of strokes based upon features associated with the strokes. See operation 136 of Figure 4. As such, the apparatus may include means, such as the processor 20, for segmenting the series of strokes into one or more groups of strokes based upon the features. As described above, the strokes may be segmented into one or more groups of strokes by a classifier 114 and in accordance with a technique as described in more detail below. The apparatus may also include means, such as the processor 20, for segmenting the series of strokes into one or more characters based upon the one or more groups of strokes into which the series of strokes has been segmented. See operation 138. The segmentation into characters based upon the groups may be performed in various manners including by utilization of segmentation rules 1 16 or the like which define the parameters to be utilized by the classifier 1 14 in order to properly evaluate the feature vector of a stroke. As such, the processor 20 of one example embodiment may include or otherwise implement the feature extraction 112, the classifier 114 and the segmentation rules 116, as described above in conjunction with Figure 3. By initially defining groups of strokes and then, in turn, defining characters based upon the groups, the segmentation of the strokes into characters may be performed in a more computationally efficient manner than in instances in which a plurality of overlapped characters are segmented without an intermediate grouping of the strokes.
[0057] With reference to Figure 6, a plurality of overlapping characters are shown at 230. By segmenting the series of strokes into groups of strokes, a plurality of groups may be defined as shown at 232. Thereafter, the series of strokes may be segmented into a plurality of characters based upon the groups as shown at 234. The resulting characters may then be recognized, such as by pattern recognition or the like, as shown at 236, such that the plurality of overlapped handwritten characters may be efficiently and effectively segmented and recognized as an individual series of characters. In the example embodiment of Figure 6, the segmentation of the strokes into characters based upon the groups identifies a plurality of possible combinations of the groups, such as all possible combinations of the groups, as shown at 234 and 236. In this regard, the characters represented by the different combinations of the groups may be recognized by a handwriting recognition engine that may be embodied by the processor 20 or by another computing device that is in communication with the processor. In one embodiment, the handwriting recognition engine may also determine the similarity between each combination of the groups and the current set of characters. Each set of characters that is represented by a combination of the groups may then be analyzed, such as by a language model embodied by the processor 20 or by another computing device that is in communication with the processor, in order to determine how meaningful each set of characters is. The apparatus, such as the processor 20, the handwriting recognition engine and/or the language model, may then assign a score to each possible set of characters that is based upon a combination of the measure of similarity that has been determined by the handwriting recognition engine, the measure of how meaningful the current set of characters is that has been determined by the language model and, in some embodiments, a measure of predefined geometric properties of groups. In this regard, each group may be analyzed, such as by the processor 20, to determine if the group satisfies one or more predefined geometric properties. For example, one geometric property may relate to the size of a group with a group that is less than a predefined threshold being considered too small to be a whole character. As another example of a geometric property, a group that is located along an edge of the writing area, such as along the leftmost or rightmost portion of the touch screen display 28, may not be considered to be a whole character. The set of characters that is determined, such as by the processor 20, to have the highest score based upon, for example, one or more of the similarity, meaningfulness and geometric properties, may be identified as the set of characters that best represents the plurality of overlapping characters, as shown at 238.
[0058] In one embodiment, one or more restrictions may be placed upon the process of identifying the plurality of possible combinations of the groups in order to increase the efficiency with which the combinations are identified and thereafter processed. For example, the possible combinations may be restricted by limiting any potential combination or character to a maximum of four groups.
[0059] In addition to increasing the efficiency with which a plurality of overlapped characters may be segmented, the segmenting of a series of strokes into one or more groups of strokes may also facilitate the display of the handwritten input. In this regard, the apparatus may also include, means, such as the processor 20, the display 28 or the like, for causing a display of at least some of the groups such that at least one group is caused to be displayed in a visibly distinct manner from at least one other group. See operation 140 of Figure 4. For example, only a subset of the groups may be displayed, such that at least one group is not caused to be displayed. In this regard, a relatively small number of the most recent group may be displayed with all preceding groups removed from the display 28. As such, the resulting display 28 is less cluttered and the user may more readily view the stroke that is currently being inputted as well as the immediately preceding strokes. In other embodiments, the groups of strokes may be displayed in such a manner that a visible distinction is drawn between different groups of strokes. For example, the groups of strokes may be displayed with different colors and/or brightness levels, such as different colors or brightness levels that vary based upon the sequence in which the strokes were received. In one example embodiment, the most recent stroke group may be displayed with the darkest color (and/or brightness), the immediately preceding stroke group may be displayed with a slightly lighter color (and/or brightness) and so on until the initial stroke group that is displayed with the lightest color (and/or brightness). Alternatively, the different stroke groups may be represented by different types of lines, such as the most recent stroke group being represented by a solid line, the immediately preceding stroke group being represented by a dotted and dashed line and so on. In each embodiment, the stroke groups are displayed such that the different stroke groups are visibly distinct and, in some embodiments, the more recent stroke groups are more readily apparent.
[0060] In one embodiment, the determination of the features associated with a stroke and the segmentation of the stroke into a current group may be performed following the input of each stroke as shown, for example, in Figure 7. In relation to the subsequent description of Figure 7, the current stroke and the next stroke (as discussed above) are designated as stroke k-1 and stroke k, respectively. With reference to operations 250, 252 and 254 of Figure 7, following the input of a stroke k, a determination is made as to whether the stroke is the initial stroke, that is, a determination is made as to whether k=0. In instances in which the stroke that has been input is the initial stroke, the real writing area may be initialized. See operation 256. In this regard, the real writing area may be initialized to be the smallest rectangle that encompasses the initial stroke and may defined in terms of a total width and a total height for a rectangle oriented and positioned relative to a coordinate system as shown in Figure 5 or in terms of the top, bottom, left and right coordinates for rectangles positioned in other orientations and positions. Thereafter, the counter k may be incremented prior to awaiting the input of the next stroke. See operation 258. Following entry of the next stroke, the real writing area may be recalculated such that the recalculated real writing area represents the smallest rectangle that encompasses each stroke. See operation 260. Thereafter, the features for the current stroke k -1 may be determined. See operation 262. Based upon the features that have been determined for stroke k -1, a classifier 1 14 may provide a value Spre based upon the features of stroke k-1. In this regard, the classifier 1 14 may provide a value Spre based a feature vector of stroke k-1 and the segmentation rules 1 16, which define the parameters to be utilized by the classifier to evaluate the feature vector of a stroke. See operation 264 of Figure 7. The Spre value provided by the classifier 1 14 may then be compared to a predefined threshold for classification Tpre. See operation 266. If the value Spre provided by the classifier 1 14 exceeds the threshold Tpre for classification, stroke k-1 and stroke k are considered to be in different groups. See operation 268. Conversely, if the value Spre provided by the classifier 1 14 is not greater than the threshold Tpre for classification, stroke k -1 and stroke k are considered to be in the same group. See operation 270. This process as described above and as shown in Figure 7 may be repeated incrementally for each stroke that is input in order to appropriately segment this series of strokes into groups.
[0061] As an alternative to analyzing the strokes on an individual basis following the input of each stroke in order to appropriately group the strokes, a plurality of strokes may be analyzed in a batch process as shown, for example, in Figure 8. In this regard, the real writing area, such as the smallest rectangle that encompasses a plurality of strokes 0, 1, ... M-l, may be determined. See operation 280. A counter k may then be initialized, such as k = 1, and a determination may be made as to whether all strokes of the batch have been considered, such as by comparing the counter to the number of strokes M. See operations 282 and 284. In an instance in which all strokes have not yet been considered, the features for stroke k -1 are determined and the value Spre for stroke k -1 may then be determined, such as by a classifier 1 14, based upon the features for stroke k -1. See operations 286 and 288. As before, a comparison of the value Spre to the threshold Tpre for classification may performed with stroke k -1 and stroke k being placed in different groups or in the same group depending on whether the value Spre is greater than the threshold Tpre for classification or not, respectfully. See operations 290, 292 and 294. This process may be repeated for each individual stroke of the batch as indicated by the incrementing of the counter k in operation 296 until each stroke has been considered and appropriately sorted into groups.
[0062] Figures 4, 7 and 8 are flowcharts of a method and program product according to example embodiments of the invention. It will be understood that each block of the flowcharts, and combinations of blocks in the flowcharts, may be implemented by various means, such as hardware, firmware, processor, circuitry and/or other devices associated with execution of software including one or more computer program instructions. For example, one or more of the procedures described above may be embodied by computer program instructions. In this regard, the computer program instructions which embody the procedures described above may be stored by a memory device of the mobile terminal 10 and executed by a processor 20 in the mobile terminal. As will be appreciated, any such computer program instructions may be loaded onto a computer or other programmable apparatus, e.g., hardware, to produce a machine, such that the instructions which execute on the computer or other programmable apparatus create means for implementing the functions specified in the flowcharts block(s). These computer program instructions may also be stored in a computer-readable memory that may direct a computer or other programmable apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowcharts block(s). The computer program instructions may also be loaded onto a computer or other programmable apparatus to cause a series of operations to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus implement the functions specified in the flowcharts block(s). [0063] Accordingly, blocks of the flowcharts support combinations of means for performing the specified functions, combinations of operations for performing the specified functions and program instructions for performing the specified functions. It will also be understood that one or more blocks of the flowcharts, and combinations of blocks in the flowcharts, can be implemented by special purpose hardware-based computer systems which perform the specified functions, or combinations of special purpose hardware and computer instructions.
[0064] Many modifications and other embodiments of the inventions set forth herein will come to mind to one skilled in the art to which these inventions pertain having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is to be understood that the inventions are not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Moreover, although the foregoing descriptions and the associated drawings describe example embodiments in the context of certain example combinations of elements and/or functions, it should be appreciated that different combinations of elements and/or functions may be provided by alternative embodiments without departing from the scope of the appended claims. In this regard, for example, different combinations of elements and/or functions other than those explicitly described above are also contemplated as may be set forth in some of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation

Claims

WHAT IS CLAIMED IS:
1. A method comprising:
receiving a series of strokes that comprise overlapped handwriting;
for each of a plurality of strokes, determining, with a processor, a plurality of features associated with the current stroke based upon properties of the series of strokes; and segmenting the series of strokes into one or more groups of strokes based upon the features associated with the strokes, wherein each group of strokes is one of a character or a portion of a character but no group of strokes includes strokes from more than one character.
2. A method according to Claim 1 further comprising segmenting the series of strokes into one or more characters based upon the one or more groups of strokes into which the series of strokes has been segmented.
3. A method according to any one of Claims 1 or 2 wherein segmenting the series of strokes comprises segmenting the series of strokes into a plurality of groups, and wherein the method further comprises causing a display of at least some of the groups such that at least one group is caused to be displayed in a visibly distinct manner from at least one other group.
4. A method according to any one of Claims 1 -3 further comprising normalizing the plurality of features associated with the current stroke.
5. A method according to Claim 4 wherein normalizing the plurality of features comprises normalizing the plurality of features based upon an overall dimension of the series of strokes.
6. A method according to any one of Claims 1 -5 wherein determining the plurality of features comprises determining the plurality of features selected from the group consisting of an end point of the current stroke, a geometric center of the current stroke, a beginning point of a next stroke, a geometric center of the next stroke, a smallest rectangle containing the current stroke, a smallest rectangle containing the next stroke and a smallest rectangle containing an immediately prior stroke.
7. A method according to any one of Claims 1 -6 wherein determining the plurality of features and segmenting the series of strokes are repeated following receipt of each successive stroke.
8. A method according to any one of Claim 1 -6 wherein determining the plurality of features and segmenting the series of strokes are performed following receipt of a plurality of strokes.
9. A method according to any of Claims 1-8 wherein determining a plurality of features associated with the current stroke comprises determining the plurality of features associated with the current stroke based only on the geometrical properties of the series of strokes.
10. An apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least:
receive a series of strokes that comprise overlapped handwriting;
for each of a plurality of strokes, determine a plurality of features associated with the current stroke based upon geometrical properties of the series of strokes; and
segment the series of strokes into one or more groups of strokes based upon the features associated with the strokes, wherein each group of strokes is one of a character or a portion of a character but no group of strokes includes strokes from more than one character.
1 1 . An apparatus according to Claim 10 wherein the at least one memory and computer program code are further configured to, with the at least one processor, cause the apparatus to segment the series of strokes into one or more characters based upon the one or more groups of strokes into which the series of strokes has been segmented.
12. An apparatus according to any one of Claims 10 or 1 1 wherein the at least one memory and computer program code are further configured to, with the at least one processor, cause the apparatus to segment the series of strokes by segmenting the series of strokes into a plurality of groups, and to cause a display of at least some of the groups such that at least one group is caused to be displayed in a visibly distinct manner from at least one other group.
13. An apparatus according to any one of Claims 10-12 wherein the at least one memory and computer program code are further configured to, with the at least one processor, cause the apparatus to normalize the plurality of features associated with the current stroke.
14. An apparatus according to Claim 13 wherein the at least one memory and computer program code are further configured to, with the at least one processor, cause the apparatus to normalize the plurality of features by normalizing the plurality of features based upon an overall dimension of the series of strokes.
15. An apparatus according to any one of Claims 10-14 wherein the at least one memory and computer program code are further configured to, with the at least one processor, cause the apparatus to determine the plurality of features by determining the plurality of features selected from the group consisting of an end point of the current stroke, a geometric center of the current stroke, a beginning point of a next stroke, a geometric center of the next stroke, a smallest rectangle containing the current stroke, a smallest rectangle containing the next stroke and a smallest rectangle containing an immediately prior stroke.
27 ctonni iOtir'
16. An apparatus according to any one of Claims 10-15 wherein the at least one memory and computer program code are further configured to, with the at least one processor, cause the apparatus to repeatedly determine the plurality of features and segment the series of strokes following receipt of each successive stroke.
17. An apparatus according to any one of Claim 10-15 wherein the at least one memory and computer program code are further configured to, with the at least one processor, cause the apparatus to determine the plurality of features and segment the series of strokes following receipt of a plurality of strokes.
18. An apparatus according to any of Claims 10-17 wherein the at least one memory and computer program code are further configured to, with the at least one processor, cause the apparatus to determine a plurality of features associated with the current stroke by determining the plurality of features associated with the current stroke based only on the geometrical properties of the series of strokes.
19. An apparatus comprising:
means for receiving a series of strokes that comprise overlapped handwriting; means for determining, for each of a plurality of strokes, a plurality of features associated with the current stroke based upon geometrical properties of the series of strokes; and
means for segmenting the series of strokes into one or more groups of strokes based upon the features associated with the strokes, wherein each group of strokes is one of a character or a portion of a character but no group of strokes includes strokes from more than one character.
20. An apparatus according to Claim 19 further comprising means for segmenting the series of strokes into one or more characters based upon the one or more groups of strokes into which the series of strokes has been segmented.
28 πτΕ ΐ ηη ΐ ΐου Τ
21. An apparatus according to any one of Claims 19 or 20 wherein the means for segmenting the series of strokes comprises means for segmenting the series of strokes into a plurality of groups, and wherein the apparatus further comprises means for causing a display of at least some of the groups such that at least one group is caused to be displayed in a visibly distinct manner from at least one other group.
22. An apparatus according to any one of Claims 19-21 further comprising means for normalizing the plurality of features associated with the current stroke.
23. An apparatus according to Claim 22 wherein the means for normalizing the plurality of features comprises means for normalizing the plurality of features based upon an overall dimension of the series of strokes.
24. An apparatus according to any one of Claims 19-23 wherein the means for determining the plurality of features comprises means for determining the plurality of features selected from the group consisting of an end point of the current stroke, a geometric center of the current stroke, a beginning point of a next stroke, a geometric center of the next stroke, a smallest rectangle containing the current stroke, a smallest rectangle containing the next stroke and a smallest rectangle containing an immediately prior stroke.
25. An apparatus according to any one of Claims 19-24 wherein the means for determining the plurality of features and the means for segmenting the series of strokes repeatedly operate following receipt of each successive stroke.
26. An apparatus according to any one of Claim 19-24 wherein the means for determining the plurality of features and the means for segmenting the series of strokes operate following receipt of a plurality of strokes.
27. An apparatus according to any of Claims 19-24 wherein the means for determining a plurality of features associated with the current stroke comprises means for determining the plurality of features associated with the current stroke based only on the geometrical properties of the series of strokes.
28. A computer program product comprising at least one computer-readable 5 memory having computer-executable code portions stored therein, the computer- executable code portions comprising:
program code instructions for receiving a series of strokes that comprise overlapped handwriting;
program code instructions for determining, for each of a plurality of strokes, a 10 plurality of features associated with the current stroke based upon geometrical properties of the series of strokes; and
program code instructions for segmenting the series of strokes into one or more groups of strokes based upon the features associated with the strokes, wherein each group of strokes is one of a character or a portion of a character but no group of strokes includes ] 5 strokes from more than one character.
29. A computer program product according to Claim 28 further comprising program code instructions for segmenting the series of strokes into one or more characters based upon the one or more groups of strokes into which the series of strokes 0 has been segmented.
30. A computer program product according to any one of Claims 28 or 29 wherein the program code instructions for segmenting the series of strokes comprise program code instructions for segmenting the series of strokes into a plurality of groups, 5 and wherein the computer-executable code portions further comprise program code instructions for causing a display of at least some of the groups such that at least one group is caused to be displayed in a visibly distinct manner from at least one other group.
31. A computer program product according to any one of Claims 28-30 further 0 comprising program code instructions for normalizing the plurality of features associated with the current stroke.
32. A computer program product according to Claim 31 wherein the program code instructions for normalizing the plurality of features comprise program code instructions for normalizing the plurality of features based upon an overall dimension of the series of strokes.
33. A computer program product according to any one of Claims 28-32 wherein the program code instructions for determining the plurality of features comprise program code instructions for determining the plurality of features selected from the group consisting of an end point of the current stroke, a geometric center of the current stroke, a beginning point of a next stroke, a geometric center of the next stroke, a smallest rectangle containing the current stroke, a smallest rectangle containing the next stroke and a smallest rectangle containing an immediately prior stroke.
34. A computer program product according to any one of Claims 28-33 wherein the program code instructions for determining the plurality of features and the program code instructions for segmenting the series of strokes are executed following receipt of each successive stroke.
35. A computer program product according to any one of Claim 28-33 wherein the program code instructions for determining the plurality of features and the program code instructions for segmenting the series of strokes are executed following receipt of a plurality of strokes.
36. A computer program product according to any of Claims 28-35 wherein the program code instructions for determining a plurality of features associated with the current stroke comprise program code instructions for determining the plurality of features associated with the current stroke based only on the geometrical properties of the series of strokes.
PCT/CN2010/076285 2010-08-24 2010-08-24 Method and apparatus for segmenting strokes of overlapped handwriting into one or more groups WO2012024829A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2013525107A JP5581448B2 (en) 2010-08-24 2010-08-24 Method and apparatus for grouping overlapping handwritten character strokes into one or more groups
KR1020137007240A KR101486174B1 (en) 2010-08-24 2010-08-24 Method and apparatus for segmenting strokes of overlapped handwriting into one or more groups
PCT/CN2010/076285 WO2012024829A1 (en) 2010-08-24 2010-08-24 Method and apparatus for segmenting strokes of overlapped handwriting into one or more groups
CN201080068735.8A CN103080878B (en) 2010-08-24 2010-08-24 One or more groups of method and apparatus are divided into for hand-written stroke will be overlapped

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2010/076285 WO2012024829A1 (en) 2010-08-24 2010-08-24 Method and apparatus for segmenting strokes of overlapped handwriting into one or more groups

Publications (1)

Publication Number Publication Date
WO2012024829A1 true WO2012024829A1 (en) 2012-03-01

Family

ID=45722811

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2010/076285 WO2012024829A1 (en) 2010-08-24 2010-08-24 Method and apparatus for segmenting strokes of overlapped handwriting into one or more groups

Country Status (4)

Country Link
JP (1) JP5581448B2 (en)
KR (1) KR101486174B1 (en)
CN (1) CN103080878B (en)
WO (1) WO2012024829A1 (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103345365A (en) * 2013-07-12 2013-10-09 北京蒙恬科技有限公司 Display method of continuous hand input and hand input device adopting method
CN103376998A (en) * 2012-04-19 2013-10-30 中兴通讯股份有限公司 Chinese typesetting method and device for handwriting equipment
US20130315483A1 (en) * 2012-05-28 2013-11-28 Kabushiki Kaisha Toshiba Handwritten document retrieval apparatus and method
US20130315484A1 (en) * 2012-05-28 2013-11-28 Kabushiki Kaisha Toshiba Handwritten character retrieval apparatus and method
WO2014166114A1 (en) * 2013-04-12 2014-10-16 Nokia Corporation An apparatus for text entry and associated methods
US9224038B2 (en) 2013-12-16 2015-12-29 Google Inc. Partial overlap and delayed stroke input recognition
US9881224B2 (en) 2013-12-17 2018-01-30 Microsoft Technology Licensing, Llc User interface for overlapping handwritten text input
JP2022116104A (en) * 2013-06-09 2022-08-09 アップル インコーポレイテッド Managing real-time handwriting recognition
US11816326B2 (en) 2013-06-09 2023-11-14 Apple Inc. Managing real-time handwriting recognition
US11941243B2 (en) 2016-06-12 2024-03-26 Apple Inc. Handwriting keyboard for screens

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5681838B2 (en) * 2012-05-31 2015-03-11 マルチタッチ オーユーMultitouch Oy User interface for drawing with electronic devices
KR102125212B1 (en) * 2013-08-29 2020-07-08 삼성전자 주식회사 Operating Method for Electronic Handwriting and Electronic Device supporting the same
JP2015099566A (en) * 2013-11-20 2015-05-28 株式会社東芝 Feature calculation device, method and program
US9524440B2 (en) * 2014-04-04 2016-12-20 Myscript System and method for superimposed handwriting recognition technology
CN105095924A (en) * 2014-04-25 2015-11-25 夏普株式会社 Handwriting recognition method and device
JP7071840B2 (en) * 2017-02-28 2022-05-19 コニカ ミノルタ ラボラトリー ユー.エス.エー.,インコーポレイテッド Estimating character stroke information in the image
CN108492349B (en) * 2018-03-19 2023-04-11 广州视源电子科技股份有限公司 Processing method, device and equipment for writing strokes and storage medium
CN110503101A (en) * 2019-08-23 2019-11-26 北大方正集团有限公司 Font evaluation method, device, equipment and computer readable storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1188283A (en) * 1996-12-06 1998-07-22 国际商业机器公司 Object oriented framework mechanism for metering objects
US20050063592A1 (en) * 2003-09-24 2005-03-24 Microsoft Corporation System and method for shape recognition of hand-drawn objects
CN101299236B (en) * 2008-06-25 2010-06-09 华南理工大学 Method for recognizing Chinese hand-written phrase

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS60254384A (en) * 1984-05-31 1985-12-16 Fujitsu Ltd Corresponding method of stroke
JPH05233599A (en) * 1992-02-18 1993-09-10 Seiko Epson Corp Online character recognizing device
JPH06162266A (en) * 1992-11-20 1994-06-10 Seiko Epson Corp Method for recognizing on-line handwritten character and device therefor
JPH0792817B2 (en) * 1994-08-19 1995-10-09 沖電気工業株式会社 Online character recognizer
JPH08161426A (en) * 1994-12-09 1996-06-21 Sharp Corp Handwritten character stroke segmenting device
JP2939147B2 (en) * 1994-12-29 1999-08-25 シャープ株式会社 Handwritten character input display device and method
JPH09161011A (en) * 1995-12-13 1997-06-20 Matsushita Electric Ind Co Ltd Handwritten character input device
JP3216800B2 (en) * 1997-08-22 2001-10-09 日立ソフトウエアエンジニアリング株式会社 Handwritten character recognition method
JP3024680B2 (en) * 1998-01-13 2000-03-21 日本電気株式会社 Handwritten pattern storage and retrieval device
JP3456931B2 (en) * 1999-12-10 2003-10-14 シャープ株式会社 Handwritten character recognition device, computer readable recording medium storing a handwritten character recognition program, and method for correcting characters recognized by handwritten characters
JP3974359B2 (en) * 2000-10-31 2007-09-12 株式会社東芝 Online character recognition apparatus and method, computer-readable storage medium, and online character recognition program
JP2005141329A (en) * 2003-11-04 2005-06-02 Toshiba Corp Device and method for recognizing handwritten character
KR100677426B1 (en) * 2005-01-14 2007-02-02 엘지전자 주식회사 Short message display method for mobile communication device
JP2009289188A (en) * 2008-05-30 2009-12-10 Nec Corp Character input device, character input method and character input program

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1188283A (en) * 1996-12-06 1998-07-22 国际商业机器公司 Object oriented framework mechanism for metering objects
US20050063592A1 (en) * 2003-09-24 2005-03-24 Microsoft Corporation System and method for shape recognition of hand-drawn objects
CN101299236B (en) * 2008-06-25 2010-06-09 华南理工大学 Method for recognizing Chinese hand-written phrase

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103376998B (en) * 2012-04-19 2016-06-15 中兴通讯股份有限公司 Handwriting equipment Chinese character type-setting method and device
CN103376998A (en) * 2012-04-19 2013-10-30 中兴通讯股份有限公司 Chinese typesetting method and device for handwriting equipment
US9230181B2 (en) * 2012-05-28 2016-01-05 Kabushiki Kaisha Toshiba Handwritten document retrieval apparatus and method
US20130315484A1 (en) * 2012-05-28 2013-11-28 Kabushiki Kaisha Toshiba Handwritten character retrieval apparatus and method
CN103455527A (en) * 2012-05-28 2013-12-18 株式会社东芝 Handwritten document retrieval apparatus, handwritten document retrieval method and recording medium
CN103455528A (en) * 2012-05-28 2013-12-18 株式会社东芝 Handwritten character retrieval apparatus, handwritten character retrieval method and recording medium
US9424477B2 (en) * 2012-05-28 2016-08-23 Kabushiki Kaisha Toshiba Handwritten character retrieval apparatus and method
US20130315483A1 (en) * 2012-05-28 2013-11-28 Kabushiki Kaisha Toshiba Handwritten document retrieval apparatus and method
CN105283882A (en) * 2013-04-12 2016-01-27 诺基亚技术有限公司 Production method for portable data carriers
WO2014166114A1 (en) * 2013-04-12 2014-10-16 Nokia Corporation An apparatus for text entry and associated methods
JP2022116104A (en) * 2013-06-09 2022-08-09 アップル インコーポレイテッド Managing real-time handwriting recognition
JP7361156B2 (en) 2013-06-09 2023-10-13 アップル インコーポレイテッド Managing real-time handwriting recognition
US11816326B2 (en) 2013-06-09 2023-11-14 Apple Inc. Managing real-time handwriting recognition
CN103345365A (en) * 2013-07-12 2013-10-09 北京蒙恬科技有限公司 Display method of continuous hand input and hand input device adopting method
US9224038B2 (en) 2013-12-16 2015-12-29 Google Inc. Partial overlap and delayed stroke input recognition
US10185872B2 (en) 2013-12-16 2019-01-22 Google Llc Partial overlap and delayed stroke input recognition
US9881224B2 (en) 2013-12-17 2018-01-30 Microsoft Technology Licensing, Llc User interface for overlapping handwritten text input
US11941243B2 (en) 2016-06-12 2024-03-26 Apple Inc. Handwriting keyboard for screens

Also Published As

Publication number Publication date
CN103080878B (en) 2017-03-29
JP5581448B2 (en) 2014-08-27
JP2013543158A (en) 2013-11-28
KR101486174B1 (en) 2015-01-23
CN103080878A (en) 2013-05-01
KR20130058053A (en) 2013-06-03

Similar Documents

Publication Publication Date Title
WO2012024829A1 (en) Method and apparatus for segmenting strokes of overlapped handwriting into one or more groups
US10445562B2 (en) AU feature recognition method and device, and storage medium
US9613296B1 (en) Selecting a set of exemplar images for use in an automated image object recognition system
Arai et al. Method for real time text extraction of digital manga comic
EP3086206B1 (en) Method, apparatus and computer program product for providing gesture analysis
CN109241861B (en) Mathematical formula identification method, device, equipment and storage medium
US20120314902A1 (en) Image processing apparatus, image processing method, and program
CN107333071A (en) Video processing method and device, electronic equipment and storage medium
CN111626371B (en) Image classification method, device, equipment and readable storage medium
CN102508606A (en) Method and system for subdividing belonged groups of users by face recognition and setting corresponding functions of mobile handsets
CN110942004A (en) Handwriting recognition method and device based on neural network model and electronic equipment
WO2017197593A1 (en) Apparatus, method and computer program product for recovering editable slide
Mali et al. Indian sign language recognition using SVM classifier
EP3200092A1 (en) Method and terminal for implementing image sequencing
CN111178310A (en) Palm feature recognition method and device, computer equipment and storage medium
CN111722717A (en) Gesture recognition method and device and computer readable storage medium
CN113392820B (en) Dynamic gesture recognition method and device, electronic equipment and readable storage medium
CN110555431B (en) Image recognition method and device
WO2013104322A1 (en) Object recognizing method and object recognizing device
CN108021918B (en) Character recognition method and device
KR20050003075A (en) A mobile communication device with optical character recognition and electronic dictionary, an electronic dictionary with optical character recognition
CN105094544B (en) Method and device for acquiring characters
CN113392821B (en) Dynamic gesture recognition method and device, electronic equipment and readable storage medium
Passos et al. A Brazilian sign language gesture recognizing system using gait energy image
Saranya et al. Sign Language Recognition Using Convolutional Neural Network

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 201080068735.8

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10856282

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2013525107

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 20137007240

Country of ref document: KR

Kind code of ref document: A

122 Ep: pct application non-entry in european phase

Ref document number: 10856282

Country of ref document: EP

Kind code of ref document: A1