WO2012071730A1

WO2012071730A1 - Method, apparatus, and computer program product for overlapped handwriting

Info

Publication number: WO2012071730A1
Application number: PCT/CN2010/079383
Authority: WO
Inventors: Yanming Zou; Yingfei Liu; Xiantao Chen; Ying Liu
Original assignee: Nokia Corporation
Priority date: 2010-12-02
Filing date: 2010-12-02
Publication date: 2012-06-07
Also published as: HK1189066A1; KR101548835B1; KR20130093672A; CN103299254A; CN103299254B; TW201237764A; JP2013546081A

Abstract

Embodiments of the present invention relates to a method, apparatus, and computer program product for use in overlapped handwriting. The method comprises: causing display of at least one segment obtained by a segmentation of overlapped handwriting; receiving a request for correcting the segmentation; and modifying, in response to the received request, the at least one segment to obtain at least one new segment.

Description

METHOD, APPARATUS, AND COMPUTER PROGRAM PRODUCT FOR

OVERLAPPED HANDWRITING

FIELD OF THE INVENTION

[0001] Embodiments of the present invention generally relate to information processing. More particularly, embodiments of the present invention relate to method, apparatus, and computer program product for overlapped handwriting.

BACKGROUND OF THE INVENTION

[0002] As known, handwriting is a natural and convenient mechanism for user to input information into an electronic device such as a desktop computer, tablet computer, laptop computer, personal digital assistant (PDA), or a cellular phone. In particular, in conjunction with appropriate character recognition engine(s), handwriting may be used to input texts which are usable for various purposes, such as editing or word-processing, within the electronic device.

[0003] In handwriting input operation, user may utilize a writing tool, such as a stylus, pen, or just his/her finger, to write on a specific writing area (e.g., a writing pad, a touch screen, or a proximity screen) on the target device, as writing on an ordinary paper. Traces of the tool on the writing area (referred as "pen trace" hereafter) are recorded, and one or more character recognition engines then execute suitable recognition algorithms to convert the recorded pen traces into characters. There are already several available recognition algorithms or products for character recognition, for example, based upon structural recognition, statistical models like Hidden Markov model, neural network, etc, Due to its naturalness and convenience, handwriting has been increasingly employed in more computing device, especially for mobile terminals.

[0004] Recently, there has been proposed a technique called overlapped handwriting, allowing user to write more than one characters continuously in a single writing area, where each character as input is overlapped on top of the previous one, as shown in Fig. l . By use of overlapped handwriting, user may write continuously without waiting for timeouts for the recognition of each character. In this way, multiple characters may be written smoothly at a time without break, thereby improving the operation efficiency. In addition, compared with other handwriting solutions in which a plurality of writing areas are provided to support continues writing, overlapped handwriting solution requires relatively small writing area, which is crucial for the devices with limited size.

[0005] Referring to Fig, 1 , a high level schematic diagram illustrating the general process of overlapped handwriting is shown. As shown, in overlapped handwriting, the pen trace 102 composed of strokes for at least one character needs to be divided into at least one segment 104. This process is referred as segmentation. The recognition engine then converts the one or more segments into recognized characters 106 as output. In the prior art, the segmentation process is performed at background. In other words, the segments obtained from the original overlapped handwriting will be not displayed to user, but directly recognized. Usually, the segmentation and recognition are performed simultaneously, with the support of specific language models. In this way, the recognition engine is expected to give out the most meaningful character string based on the input pen traces and corresponded segmentation result. Apparently, the final recognition accuracy relies so much on the quality of segmentation process,

[0006] Ideally, the recognized character string shall be exactly the one user originally wants to input. Unfortunately, though several solutions have been proposed to improve the accuracy of recognition for overlapped handwriting, errors are somewhat inevitable. The segmentation and recognition may both suffer from errors. In practice, when the segmentation is correct but the recognized character string is not, several existing error correct methods, like candidate list or rewriting a single character, can work well. However, in the case that an error occurs in the segmentation process and therefore causes an error in output recognized character string, traditional correction methods (e.g., a candidate list) are usually frustrated.

[0007] For example, a type of common error in segmentation is so-called one-stroke error, in which a segment has one extra stroke, or lacks one stroke. According to statistics, among all of possible segmentation errors, on-stroke error is dominant. There are also other types of segmentation errors as will be discussed below. In the event that an error occurs in segmentation, the final recognition results are likely to be incorrect. In such case, there is no way in the prior by which the user may correct the recognition results efficiently, and thus the user has to re -write one or more characters, which brings negative impacts on operation efficiency and thus user experience.

SUMMARY OF THE INVENTION

[0008] In view of the foregoing problems in the existing approaches for overlapped handwriting, there is a need in the art to provide a method, apparatus, and computer program product for correcting errors in overlapped handwriting recognition caused by faulty segmentation, so as to overcome the above defects.

[0009] In one aspect, embodiments of the present invention provide a method for use in overlapped handwriting. The method comprises: causing display of at least one segment obtained by a segmentation of overlapped handwriting; receiving a request for correcting the segmentation; and modifying, in response to the received request, the at least one segment to obtain at least one new segment.

[0010] In another aspect, embodiments of the present invention provide apparatus for use in overlapped handwriting. The apparatus comprising: at least one processor; and at least one memory storing program of computer executable code. The at least one memory and the computer executable code configured, with the at least one processor, to cause the apparatus to at least: cause display of at least one segment obtained by a segmentation of overlapped handwriting; receive a request for correcting the segmentation; and modify, in response to the received request, the at least one segment to obtain at least one new segment.

[0011] In still another aspect, embodiments of the present invention provide a computer program product, comprising at least one computer readable storage medium having a computer readable program code portion stored thereon. The computer executable code portion comprising: code for causing display of at least one segment obtained by a segmentation of overlapped handwriting; code for receiving a request for correcting the segmentation; and code for modifying, in response to the received request, the at least one segment to obtain at least one new segment. [0012] In yet another aspect, embodiments of the present provide an apparatus for use in overlapped handwriting. The apparatus comprises: means for causing display of at least one segment obtained by a segmentation of overlapped handwriting; means for receiving a request for correcting the segmentation; and means for modifying, in response to the received request, the at least one segment to obtain at least one new segment.

[0013] With embodiments of the present invention, for example, when there is any error in the recognized character string, segmentation result (i.e., at least one segment) may be presented to user in a modifiable or editable way. As such, user may find errors, if any, in segmentation intuitively. Then he/she may request to correct segmentation errors in efficient and easy manner, for example, by merely a few actions, According to some embodiments of the present invention, the user is enabled to initiate correction or modification of segmentation results by use of a set of pre-defined gestures, such as dragging, click, or sway. In response, modifications or corrections on the segmentation results may be performed in accordance with the user's request.

[0014] As can be appreciated by a skilled in the art, embodiments of the present invention provide user with a convenient mechanism by which the segmentation errors could be corrected interactively, so as to correct recognition errors resulted from faulty segments. In this way, there is provided an efficient solution by which the user may correct the recognition error caused by faulty segmentation, and thus user experience may be significantly improved. Alternatively or additionally, the user may also be enabled to check and correct the segmentation results before executing recognition. In this way, it may be ensured that the segments to be recognized are error-free. Besides, according to some embodiments of the present invention, the user may also change the input on his/her own initiative by, for example, adding one or more strokes to existing segments.

[0015] Other features and advantages of embodiments of the present invention will also be understood from the following description of specific embodiments when read in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of embodiments of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS [0016] Embodiments of the invention are presented in the sense of examples and their advantages are explained in greater detail below, with reference to the accompanying drawings, where

[0017] FIG. 1 shows a high level schematic diagram illustrating a general process of overlapped handwriting;

[0018] FIG. 2 shows a flowchart of a method for use in overlapped handwriting according to embodiments of the present invention;

[0019] FIGs. 3A-3F show examples of several gestures that can be used to requesting for correcting segmentation and their respective modifications according to embodiments of the present invention;

[0020] FIGs. 4A-4B show examples where the portion or segment being operated is highlighted according to embodiments of the present invention;

[0021] FIG. 5 shows a schematic diagram of a graphical user interface (GUI) that is suitable for use in practicing embodiments of the present invention; [0022] FIG. 6 shows a simplified block diagram of an apparatus for use in overlapped handwriting according to embodiments of the present invention; and

[0023] FIG. 7 shows a simplified block diagram of an apparatus that is suitable for use in practicing embodiments of the present invention. DETAILED DESCRIPTION OF EMBODIMENTS

[0024] Embodiments of the present invention provide a method, apparatus, and computer program product for use in overlapping handwriting, in particular, for enabling user to correct segmentation error interactively. According to embodiments of the present invention, the segments obtained by a segmentation of overlapped handwriting may be displayed to user in a modifiable or editable manner. In particular, the positional and temporal information for each portion (i.e., one or more strokes) contained within each segment is available. Therefore, the user is able to select any portion of any segment, and modify the portion as desired, for example, using a set of predefined gestures. As a result, the user is enabled to correct any segmentation error efficiently, thereby correcting the final recognition results. Alternatively or additionally, the user may also be enabled to check and correct the segmentation results before executing recognition. In this way, it is ensured that the segments to be recognized are error-free, such that the final recognition accuracy may be increased, [0025] Embodiments of the present invention will be described below with reference to the figures. It will be appreciated that though the specific embodiments of the present invention will be discussed below primarily with reference to Chinese characters, the scope of invention is not limited in this regard. Embodiments of the present invention are equally applicable to characters of any language which is suitable for overlapped handwriting.

[0026] As mentioned above, FIG. 1 shows a high level schematic diagram illustrating the general process of overlapped handwriting. In the prior art, the intermediate results of segmentation are not displayed for user. On the contrary, according to embodiments of the present invention, such segmentation results, that is, one or more segments obtained by segmentation of the original overlapped handwriting, may be presented to the user and modified by the user, which will be discussed in detail below.

[0027] FIG. 2 shows a flowchart of a method 200 for use in overlapped handwriting according to embodiments of the present invention. As shown, the method 200 starts at step S205, where at least one segment obtained by a segmentation of overlapped handwriting is displayed to user. The at least one segment is obtained, for example, from a segmentation module, which is able to perform segmentation process on the overlapped handwriting to generate one or more segment. It should be understood that the segments may be displayed upon the user's request, for example, when the user wants to correct an error in recognized character string. Additionally or alternatively, the segments may be automatically displayed along with the final recognized string. In some embodiments, the segments may be displayed via a graphical user interface such as that shown in FIG. 5, which will be discussed in detail below. [0028] A segment serves as a basic unit of subsequent character recognition, and thus shall correspond to a single and whole character in ideal case, as described above. A segment is composed of one or more strokes. As used herein, term "stroke" refers to the basic and inseparable element of a character or a segment.

[0029] It should be understood that, at step S205, the at least one segment is displayed in a modifiable or editable manner. That is, the user is enabled to select and then manipulate one or more strokes of a segment, or a segment as whole. To this end, the positional information of the segment strokes is needed. Such information may be obtained from the preceding segmentation process. As known in the art, while performing segmentation on the user's overlapped handwriting, for each resulting segment, the positional information of every stroke contained in this segment can be determined, regardless of the specific algorithm as employed. In other words, based upon the segmentation process, the positional information of each stroke is available for further use, e.g., the selection or pick of a stroke as discussed below. Therefore, for each point in the pen trace, the segment and stroke (s) to which this point belongs is determinable. [0030] Besides, the temporal information of each stroke is also available.

For example, the temporal information of strokes may be obtained by recording user's writing order during handwriting. In practice, it might be difficult for user to select a specific stroke from a segment merely based upon positional information, for example, when the structure of a segment is complex. At this point, the temporal information of strokes may be used, alone or in combination with the positional information, in selecting a stroke. For example, in a preferred embodiment, a time line for the strokes may be drawn under each segment, such that the user can select a portion of timeline and then the corresponding one or more strokes in the segment are selected. Other alternatives for selecting a stroke or a portion based upon the positional and/or temporal information are possible as well.

[0031] It should be understood that any current existing or future developed method may be used for performing segmentation of overlapped handwriting, and the positional and/or temporal information of strokes may be obtained in different ways accordingly. The scope of invention is not limited in this regard. [0032] Next, at step S210, a request for correcting the segmentation is received. In the case that the user finds one or more errors in the segments as displayed at step S205, he/she may initiate correction request in several ways. Then, at step S215, in response to the request received at step S210, the at least one segment is modified to obtain at least one new segment. The operations of steps S210 and S215 will now be discussed in detail. [0033] In preferred embodiments of the present invention, at step S205, the request may be received via a set of predefined gestures by the user, and the modification of segments may be made accordingly at step S210. Some typical gestures that may be used to request for correcting segmentation as shown in FIGs. 3A-3F. [0034] Referring to FIG. 3 A, in some cases, it is possible that the segmentation process improperly divides the pen trace of a single character as more than one separated segments. For example, as shown in FIG. 3 A, one of the Chinese characters that user wants to input by overlapped handwriting is " ." However, in the segmentation, the pen trace of this single character is divided as two separate segments, a first segment 310 " E!" and a second segment 312 "fl ." To correct such error, according to some embodiments of the present invention, at step S210 of method 200, the user may request to modify the segmentation in the following way.

[0035] Firstly, the user may select or pick the first segment 310, for example, by clicking within or adjacent to the bounding box of the first segment 310 using a pointing device (such as a pen, stylus, finger, or mouse), or by pressing a specific button. Then the user is allowed to drag the selected first segment 310 to a predefined proximity of the second segment 312, i.e., to a position where the distance between the first arid second segments 310 and 312 is below a predefined threshold distance. In response, modification will be performed where the first and second segments 310 and 315 are combined into one segment 314 ^{li f>}M " as shown in FIG. 3 A. It should be understood that the gestures described above are merely illustrative, and other gestures are possible as well,

[0036] On the other hand, another type of segmentation error is just opposite with respect to the situation as shown in FIG. 3A, where the pen traces of two or more separated characters are incorrectly combined as a single segment during the segmentation. FIG. 3B shows such an example. In this example, the user wants to input two Chinese characters "EJ" and but the segmentation engine generates a single segment 320 According to embodiments of the present invention, such error may be corrected by performing re-segmentation on the segment in question. The user may request re-segmentation by making a predefined gesture on the segment, In some embodiments, the predefined gesture may be, for example, double click, long press, or any other suitable gestures. As shown FIG. 3B, the user may make the predefined gesture (e.g., double click) on the segment 320 Then, by use of the positional and temporal information of related strokes, a re-segmentation may be performed in response, such that the segment 320 is partitioned into a number of segments (two segments 322 and 324 in the example shown in FIG. 3B), It should be understood that double click is just an illustrative example of the predefined gesture that can be used to request a re-segmentation, and other gestures are possible as well.

[0037] FIG. 3C shows a situation similar to that of FIG. 3B, in that two or more segments are improperly combined as a single one during segmentation. However, unlike FIG. 3B, some of the combined segments are quite simple, each of which is composed of a few strokes (e.g., just one or two strokes). According to embodiments of the present invention, at this point, rather than calling the segmentation module to perform re-segmentation, the user may choose to simply separate a delineated portion from a segment by dragging the portion to a predefined area, for example, an empty area.

[0038] As used herein, term "portion" refers to a part of a segment which is composed of one or more strokes. It should be understood that the selection of a portion may be done by selecting its strokes, and the selection of a stroke may be done by utilizing its positional and/or temporal information, which is available from the segmentation process, as described above. For example, when any point on a stroke is clicked, this stroke may be selected.

[0039] Continuing the example of FIG. 3C, upon selected and dragged to a predefined area, the portion being operated may be removed from the original segment, and a new segment will be obtained based upon the portion. As such, one segment is partitioned to two, and this procedure may be repeated until the results are correct. Specifically, as shown in FIG. 3C, the pen trace of Chinese character "A" and Chinese full stop "©" are combined as one segment 330 by error. At this point, the user may first select the stroke of the full stop, and then drag the selected stroke into an empty area. In response, the stroke will be removed from the combined segment 330, leaving only " A.." In addition, a new segment 334 will be generated which is composed of the dragged portion. As a result, two separated segments 332 "A." and 334 "o" are obtained. It should be understood that though the delineated portion shown in FIG. 3C contains only one stroke, embodiments of the present invention is also applicable to a portion composed of a plurality of strokes.

[0040] Another type of segmentation error involves the order of two or more segments. Unlike the single character handwriting, when utilizing overlapped handwriting, a user usually wants to input a plurality of character, e.g., a phase or even a sentence, at a time. Therefore, the recognition results shall ensure not only the correctness of each and every character, but also the proper order of the character sequence. Referring to FIG. 3D for example, the user originally wants to input a Chinese phase "τη ^αΆ ~ψ". However, due to user's clerical error or a segmentation error, the order of segments 342 " ^SB " and 340 " y " are reversed. According to embodiments of the present invention, the user may request to correct the order of segments by, for example, selecting a segment and dragging it to a position in the segment sequence. Then the order of this segment in the sequence will be altered based upon the target position. Specifically, as shown in FIG. 3D, the user selects the segment 340 " TP " and drag it to a position after the segment 342 " ^H I ". In response, the order of the segments 340 and 342 is altered. In some further embodiments, dragging a segment to, for example, a position in an empty area may also cause the re-ordering of the segment sequence. [0041] Referring now to FIG. 3E₅ another example of segmentation correction is shown. In this example, a stroke (a vertical stroke or "Shu") of a segment 352 " -¾" is incorrectly assigned to a second segment 350 That is, a one-stroke error as described above occurs in the segmentation. As a result, the segment 350 "†" has one more stroke while the segment 352 lacks a stroke, which is very likely to cause the recognition errors of both segments. According to embodiments of the present invention, the user may request for corrections in the following way to deal with such segmentation error.

[0042] The user is allowed to select the incorrectly assigned portion, which may contain one or more strokes, of a first segment. Then, the selected portion of the first segment may be dragged to a predefined proximity of a second segment to which the portion shall belong. In response to such gesture, the modifications on the first and second segments will be carried out, such that the selected portion is removed from the first segment, and added to the second segment based upon, for example, the positional and temporal information of the related strokes. Specifically, as shown in FIG. 3E, the user may select the vertical stroke in the segment 350, and drag it to a predefined proximity of the segment 352. In this way, two modified new segments 354 and 356 are obtained.

[0043] In practice, it is found that the most error-prone strokes are the ones on the "boundary" of two adjacent characters, since such strokes are most likely to be incorrectly assigned to another segment during segmentation. For example, the first one or more strokes of a segment may be assigned to the one which is immediately before the segment (e.g., FIG. 3E). Similarly, the last one or more strokes of a segment is often assigned to the one which is immediately next to the segment in question. According to embodiments of the present invention, in order to better deal with such common situations, specific gestures may be provided to allow user to select the first or last stroke(s) of a character more easily and accurately. An example of such gesture is sway. For example, the first one or more strokes of a segment may be selected by clicking within the bounding box of this segment and then quickly moving left, while the last one or more strokes may be selected by clicking within the bounding box and quickly moving right. As described above, given a segment, due to the availability of the temporal information (i.e., the writing order) of each and every stroke, the selection of first or last one or more strokes is possible. Of course, other gestures for selecting the first and/or last stroke of a segment are possible as well. The scope of invention is not limited in this regard.

[0044] Please note, although one-stroke error is shown in FIG. 3E, embodiments of the present invention is applicable to the case in which more than one strokes of a segment are incorrectly assigned to another segment. [0045] Referring to FIG. 3F, according to embodiments of the present invention, the user is also allowed to add one or more strokes to an existing segment. Sometimes due to user's clerical error or a segmentation error, one or more strokes of a character may be missed. Therefore, compared with the character that the user originally wants to input, the segment may lack one or more strokes. In another event, the user might want to change the input on his/her own initiative. In either case, the user may directly add a portion (i.e., one or more strokes) to any existing segment, rather than re-typing from scratch. In response, a new segment may be generated based upon the segment and the portion in question. As shown in FIG. 3F, by adding a stroke to a segment 360 "A", another segment 362 is generated which is composed of the segment 360 "A." and the added portion "— ."

[0046] From the above discussion, it can be seen that when requesting for segmentation correction, a user is enabled to select a specific portion containing one or more strokes, or a segment as whole. To this end, the gestures for portion selection and those for segment selection shall be differentiated clearly enough. According to some embodiments of the present invention, the differentiation may be achieved by the duration of press or click. For example, a short press or click may correspond to the selection of stroke, while a long press or click may correspond to the selection of a whole segment. As another example, there may be provided an option or a flag that the user is able to set. If the option or flag is set to a first status, then the gesture will be interpreted as portion selection. On the other hand, if the option or flag is set to a second status, then the gesture will be used for segment selection. These are merely some illustrative examples. Any other mechanism for differentiating the sections of a portion and of a whole segment may be employed. [0047] It should be understood that the gestures as describe above may be configurable. That is, a user may add new gestures, remove gestures, and/or modify existing gestures, depending on his/her preference and/or the status of the device. Alternatively or additionally, various parameters related to the gestures, such as the above mentioned predefined proximity (the threshold distance), predefined area, and/ or predefined position, may be also configured by the user.

[0048] Moreover, as can be appreciated by a skilled in the art, FIGs. 3 A- 3E just show some illustrative and non-limiting examples of gestures and corresponding operations for segmentation correction according to embodiments of the present invention. Given the above teaching, a skilled in the art will easily envisage other gestures for requesting segmentation corrections. In addition, interaction means other than gesture may be employed to request for correcting segmentation. For example, controls like button, soft key, and dropdown list may be used, alone or in combination, to input correction request. As another example, the user may directly type specific pre-determined commands to initiate the request. The scope of invention is not limited in these regards. [0049] Returning to FIG, 2, at step S220, the portion or segment being operated is highlighted during the modification. By means of highlighting the element being operated, the user is able to always be aware of interaction focus and avoid being confused during the modification. The highlight may be done in many different ways. For example, color, appearance, shadow, font and /or size of the operated element may be changed. Alternatively or additionally, animation effects may be applied to the portion and/or segment being operated.

[0050] Referring to FIGs. 4A-4B, examples according to embodiments of the present invention are shown, where the portion or segment being operated is highlighted by a Fish Eye lens. In the shown examples, when the user initiates the operation on a specific portion (FIG. 4A) or a segment (FIG. 4B), the portion or segment being operated will be displayed in a Fish Eye. The Fish Eye may be moved during the modification, for example, according to the motion of the operated portion or segment. In addition to the position, the contents as displayed within the Fish Eye may also be updated dynamically during the manipulation. For instance, in the example as shown in FIG 3E, when the user first selects a stroke of segment 350, the segment 350 and the selected stroke are included in the Fish Eye, possibly with the color of the selected stroke changed. While being dragged to the segment 352, the selected stroke may be highlighted in the Fish Eye. Finally, as the stroke and the segment 352 are combined, the resulting segment will be displayed in the Fish Eye. It should be understood that the Fish Eye is merely an illustrative example, and may be used in conjunction with one or more other highlight effects (e.g., color change, as shown in FIGs. 4A-4B). [0051] It shall also be noted that step S220 is optional, since the user may select to enable/disable the highlight of element being operated. In addition, the highlight effects may be configurable. That is, the user may, depending on his/her preference or the status of the device, add new effect, remove existing effect, and/or configure various related parameters.

[0052] Next, at step S225, the at least one new segment obtained at step S215 is recognized, so as to output one or more characters. The method 200 then terminates. As described above, any current existing or later developed recognition method may be employed to convert the segments into characters, and the scope of invention is not limited in this regard. Moreover, step S225 is optional, because in some applications the segment representing the original pen trace of the user may be stored and used directly without recognition. An example of such application is writing exercise application.

[0053] FIG. 5 shows a graphical user interface (GUI) 500 that is suitable for use in practicing embodiments of the present invention. According to some embodiments of the present invention, the GUI 500 may be used with the method 200, and may be generated and rendered, for example, by the apparatus that will be described below with reference to FIGs, 6 and 7.

[0054] As shown, the GUI 500 includes an editor area 505, a candidate character area 510, a segmentation result area 515, a writing area 520, and a control area 525. In operation, the user may write a plurality of character in an overlapped manner within the writing area 520. Then segmentation and recognition of the user's handwriting may be performed. The final characters may be displayed in the editor area 505, and a candidate list may be displayed in the candidate character area 510. Specifically, according to embodiments of the present invention, the resulting one or more segments may be displayed in the segmentation result area 515. When, for example, the user finds a recognition error caused by faulty segmentation, he/she may interact with the segmentation result area 515 in the way described above, in order to correct errors (if any) of the segmentation. After correction, when the user ensures that the segmentation is error free, the segments may be submitted to be recognized. Then the editor area 505 and the candidate character area 510 may be updated accordingly. Alternatively or additionally, the user may also be enabled to check and correct the segmentation results as displayed in the segmentation result area 515 before executing recognition. In the control area 525, one or more controls like buttons, soft keys, or any other appropriate controls may be provided, by which the user may control the process of overlapped handwriting.

[0055] It should be understood that the GUI 500 as shown in FIG. 5 is merely an illustrative example. The GUI suitable for use in practicing embodiments of the present invention may contain less, additional, and/or different components. Alternatively or additionally, the layout or arrangement of components on the GUIs in other embodiments may be varied.

[0056] FIG. 6 shows a simplified block diagram of an apparatus 600 for use in overlapped handwriting according to embodiments of the present invention. As described above, the apparatus 600 may generate and render a GUI like the GUI 500 as shown in FIG. 5. As shown in FIG. 6, the apparatus 600 includes segmentation displaying means 605, correction requesting means 610, segmentation modifying means 615, modification highlighting means 620 (optional), and character recognizing means 625 (optional).

[0057] The segmentation displaying means 605 may be configured to cause the display of at least one segment obtained by a segmentation of overlapped handwriting. The correction requesting means 610 may be configured to receive a request for correcting the segmentation. The segmentation modifying means 615 may be configured to modify, in response to the received request, the at least one segment to obtain at least one new segment.

[0058] Specifically, the segmentation modifying means 615 may be configured to perform one or more of the following: in response to dragging a first segment to a predefined proximity of a second segment, combing the first and second segments into one segment; in response to a predefined gesture on a portion of a first segment, performing a re-segmentation of the first segment to obtain two or more segments; in response to dragging a portion of a first segment to a predefined area, removing the portion from the first segment, and creating a new segment based upon the portion; in response to dragging a first segment to a position in segment sequence, altering order of the first segment in the sequence; in response to dragging a portion of a first segment to a predefined proximity of a second segment, removing the portion from the first segment, and adding the portion to the second segment; and in response to adding a portion to a first segment, generating a new segment that is composed of the first segment and the added portion to replace the first segment.

[0059] During the operation of the segmentation modifying means 615, modification highlighting means 620 may be configured to highlight portion or segment being operated. The character recognizing means 625 may be configured to perform recognition in collaboration with the segmentation modifying means 615, in order to convert the corrected segments into editable characters.

[0060] It should be understood that the apparatus 600 may be embodied in hardware or special purpose circuits, software, logic or any combination thereof. Accordingly, the various means of the apparatus 600 may be embodied as software modules, and/or hardware blocks, units, or components. [0061] Embodiments of the above described process or method (e.g., method

200 as shown in FIG.2) may be implemented by any suitable apparatus. FIG. 7 shows a simplified block diagram of an apparatus 700 that is suitable for use in practicing embodiments of the present invention. In some embodiments, the apparatus 700 may be embodied as a mobile terminal. It should be understood, however, that a mobile terminal is merely illustrative of one type of apparatus that would benefit from embodiments of the present invention and, therefore, should not be taken to limit the scope of embodiments of the present invention.

[0062] The apparatus 700 includes an antenna 712 (or multiple antennae) in operable communication with a transmitter 714 and a receiver 716. The apparatus 700 further includes at least one processor or controller 720. It should be understood that the controller 720 includes circuitry required for implementing overlapped handwriting and logic functions of the apparatus 700. For example, the controller 720 may be comprised of a digital signal processor device, a microprocessor device, and various analog to digital converters, digital to analog converters, and other support circuits. Control and information processing functions of the apparatus 700 are allocated between these devices according to their respective capabilities. [0063] The apparatus 700 also comprises a user interface including an output device such as an earphone or speaker 724, a ringer 722, a microphone 726, a display 728, and a user input interface, all of which are coupled to the controller 720, The user input interface may include any of a number of devices allowing the apparatus 700 to receive data, such as, a keypad or keyboard 730, a writing device 7 1, or other input devices. For example, the writing device 731 may include, but not limited to, a touch screen, a proximity screen, a writing pad, etc. Specifically, the writing device 731 is capable of receiving overlapped handwriting input by a user. As such, the writing device 731 includes all hardware, software and/or firmware necessary for supporting overlapped handwriting.

[0064] Additionally or alternatively, the apparatus 700 may include an interface device such as a joystick or other user input interface. The apparatus 700 further includes a battery 734, such as a vibrating battery pack, for powering various circuits that are required to operate the apparatus 700, as well as optionally providing mechanical vibration as a detectable output.

[0065] In some embodiments, the apparatus 700 includes a media capturing element, such as a camera, video and/or audio module, in communication with the controller 720. The media capturing element may be any means for capturing an image, video and/or audio for storage, display or transmission. For example, in an exemplary embodiment in which the media capturing element is a camera module 736, the camera module 736 may include a digital camera capable of forming a digital image file from a captured image. When embodied as a mobile terminal, the apparatus 700 may further include a universal identity module (UIM) 738. The UIM 738 is typically a memory device having a processor built in. The UIM 738 may include, for example, a subscriber identity module (SIM), a universal integrated circuit card (UICC), a universal subscriber identity module (USIM), a removable user identity module (R-UIM), etc. The UIM 738 typically stores information elements related to ;a subscriber.

[0066] The apparatus 700 may be equipped with at least one memory. For example, the apparatus 700 may include volatile memory 740, such as volatile Random Access Memory (RAM) including a cache area for the temporary storage of data. The apparatus 700 may also include other non-volatile memory 742, which can be embedded and/or may be removable. The non-volatile memory 742 can additionally or alternatively comprise an EEPROM, flash memory or the like. The memories can store any of a number of pieces of information, program, and data, used by the apparatus 700 to implement the functions of the apparatus 700. For example, the memories may store program of computer executable code, which may be configured, with the controller 720, to cause the apparatus 700 to at least perform the steps of method 200 as described above.

[0067] Embodiments of the present invention may also be implemented as a computer program product, comprising at least one computer readable storage medium having a computer readable program code portion stored thereon. In such embodiments, the computer readable program code portion comprises at least codes for performing the steps of method 200 as described above.

[0068] For the purpose of illustrating the principle of the present invention, some specific embodiments thereof have been described. As described above, embodiments of the present invention provide a method, apparatus, and computer program product for use in overlapping handwriting, and in particular, for enabling user to correct segmentation error interactively. According to embodiments of the present invention, the segments obtained by overlapped handwriting as input by a user may be displayed in a modifiable or editable manner. In particular, the positional information and temporal information (writing order) for each portion (or stroke) contained within each segment is available, such that the user may select any portion of any segment and modify the portion as desired, for example, using a set of predefined gestures. As a result, the user is provided with an efficient and convenient way in which the overlapped handwriting recognition results may be corrected. Alternatively or additionally, the user may also choose to check and correct the segmentation results before executing recognition, and therefore ensure that the segments to be recognized are error-free. In effect, the correction of segmentation may be performed before, along with, or after the character recognition, and the scope of invention is not limited in this regard.

[0069] Experiments have indicated that with embodiments of the present invention, the efficiency of correcting recognition errors for overlapped handwriting caused by segmentation errors is increased compared with prior art overlapping handwriting solutions. Additionally, the user experience and satisfaction are improved, since user need not have to re-write one or more characters in the case of recognition error.

[0070] In general, the various exemplary embodiments may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although the invention is not limited thereto. While various aspects of the exemplary embodiments of this invention may be illustrated and described as block diagrams, flowcharts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

[0071] The various blocks shown in FIG. 2 may be viewed as method steps, and/or as operations that result from operation of computer program code, and/or as a plurality of coupled logic circuit elements constructed to carry out the associated function(s). At least some aspects of the exemplary embodiments of the inventions may be practiced in various components such as integrated circuit chips and modules, and that the exemplary embodiments of this invention may be realized in an apparatus that is embodied as an integrated circuit. The integrated circuit, or circuits, may comprise circuitry (as well as possibly firmware) for embodying at least one or more of a data processor or data processors, a digital signal processor or processors, baseband circuitry and radio frequency circuitry that are configurable so as to operate in accordance with the exemplary embodiments of this invention.

[0072] Various modifications, adaptations to the foregoing exemplary embodiments of this invention may become apparent to those skilled in the relevant arts in view of the foregoing description, when read in conjunction with the accompanying drawings. Any and all modifications will still fall within the scope of the non-limiting and exemplary embodiments of this invention. Furthermore, other embodiments of the inventions set forth herein will come to mind to one skilled in the art to which these embodiments of the invention pertain having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. [0073] Therefore, it is to be understood that the embodiments of the invention are not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terms are used herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

WE CLAIM:

1. A method for use in overlapped handwriting, comprising:

causing display of at least one segment obtained by a segmentation of overlapped handwriting;

receiving a request for correcting the segmentation; and

modifying, in response to the received request, the at least one segment to obtain at least one new segment.

2. The method according to Claim 1, wherein the request is received via a gesture by a user.

3. The method according to Claim 1, wherein the modification comprises: in response to dragging a first segment to a predefined proximity of a second segment, combining the first and second segments into one segment.

4. The method according to Claim 1 , wherein the modification comprises: in response to a predefined gesture on a portion of a first segment, performing a re-segmentation of the first segment to obtain two or more segments.

5. The method according to Claim 1, wherein the modification comprises: in response to dragging a portion of a first segment to a predefined area, removing the portion from the first segment, and creating a new segment based upon the portion.

6. The method according to Claim 1, wherein the modification comprises: in response to dragging a first segment to a position in segment sequence, altering order of the first segment in the sequence.

7. The method according to Claim 1 , wherein the modification comprises: in response to dragging a portion of a first segment to a predefined proximity of a second segment, removing the portion from the first segment, and adding the portion to the second segment.

8. The method according to Claim Ί, wherein the portion contains the last one or more strokes of the first segment, and the first segment is immediately before the second segment.

9. The method according to Claim 7, wherein the portion contains the first one or more strokes of the first segment, and the first segment is immediately next to the second segment.

10. The method according to Claim 1, wherein the modification comprises: in response to adding a portion to a first segment, generating a new segment that is composed of the first segment and the added portion to replace the first segment;

1 1. The method according to any of Claims 1-10, further comprising:

highlighting portion or segment being operated during the modification,

12. The method according to any of Claims 1-10, further comprising:

recognizing the obtained at least one new segment into at least one character.

13. An apparatus for use in overlapped handwriting, comprising: ■ at least one processor; and

at least one memory storing program of computer executable code;

the at least one memory and the computer executable code configured, with the at least one processor, to cause the apparatus to at least:

cause display of at least one segment obtained by a segmentation of overlapped handwriting;

receive a request for correcting the segmentation; and

modify, in response to the received request, the at least one segment to obtain at least one new segment.

14. The apparatus according to Claim 13, wherein the at least one memory and the computer program code are configured with the at least one processor to cause the apparatus to receive the request via a gesture by a user,

15. The apparatus according to Claim 13, wherein the modification comprises: in response to dragging a first segment to a predefined proximity of a second segment, combining the first and second segments into one segment.

16. The apparatus according to Claim 13, wherein the modification comprises: in response to a predefined gesture on a portion of a first segment, performing a re-segmentation of the first segment to obtain two or more segments.

17. The apparatus according to Claim 13, wherein the modification comprises: in response to dragging a portion of a first segment to a predefined area, removing the portion from the first segment, and creating a new segment based upon the portion.

18. The apparatus according to Claim 13, wherein the modification comprises: in response to dragging a first segment to a position in segment sequence, altering order of the first segment in the sequence.

19. The apparatus according to Claim 13, wherein the modification comprises: in response to dragging a portion of a first segment to a predefined proximity of a second segment, removing the portion from the first segment, and adding the portion to the second segment.

20. The apparatus according to Claim 19, wherein the portion contains the last one or more strokes of the first segment, and the first segment is immediately before the second segment.

21. The apparatus according to Claim 19, wherein the portion contains the first one or more strokes of the first segment, and the first segment is immediately next to the second segment,

22. The apparatus according to Claim 13, wherein the modification comprises: in response to adding a portion to a first segment, generating a new segment that is composed of the first segment and the added portion to replace the first segment.

23. The apparatus according to any of Claims 13-22, wherein the at least one memory and the computer program code are configured with the at least one processor to cause the apparatus further to at least:

highlight portion or segment being operated during the modification

24. The apparatus according to any of Claims 13-22, wherein the at least one memory and the computer program code are configured with the at least one processor to cause the apparatus further to at least:

recognize the obtained at least one new segment into at least one character,

25. A computer program product, comprising at least one computer readable storage medium having a computer readable program code portion stored thereon, the computer readable program code portion comprising:

code for causing display of at least one segment obtained by a segmentation of overlapped handwriting;

code for receiving a request for correcting the segmentation; and

code for modifying, in response to the received request, the at least one segment to obtain at least one new segment.

26. The computer program product according to Claim 25, wherein the code for receiving the request comprises code for receiving the request via a gesture by a user.

27. The computer program product according to Claim 25, wherein the code for modifying comprises:

code for, in response to dragging a first segment to a predefined proximity of a second segment, combing the first and second segments into one segment.

28. The computer program product according to Claim 25, wherein the code for modifying comprises:

code for, in response to a predefined gesture on a portion of a first segment, performing a re- segmentation of the first segment to obtain two or more segments.

29. The computer program product according to Claim 25, wherein the code for modifying comprises:

code for, in response to dragging a portion of a first segment to a predefined area, removing the portion from the first segment, and creating a new segment based upon the portion.

30. The computer program product according to Claim 25, wherein the code for modifying comprises:

code for, in response to dragging a first segment to a position in segment sequence, altering order of the first segment in the sequence.

31. The computer program product according to Claim 25, wherein the code for modifying comprises:

code for, in response to dragging a portion of a first segment to a predefined proximity of a second segment, removing the portion from the first segment, and adding the portion to the second segment.

32. The computer program product according to Claim 31, wherein the portion contains the last one or more strokes of the first segment, and the first segment is immediately before the second segment.

33. The computer program product according to Claim 31, wherein the portion contains the first one or more strokes of the first segment, and the first segment is immediately next to the second segment.

34. The computer program product according to Claim 25, wherein the code for modifying comprises:

code for, in response to adding a portion to a first segment, generating a new segment that is composed of the first segment and the added portion to replace the first segment.

35. The computer program product according to any of Claims 25-34, wherein the computer readable program code portion further comprises:

code for highlighting portion or segment being operated during the modification.

36. The computer program product according to any of Claims 25-34, wherein the computer readable program code portion further comprises:

code for recognizing the obtained at least one new segment into at least one character.

37. An apparatus for use in overlapped handwriting, comprising:

means for causing display of at least one segment obtained by a segmentation of overlapped handwriting;

means for receiving a request for correcting the segmentation; and

means for modifying, in response to the received request, the at least one segment to obtain at least one new segment.