JP4741977B2 - Handwritten character processing apparatus and handwritten character processing method - Google Patents

Description

  The present invention relates to a handwritten character processing apparatus and a handwritten character processing method suitable for processing handwritten characters input at an angle.

  In recent years, electronic conference systems using computer systems have become widespread. Display devices include a plasma display, a rear projection display, a front projector, and the like, and screens are becoming larger. In these display devices, various image sources such as presentations, various materials, video conferences, videos, movies, and the like are displayed, and the scenes that are used not only for offices but also for educational purposes are increasing.

  In particular, at educational sites and various meetings, along with displaying these image information, teachers and conference organizers draw directly on the screen using handwriting using an input device such as a digitizer pen when conducting classes and meetings. A display device having an electronic whiteboard function for performing the above has become useful. As a configuration of an electronic whiteboard system, a configuration including the above-described large display device, a coordinate input device such as a touch pad or an optical digitizer, and a personal computer (hereinafter also referred to as a PC) is well known. The PC performs control such as managing drawing information input from the coordinate input device and displaying it on a display device.

  In addition, electronic notebooks and PDAs (Personal Digital Assistants), which have become popular in recent years, use tablets as input devices to convert handwritten input strokes as text data and record them. Is also common.

  Since the electronic whiteboard performs drawing with a stroke using a pen, there is also a form in which drawing data drawn on the electronic whiteboard is recognized.

  In these apparatuses, the following processing is performed. First, a character recognition frame area for character recognition is displayed on the electronic whiteboard, or a character recognition frame cutout process is performed to cut out a drawing portion that becomes one character from the drawn drawing data as a rectangle. Next, a process for applying the stroke data drawn in the character recognition area or in the extracted character recognition frame to the character recognition engine is performed. Then, the recognized character data is displayed on the drawing surface. In addition, there is a type in which drawing data drawn on a whiteboard is collectively stored and applied to an OCR (Optical Character Reader) for character recognition processing.

  A proposal has also been made to improve the efficiency of document creation work by converting text / documents described on a whiteboard into text data (Patent Document 1).

  In this way, by combining the electronic whiteboard and character recognition, it is possible to improve the efficiency of the work at the time of creating the written work of the board and the minutes of the meeting.

JP-A-9-130521

  However, when writing on an electronic whiteboard, unlike the desktop, it may be drawn standing in front of the board. When writing to the corner of the board or annotation, the text is tilted. May be. For this reason, it is difficult to obtain a sufficient recognition rate by the conventional character recognition process using a rectangle. In addition, since the electronic whiteboard has a large board surface and can be drawn anywhere on the entire surface instead of a divided frame called a character recognition area, there is a variation in character size and character size.

  Therefore, in view of the above-described problems, the present invention can improve the character recognition rate of handwritten characters input at an angle, and preferably performs display such as correction of character inclination and character spacing alignment at that time. It is an object of the present invention to provide a handwritten character processing apparatus capable of performing the above.

  As a result of intensive studies to solve the above problems, the present inventor has come up with various aspects of the invention described below.

The handwritten character processing apparatus according to the present invention cuts out a character string input by handwriting for each character, and sets a character frame consisting of horizontal and vertical line segments with respect to the display screen for each cut out character. detecting means and the the reference line creating means for creating a reference line indicating the tilt of the character string based on a line segment connecting the points each other as a reference for each character frame for each character, the display screen and the reference line A tilt determining unit that calculates a rotation angle of the character frame from the predetermined direction based on an angle with respect to the predetermined direction, and a correction that tilts the character frame based on the rotation angle calculated by the tilt determining unit. Character frame correction means for performing , and recognition means for recognizing characters for each character frame corrected by the character frame correction means .

The handwritten character processing method according to the present invention cuts out a character string input by handwriting for each character, and sets a character frame consisting of horizontal and vertical line segments with respect to the display screen for each cut out character. out step a, the a reference line creating step of creating a reference line indicating the tilt of the character string based on a line segment connecting the reference becomes a point each other in the character frame for each character, the display screen and the reference line An inclination determination step for calculating a rotation angle of the character frame from the predetermined direction from an angle with respect to the predetermined direction, and a correction for inclining the character frame based on the rotation angle calculated in the inclination determination step A character frame correcting step; and a recognition step of recognizing a character for each character frame corrected in the character frame correcting step .

A program for handwritten character processing according to the present invention cuts out a character string input by handwriting into a computer for each character, and sets a character frame made up of horizontal and vertical line segments with respect to the display screen for each cut out character a character frame cutout to set a reference line creating procedure for creating a reference line indicating the tilt of the character string based on a line segment connecting the reference becomes a point each other in the character frame for each character, the reference from the angle between the predetermined direction with respect to the line with the display screen, the slope determination procedure that calculates the rotation angle of the character frame from the predetermined direction, based on the rotation angle calculated in the tilt determination procedure, the character frame A character frame correction procedure for performing a correction for tilting and a recognition procedure for recognizing a character for each character frame corrected in the character frame correction procedure .

  According to the present invention, a handwritten character can be properly recognized even when a handwritten character is input with an inclination. Furthermore, it is possible to correct the position, size, and spacing of the character frame and perform appropriate shaped display.

  Hereinafter, embodiments of the present invention will be specifically described with reference to the accompanying drawings.

(First embodiment)
First, a first embodiment of the present invention will be described. FIG. 1 is a diagram showing a handwritten character processing apparatus according to the first embodiment of the present invention. FIG. 1 is a configuration diagram showing the whole handwritten character processing apparatus according to the first embodiment of the present invention. A display device 101 and an input device 102 are connected to the handwritten character processing device 120.

  The display device 101 is a relatively large display for displaying drawing data or the like, and is connected to a handwritten character processing device 120 formed of a computer or the like. And the information output from the display control part 107 in the handwritten character processing apparatus 120 is displayed.

  The input device 102 is an optical digitizer that is a pen-type pointing device. The input device 102 includes firmware for outputting the detected coordinate information to the handwritten character processing device 120.

  The handwritten character processing device 120 includes a display control unit 107 that controls an image displayed on the display device 101. Further, a control unit 103, a character frame cutout unit 104, a reference line creation unit 105, an inclination determination unit 106, a character frame correction unit 108, a character recognition unit 109, and a storage unit 110 are included. The control unit 103 includes, for example, a CPU, an MPU, and the like, and controls each unit of the handwritten character processing apparatus 120. The character frame cutout unit 104 cuts the input handwritten drawing data into rectangles in units of one character. The reference line creation unit 105 creates a reference line that connects the centers of the character frames cut out by the character frame cutout unit 104 and indicates the inclination of the character string. The inclination determination unit 106 calculates the rotation angle of the character frame based on the reference line created by the reference line creation unit 105. The character frame correction unit 108 corrects the inclination, position, size, interval, and the like of the character frame cut out by the character cutout unit 104. The character recognition unit 109 performs character recognition from a stroke or an image in the character frame. The storage unit 110 includes an external storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), and an HDD, and stores a dictionary for character recognition, a processing program executed by the control unit 103, and the like. .

  Next, the operation of the handwritten character processing device 120 configured as described above, that is, the contents of processing by the handwritten character processing device 120 will be described. FIG. 2 is a flowchart showing the operation of the handwritten character processing apparatus 120 according to the first embodiment.

  First, in step S201, it is determined whether there has been a request for correction of handwritten characters. This determination is performed as follows, for example.

  In the present embodiment, the stroke from when the pen is lowered with respect to the board surface to one up is defined as one stroke, and a character is drawn by one or more strokes. And it is thought that the timing which does not disturb the thought of the user who is drawing is appropriate as a correction timing of a handwritten character. Therefore, when the stroke drawing interval exceeds a certain time, it is determined that there is a request for correction of handwritten characters, and when the stroke start coordinates are more than a certain distance away from the previous input stroke coordinates, And a determination that there is a correction request. The former is based on the idea that the user who draws finishes drawing a character string when the stroke drawing interval exceeds a certain time. The latter is based on the idea that input of a line feed or another character string starts when the stroke start coordinate is more than a certain distance from the previous input stroke coordinate. Alternatively, it may be determined that there has been a request for correction of handwritten characters at a timing such as when a new whiteboard page is displayed or when the whiteboard program ends. Furthermore, a graphical user interface such as a correction start button may be prepared on the screen, and it may be determined that there has been a request for correction of handwritten characters at the timing indicated by the user.

  If there is a handwritten character correction request in step S201, the process proceeds to step S202, and the character frame cutout unit 104 performs character frame cutout processing. The character frame cut-out process is a process of cutting out a drawing portion that becomes one character from the drawn drawing data as a rectangle. In this process, for example, the drawing surface is scanned, and a portion where a stroke exists on the scanning line and a portion which does not exist are partitioned. Then, an algorithm is adopted that determines a space representing a delimiter between characters if the width of a portion where no stroke exists is greater than or equal to a certain value, and if the width is less than that, determines a gap between the side and the eyelid.

  FIG. 3 is a diagram illustrating an example of a screen display of the display device 101 when the character frame cut-out process is performed. As shown in FIG. 3, after the character frame cutout process, the drawn characters are divided by a rectangular character cutout frame 301. In the example shown in FIG. 3, characters and character strings are drawn with an inclination, but the character frame is a rectangle composed of horizontal and vertical line segments. For this reason, if character recognition processing is performed in the character frame as it is, erroneous recognition increases.

  Therefore, in the present embodiment, correction processing for the inclination of the character frame is performed. First, in step S203, the reference line creation unit 105 creates a reference line for obtaining the inclination of the character string as shown in FIG. For example, as shown in FIG. 4, the centers of the first character frame and the last character frame of the character string are connected to each other by line segments from the character frame obtained in step S202. Next, the inclination determination unit 106 obtains an angle α between the line segment and the horizontal line.

  Next, in step S204, the character frame correction unit 108 performs character frame cut-out processing from a direction inclined by an angle α from the horizontal direction. That is, the character frame is corrected. FIG. 5 is a diagram illustrating a character frame when the character frame cut-out process is performed from the angle α direction.

  Thereafter, in step S205, the character frame recognition unit 109 performs character recognition processing in units of character frames. FIG. 6 is a diagram illustrating an example of a screen display when character recognition is performed. As shown in FIG. 6, in the present embodiment, the process of tilting the display of text data of the character recognition result in accordance with the frame is also performed.

  Subsequently, in step S206, the character frame correction unit 108 corrects the position and size of the character. This correction includes, for example, the following three stages of processing. In the first stage, as shown in FIG. 7, the position is corrected so that the center of the character frame is on the reference line, and the vertical blur of the character string is corrected. In the second stage, as shown in FIG. 8, the position of the center of the character frame is corrected so as to be equally spaced on the reference line so that the characters are equally spaced. Then, in the third stage, as shown in FIG. 9, the character spacing is corrected and the character size irregularity is corrected. That is, the character size is aligned with the first character (“clear”) of the character string, and the characters are arranged at equal intervals.

  According to such a first embodiment, it is possible to improve the character recognition rate of handwritten characters input at an angle. In addition, it is possible to correct the inclination of the character string at that time and to perform shaping display such as character spacing alignment.

  Although FIG. 7, FIG. 8 and FIG. 9 show text data, drawing data may be displayed. In this case, the processing results corresponding to FIGS. 7, 8, and 9 are as shown in FIGS. 10, 11, and 12, respectively. In this embodiment, since only the converted text data code is stored to reduce the amount of stored data and to be used for text search, text data is displayed. However, the display form is not limited.

  In the flowchart shown in FIG. 2, the character frame is corrected after the character recognition. However, the character recognition process may be performed after the character frame correction.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. The configuration of the second embodiment is the same as that of the first embodiment, but the operation is different. FIG. 13 is a flowchart showing the operation of the handwritten character processing apparatus 120 according to the second embodiment.

  First, in step S1301, it is determined whether there has been a request for correction of handwritten characters. This determination is performed, for example, in the same manner as in the first embodiment.

  If there is a handwritten character correction request in step S1301, the process proceeds to step S1302, and the character frame cutout unit 104 performs character frame cutout processing.

  FIG. 14 is a diagram illustrating an example of a screen display of the display device 101 when the character frame cutout process is performed. As shown in FIG. 14, similarly to the first embodiment, after the character frame cutout process, the drawn characters are separated by a rectangular character cutout frame 1401 by the character frame cutout process.

  Next, in step S1303, the reference line creation unit 105 creates a reference line candidate for obtaining the inclination of the character string, and displays this candidate via the display control unit 107. On the screen, a drawing character string, a plurality of reference lines or a corrected character string, and input means such as a button for selecting the reference line are displayed.

  Character strings drawn by handwriting on the electronic whiteboard are not only arranged in a straight line, but may be arranged in a curved line (for example, an arc) by chance. In addition, when a circular figure is drawn and a character string is written along the circumference, the drawing person may intentionally draw a curved line. As described above, in addition to the case where the drawing person desires to shape the character string into a straight line shape, there is a case where he desires to shape the character string into a curved line shape. Therefore, in the second embodiment, a plurality of reference line candidates including a curved one are displayed, and the drawing person is allowed to select a desired reference line. However, it is possible to omit such a selection procedure by setting an initial value or the like for selecting a reference line in advance.

  Here, reference line candidates will be described. FIG. 15 is a diagram illustrating an example of a method for creating a linear reference line candidate from the character string illustrated in FIG. 14.

  In this method, the average of the inclinations of the character frames before and after the target character frame is obtained, and the inclination of the character frame when there are no preceding and following characters is obtained. Then, these plural types of inclination straight lines are used as reference line candidates. For example, as shown in FIG. 15, the centers of the character frames are connected by line segments, and angles (α1, β1, γ1) formed by the line segments and the horizontal lines are obtained. As a result, the inclination of the character frame 1502 is (α1 + β1) / 2, and the inclination of the character frame 1503 is (β1 + γ1) / 2. Further, the inclination of the character frame 1501 is α1, and the inclination of the character frame 1504 is γ1. As a result, four types of inclination reference line candidates are created.

  A curved reference line candidate is also created. FIG. 16 is a diagram illustrating an example of a method for creating a curved reference line candidate from the character string illustrated in FIG. 14.

  In this method, the centers of the character frames are connected by a smooth curve 1601. In the example shown in FIG. 16, the curve 1601 is a spline curve with the center of each character frame as a control point, but a Bezier curve with the center of the first character frame and the center of the end character frame of the character string as control points, etc. It is good.

  In this way, a total of five or more types of reference line candidates are created from the character string shown in FIG.

  In step S1304, the user is allowed to select one reference line from the plurality of reference lines displayed in step S1303. In step S1305, the character frame correction unit 108 determines the inclination of the character clipping frame based on the selected reference line. Make corrections. FIG. 17 is a diagram illustrating a result of correcting the character cutout frame when the reference line illustrated in FIG. 16 is selected. When the curved reference line shown in FIG. 16 is selected, the character frame correction unit 108 draws a vertical line from the center of each character frame toward the reference line, and sets each character cut frame to be perpendicular to the vertical line. Correct the tilt.

  In step S1306, as in the first embodiment, the character recognition unit 109 performs character recognition processing in units of corrected character cutout frames. In step S1307, the recognition result is obtained via the display control unit 107. Displays text data.

  According to the second embodiment, it is possible to correct a smooth inclination of a character string having a slope or a handwritten character string drawn in a curved line and to improve a character recognition rate.

  In these embodiments, the display device 101 may be a liquid crystal projector or a large-screen liquid crystal panel or PDP (Plasma Display Panel). Further, the display device 101 and the handwritten character processing device 120 may be integrated. The character frame cutout unit 104, the reference line creation unit 105, the inclination determination unit 106, the display control unit 107, the character frame correction unit 108, and the character recognition unit 109 are configured using, for example, a CPU and software. A card or the like may be used.

  The object of the present invention can also be achieved by supplying a software program that implements the functions of the above-described embodiments to a computer or CPU, and the computer or CPU reads and executes the supplied program. .

  The program is supplied, for example, by downloading it directly from a storage medium storing the program or from another computer or database (not shown) connected to the Internet, a commercial network, a local area network, or the like.

  Further, the above-described program only needs to be able to realize the functions of the above-described embodiments by a computer, and the form may include object code, a program executed by an interpreter, script data supplied to the OS, and the like. Good.

  Furthermore, the object of the present invention is also provided by supplying a computer with a storage medium that records a software program for realizing the functions of the above-described embodiments, and reading or executing the program stored in the storage medium by the computer or CPU. Is achieved.

  In the above-described embodiment, the program is stored in the ROM. However, the present invention is not limited to this. Examples of storage media for supplying the program include RAM, NV-RAM, flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, MO, CD-ROM, CD-RW, DVD (DVD-ROM, DVD- R). Moreover, you may use what can memorize | store the said programs, such as a magnetic tape, a non-volatile memory card, and other ROM as a storage medium.

  That is, the embodiment of the present invention can be realized by, for example, a computer executing a program. Also, means for supplying a program to a computer, for example, a computer-readable recording medium such as a CD-ROM recording such a program, or a transmission medium such as the Internet for transmitting such a program is also applied as an embodiment of the present invention. Can do. The above program can also be applied as an embodiment of the present invention. The above program, recording medium, transmission medium, and program product are included in the scope of the present invention.

It is a figure which shows the handwritten character processing apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the handwritten character processing apparatus 120 which concerns on 1st Embodiment. 6 is a diagram illustrating an example of a screen display of the display device 101 when a character frame cutout process is performed in the first embodiment. FIG. It is a figure which shows the inclination (alpha) of a reference line and a character string. It is a figure which shows the character frame at the time of performing a character frame cut-out process from the angle (alpha) direction. It is a figure which shows an example of the screen display at the time of performing character recognition. It is a figure which shows the result of having corrected the position so that the center of a character frame may become on a reference line, and correcting the up-and-down blur of a character string. It is a figure which shows the result of having corrected the position of the center of a character frame so that it may become equal intervals on a reference line so that a character may become equal intervals. It is a figure which shows the result of having corrected the correction of the character space | interval, and the unevenness of the size of a character. It is a figure which shows the drawing data corresponding to FIG. It is a figure which shows the drawing data corresponding to FIG. It is a figure which shows the drawing data corresponding to FIG. It is a flowchart which shows operation | movement of the handwritten character processing apparatus 120 which concerns on 2nd Embodiment. In 2nd Embodiment, it is a figure which shows an example of the screen display of the display apparatus 101 at the time of performing a character frame cutout process. It is a figure which shows an example of the method of producing a linear reference line candidate from the character string shown in FIG. It is a figure which shows an example of the method of producing a curvilinear reference line candidate from the character string shown in FIG. It is a figure which shows the result of having corrected the character cutting frame when the reference line shown in FIG. 16 is selected.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101: Display apparatus 102: Input device 103: Control part 104: Character frame extraction part 105: Reference line preparation part 106: Inclination determination part 107: Display control part 108: Character frame correction part 109: Character recognition part 110: Storage part 301, 1401: Character cutout frame 1501, 1502, 1503, 1504: Character frame 1601: Curve

Claims (11)

A character frame cutting means for cutting out a character string inputted by hand for each character, and setting a character frame composed of horizontal and vertical line segments with respect to the display screen for each cut out character;
And the reference line creating means for creating a reference line indicating the tilt of the character string based on a line segment connecting the points each other as a reference for each character frame for each character,
Inclination determining means for calculating a rotation angle of the character frame from the predetermined direction from an angle between the reference line and the predetermined direction with respect to the display screen ;
A character frame correcting means for correcting the character frame to be tilted based on the rotation angle calculated by the tilt determining means ;
Recognizing means for recognizing characters for each character frame corrected by the character frame correcting means;
A handwritten character processing apparatus comprising: The reference line creating means creates a reference line from each character frame set by the character frame cutting means by connecting the center of the first character frame and the last character frame of the character string with a line segment. The handwritten character processing apparatus according to claim 1. The character frame correcting means is inclined by the rotation angle calculated by the inclination determining means around a specific vertex of each character frame set by the character frame cutting means when correcting the inclination of the character frame. handwriting processing apparatus according to recreate in claim 1 or 2, wherein the character frame was. The reference line creating means creates a curve connecting the centers of character frames as the reference line,
The character frame correcting means draws a perpendicular line from the center of the character frame toward the curve for each character frame, and corrects the perpendicular line and the character frame to be at right angles. The handwritten character processing apparatus according to 1. The character frame correcting means further arranges the character frame positions so that the character strings are equally spaced between the first character and the last character of the character string, and characters are arranged in accordance with the arranged character frames. The handwritten character processing apparatus according to any one of claims 1 to 4 , wherein the correction is performed so that the size of one character in a character string is aligned with the size of another character. . A character frame cutting step for cutting out a character string inputted by hand for each character, and setting a character frame composed of horizontal and vertical line segments with respect to the display screen for each cut out character;
A reference line creating step of creating a reference line indicating the tilt of the character string based on a line segment connecting the reference becomes a point each other in the character frame for each character,
An inclination determination step of calculating a rotation angle of the character frame from the predetermined direction from an angle between the reference line and the predetermined direction with respect to the display screen ;
A character frame correction step for correcting the tilt of the character frame based on the rotation angle calculated in the tilt determination step ;
A recognition step for recognizing characters for each character frame modified in the character frame modification step;
A handwritten character processing method characterized by comprising: In the reference line creation step, a reference line is created by connecting the center of the first character frame and the end character frame of the character string with a line segment from each character frame set in the character frame cutting step. The handwritten character processing method according to claim 6. In correcting the inclination of the character frame in the character frame correction step, the character frame is tilted by the rotation angle calculated by the inclination determination step with a specific vertex of each character frame set by the character frame cutting step as a center . handwritten character processing method according to claim 6 or 7, characterized in that re-create the character frame. In the reference line creation step, create a curve connecting the center of the character frame as the reference line,
In the character frame correction step, for each of the character frames, a perpendicular line is drawn from the center of the character frame toward the curve , and correction is performed so that the perpendicular line and the character frame are perpendicular to each other. 6. The handwritten character processing method according to 6. In the character frame correction step, further, the character frames are arranged so that the character strings are equally spaced between the first character and the last character of the character string, and the characters are arranged in accordance with the arranged character frames. 10. The handwritten character processing method according to claim 6 , wherein correction is performed so that the size of one character in a character string is aligned with the size of another character. . On the computer,
A character frame cutting procedure for cutting out a character string inputted by hand for each character and setting a character frame made up of horizontal and vertical line segments with respect to the display screen for each cut out character;
And the reference line creation procedure for creating a reference line indicating the tilt of the character string based on a line segment connecting the reference becomes a point each other in the character frame for each character,
An inclination determination procedure for calculating a rotation angle of the character frame from the predetermined direction from an angle between the reference line and the predetermined direction with respect to the display screen ;
A character frame correction procedure for correcting the character frame to be tilted based on the rotation angle calculated in the tilt determination procedure ;
A recognition procedure for recognizing characters for each character frame modified in the character frame modification procedure;
A program for handwritten character processing, characterized in that

Images