JPH06119496A - Handwritten information processor - Google Patents

Abstract

PURPOSE:To unnecessitate the switching operation of an input mode and the group classification setting of an input area by providing an area information setting means for setting the character area and the graphic area corresponding to the inputted condition of a document to be edited. CONSTITUTION:This device is provided with an area information setting means 4 for setting the character area and the graphic area corresponding to the inputted condition of the document to be edited. Therefore, even when neither character area nor graphic area is set, the classification of the set of this handwritten stroke can be judged corresponding to which area the inputted handwritten stroke set is positioned in. On the other hand, a write position condition judging means 10 judges position relation between the position, where the stroke set is written, and the character/graphic area, and discriminates to which recognizing object group this stroke set belongs by calculating and comparing the degree of a certanity to each recognizing object group based on the judged result and recognition points outputted from a character recognizing means 7, graphic recognizing means 8 and editing symbol recognizing means 9.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a handwritten information processing apparatus having a function of recognizing handwritten information.

[0002]

2. Description of the Related Art In a handwritten information processing apparatus by handwriting input, characters, figures, edit symbols, etc. are considered as types of information input by a user. Therefore,
In order to edit a document in which characters and figures are mixed, it is necessary to distinguish whether the input information is characters, figures, or edit symbols by some method.

Conventionally, in this type of apparatus, a method of designating the type of information to be input in advance by an input mode switching operation has been widely used as a simple method for distinguishing the type of input information. . However, in this method, the switching operation of the input mode is a heavy burden on the user, and the operability is deteriorated.

In order to solve this problem, it is necessary to add a function of automatically identifying the type of input information. Therefore, JP-A-64-13682 (G06F15 / 20)
Then, it is determined whether the handwritten information is a character or a figure based on the shape features such as the number of strokes and the size of the writing pattern and the group type determined immediately before.

[0005]

SUMMARY OF THE INVENTION The present invention relates to an improvement of a handwriting information processing apparatus for editing a document in which characters and figures are mixed without switching the input mode.
This improves the performance of distinguishing between characters and figures.

[0006]

According to the handwritten information processing apparatus of the present invention, a character area (21,
(22,23,24) and area information setting means (4) for setting the graphic area (25) and the handwriting stroke set (31,32,33) to which recognition target group of characters and graphics belongs. , At least this handwriting stroke set (31,32,33)
And a character area (21, 22, 23, 24) or a graphic area (25) for determining the positional relationship between the character area (21, 22, 23, 24) and the graphic area (25).

Further, the handwritten information processing apparatus of the present invention is, at a minimum, a handwritten stroke set (31, 32, 33) and a character area.
(21,22,23,24) or the positional relationship with the graphic area (25), the recognition points (Vc,
A conversion candidate is determined by the certainty factor (P, Pf, Ps) obtained by correcting Vf, Vs).

[0008]

The handwritten information processing apparatus of the present invention is provided with the area information setting means (4) for setting the character areas (21, 22, 23, 24) and the graphic area (25) according to the already input state of the document to be edited. Since there is a character area (21, 22, 23, 24) and a graphic area (25)
Even if you do not set
The type of this handwriting stroke set can be determined depending on which area (1, 32, 33) is located.

Further, the handwritten information processing apparatus of the present invention has a position where the stroke set is written and a character area (21, 22, 23, 2).
4) and the graphic area (25) are judged by the writing position status judging means (10), and the judgment result (R) and the recognition score (21) output from each recognition means (7, 8, 9) , 22,23,24) based on the certainty factor (Pc, Pf, P
s) is obtained and compared to determine which recognition target group this stroke set belongs to.

[0010]

FIG. 1 is a block diagram showing the functional arrangement of a handwritten information processing apparatus according to an embodiment of the present invention. In FIG. 1, (1) is a display / input integrated tablet (hereinafter referred to as a tablet). This tablet (1) is a pen coordinate detection unit.
The device (1a) and the LCD display unit (1b) are integrated.
The coordinates of the coordinate detection unit (1a) and the LCD display unit (1b) are in one-to-one correspondence.

When writing on the tablet (1) with a pen, the coordinate detecting section (1a) causes the coordinate (x, x,
y) is detected, and time-series coordinate data (hereinafter referred to as handwriting coordinate data) indicating the pen handwriting is sent to the document editing device (2). (2) is a document editing device. This document editing device (2)
Is mainly composed of a CPU. This document editing device (2) reads the contents of the document memory (3) and writes the document on the tablet (1).
It is displayed on the LCD display section (1b).

Further, the document editing device (2) performs various controls such as data control between the devices shown in FIG. (3) is
Document memory. This document memory (3) stores a document in which graphics and characters are mixed. (4) is an area information setting device. This area information setting device (4) performs the next area detection in cooperation with a distribution information extraction device (5) that extracts information about the distribution of characters and figures in a document. At the start of operation of this handwriting information device, a document editing device for displaying is displayed.
When the contents of the document memory (3) are read by (2), the position and shape of the character area and the figure area are calculated according to the contents of each character and figure read from this document memory (3). To do.

Further, when the contents of the document memory (3) are updated by the editing operation of the document editing device (2), the area information setting device (4) is a character area according to the contents of the updated document. And correct the position and shape of the graphic area. Note that this is not a correction, and the position and shape of the area may be calculated again from the beginning. The area information setting device (4) calculates the “depth of the character area” at predetermined intervals within the character area. The "character area depth" is a parameter that reflects the character density at each point, and the larger the density of characters, the greater the value.

(6) is a confidence factor calculation / group determination device. This confidence factor calculation / group determination device (6), for the handwriting coordinate data input from the tablet (1) via the document editing device (2), each recognition device (7,8,9) and the writing position status In conjunction with the determination device (10), it is determined which of the character, graphic, and edit symbol belongs to the recognition target group (hereinafter referred to as grouping).

Confidence factor calculation / group determination device
(6) is a document editing device that displays the information indicating the recognition target group as the sentence processing result and the output from the corresponding recognition device.
Send to (2). (6a) is a certainty factor calculation unit. The certainty factor calculation unit (6a) calculates the recognition score (Vc, Vf, Vs) obtained from each recognition device (7, 8, 9) according to the handwriting input position.
The corrected certainty factor (Pc, Pf, Ps) is calculated.

(6b) is a group determination unit. The group unit (6b) compares the certainty factors (Pc, Pf, Ps) and determines which of the recognition target group the handwritten input stroke belongs to, a character, a figure, or an edit symbol. .
(7) is a character recognition device. This character recognition device (7)
Document editing device (2) consisting of a known character recognition processor
A character is recognized from the handwriting coordinate data input from the tablet (1) through the character code Cc which is the recognition result.
Character recognition score Vc and confidence factor calculation / group determination device (6)
To put out.

(8) is a figure recognition device. This figure recognition device (8) consists of a known figure recognition processor, recognizes a figure from the handwriting coordinate data input from the tablet (1) through the document editing apparatus (2), and the figure code which is the recognition result. Cf, the figure recognition score Vf, and information about the position, shape, size, and the like regarding the figure are output to the confidence factor calculation / group determination device (6).

(9) is an editing symbol recognition device. This editing symbol recognition device (9) consists of a known editing symbol recognition processor, recognizes the editing symbol from the handwriting coordinate data input from the tablet (1) via the document editing device (2), and the recognition result An edit symbol code Cs, an edit symbol recognition score Vs, information such as movement and insertion point corresponding to the symbol, and edit range information corresponding to the symbol are output to the confidence factor calculation / group determination device (6).

(10) is a writing position situation determination device. This writing position situation determination device (10) is based on the information on the position and shape of the character area and the graphic area input from the area information setting device (4) via the document editing device (2), and the document editing device ( 2) and the confidence factor calculation / group determination device (6), the situation of the writing position of the handwriting coordinate data input from the tablet (1) is checked, and the writing position situation determination result R is calculated as the confidence factor / group determination device ( Send to 6).

Here, the writing position condition determination result R is composed of four pieces of information. The four pieces of information are a flag Fc indicating whether or not the writing position is within the character area,
Flag F indicating whether the writing position is in the figure area
f, a flag Fr indicating whether the writing stroke string has a closed loop shape and surrounds the character area, and the depth Dc of the character area at the writing position.

Here, the confidence factor calculation / group determination device
The method of grouping in (6) will be described in more detail. The certainty factor calculation / group determination device (6) outputs the recognition scores Vc, Vf, Vs output from the recognition devices (7, 8, 9) and the writing position status output from the writing position status determination device (10). From the determination result R, the certainty factors Pc, Pf, and Ps for each recognition target group of characters, figures, and edit symbols are obtained. These certainty factors represent the probability that the writing stroke sequence belongs to each group, and are calculated by the following equations 1, 2, and 3.

Pc = Wc (Vc) + Ac (R) Formula 1 Pf = Wf (Vf) + Af (R) Formula 2 Ps = Ws (Vs) + As (R) Formula 3 In Formula 1, the function Wc is the character recognition score Vc. Is a weighting function for, and is a function that is predetermined so as to output an appropriate value according to the value of Vc.

Similarly, in equations 2 and 3, the functions Wf and Ws
Are weighting functions for the figure recognition score Vf and the edit symbol recognition score Vs, respectively, and are functions predetermined to output appropriate values according to the values of Vf and Vs. The reason for using the functions Wc, Wf, Ws will be described below. Each recognition score Vc, Vf, Vs corresponds to the corresponding recognition result Cc,
The degree of similarity of Cf and Cs to the model pattern is shown.

Therefore, for example, if the character recognition score Vc has an extremely large value, it can be interpreted that there is a high possibility that it is a character. However, the recognition score Vc, V
f and Vs are values obtained by separate recognition devices. Therefore, these values cannot be directly compared, and just because Vc <Vf, for example, does not necessarily mean that it is a figure.

Therefore, it is necessary to use the functions Wc, Wf, Ws to convert the recognition scores Vc, Vf, Vs into values that can be compared with each other. That is, when both the character and the figure can be received only by the shape of the stroke, the function values Wc (Vc) and W
f (Vf) has almost the same value, and when the stroke shape clearly indicates a character, the function value Wc (Vc)
Is an extremely large value as compared with Wf (Vf) and Ws (Vs).

Therefore, the group can be determined by comparing these function values. The above is the reason for using the functions Wc, Wf, and Ws. Next, the functions Ac and A in equations 1, 2 and 3
f and As will be described. The function Ac in Expression 1 is a function that operates the value of the certainty factor Pc for a character in accordance with the writing position situation determination result R, and in the case of “flag Fr = true”, a negative constant value Vac, In the case, it has 0 as a value. This is because the writing stroke sequence that surrounds the character area in a closed loop shape is likely to be a figure or a part of an edit symbol, and is unlikely to be a character.

The function Af in Expression 2 is a function for manipulating the value of the certainty factor Pf for the figure in accordance with the writing position situation determination result R, and in the case of "flag Ff = true" and "flag Fc = false". It has a positive constant value Vaf as a value. This is because the stroke sequence written in the graphic area is considered to be highly likely to be a graphic.

Also, "flag Fc = true" and "flag Ff =
In the case of "false", it has a function value Uaf (Dc) that takes a negative value as a value. The value of the function Uaf is the depth Dc of the character area at the writing position.
Uaf takes a smaller value as Dc increases. It is considered that the stroke sequence written in the character area is highly likely to be a character or an edit symbol and is unlikely to be a figure, and the higher the density of the characters near the writing position, the more the figure. This is because it is considered unlikely.

Further, "flag Fc = true" and "flag Ff =
In the case of “true”, Af has a value of Vaf + Uaf (Dc), and takes a value of 0 when none of the above is satisfied.
The function As in Expression 3 is a function that operates the value of the certainty factor Ps for the edit symbol according to the writing position situation determination result R, and is negative in the case of “flag Fc = false” and “flag Ff = false”. It has a constant value Vas, and 0 otherwise. This is because it is considered that a writing stroke sequence that is not included in either the character area or the graphic area is unlikely to be an edit symbol.

The confidence factor calculation / group determination device (6) compares the magnitudes of the confidence factors Pc, Pf, and Ps with respect to the characters, figures, and edited symbols obtained as described above, and finally the highest confidence factor is obtained. Items are the result of grouping. FIG. 2 is a diagram for explaining the operation. Figure 2 (a) shows the document memory (3)
The content of the document stored in the table is displayed on the LCD display unit (1b) of the tablet (1) via the document editing device (2).

The contents of the document memory (3) are sent to the distribution information extraction device (5). Distribution information extraction device (5) is a document memory
From the contents of (3), the position and size of each input character and the position, shape, and size of each figure are obtained and sent to the area information setting device (4). The area information setting device (4) uses this to position the character area and the graphic area as shown in FIG.
Calculate the shape.

In FIG. 2 (b), (21,22,23,24) is a character area. (25) is a graphic area. Further, for each point in the character area (21, 22, 23, 24), the depth of the character area is calculated according to the character density. Figure 2 (c) shows the writing stroke sequence (31, 32, 33) on the tablet (1) from the state of Figure 2 (a).
Represents the input state.

When the input end is detected in this state, the recognition process is started for the handwriting coordinate data corresponding to the writing stroke sequence (31, 32, 33). The input handwriting coordinate data is
It is sent to each recognition device (7, 8, 9) and the writing position status determination device (10) via the document editing device (2). At this time, the writing position status determination device (10) is moved to the position of the character area (21, 22, 23, 24) and the graphic area (25) from the area information setting device (4) via the document editing device (2). And information about the shape is also sent.

The writing position situation determination device (10) checks the writing position situation of the input handwriting coordinate data, and outputs the writing position situation determination result R to the confidence factor calculation / group determination device (6). Each recognition device (7,8,9) recognizes a character, a figure, and an editing symbol from the input handwriting coordinate data, and calculates the recognition result codes Cc, Cf, Cs and the recognition scores Vc, Vf, Vs. -Send to the group judgment device (6).

In this example, the character recognition device (7) recognizes it as a "mouth" and the graphic recognition device (8) recognizes it as a "rectangle". Further, the edit symbol recognition device (9) rejects because there is no edit symbol similar to the input stroke sequence. In this example, the writing position (31, 32, 33) is the character area.
Since it is not included in any of (21,22,23,24) and the graphic area (25), "flag Fc = false" and "flag Ff = false".
Further, since the writing stroke string has a closed loop shape and surrounds the character area (24), "flag Fr = true" is set.

The confidence factor calculation / group determination device (6) uses the results of the above-mentioned processing output from each device (7,8,9,10) in accordance with equations 1, 2, and 3 to edit characters, figures, and edits. Confidence Pc, Pf, Ps for each group of symbols is obtained and grouped. In this example, the function values Wc (Vc) and Wf (Vf) are almost the same value because the character "mouth" and the figure "rectangle" can be interpreted only by the shape of the writing stroke sequence ( The function value Ws (Vs) is extremely small compared to these values.

On the other hand, since the results of the determination of the writing position situation are "flag Fc = false", "flag Ff = false", and "flag Fr = true", the values of the functions Ac, Af, As are respectively Ac ( R) = Vac,
As (R) = 0, As (R) = Vas. Using these results, the confidence level for characters, figures, and edit symbols is obtained, and by comparing the magnitudes, the writing stroke sequence (31, 32, 33)
Is determined to be a figure "rectangle".

The confidence factor calculation / group determination device (6) sends this result to the document editing device (2). The document editing device (2)
Upon receipt of this result, the editing operation is executed to rewrite the contents of the document memory (3), and the contents of the edited document are displayed on the LCD display section (1b) of the tablet (1). This result is shown in Figure 2.
It is shown in (d). Confidence factor calculation / group determination device (6)
When grouping in, when the group to which it belongs is apparent only from the geometrical features of the writing stroke sequence, that is, the certainty factor obtained from the recognition score given by each recognition device (7,8,9), that is, Equation 1, The first term Wc (Vc) on the right side of a few
When it is possible to clearly identify a certain group from the comparison of Wf (Vf) and Ws (Vs), it may be decided at that stage.

The expressions 1, 2 and 3 used in the present embodiment do not limit the method of calculating the certainty factor for each group in the present invention, and are appropriately desirable within the scope of the claims. There is no problem in calculating with. Further, in the present embodiment, the grouping is performed using both the recognition score obtained from each recognition device and the determination result obtained from the writing position situation determination device, but the present application uses only the determination result R. It is also possible and sufficient effect can be obtained. However, it goes without saying that the effect is increased by combining both.

Further, in the present embodiment, each recognition device outputs only one code, but it may output a plurality of codes and a plurality of recognition scores corresponding thereto. Further, in the present embodiment, only the code with the highest confidence value is selected and output. However, a plurality of candidates may be displayed in order of the confidence value, and the candidates may be selected from this. It also displays multiple candidates with high confidence values,
You may make it select. Alternatively, the code with the highest confidence value may be displayed at the stroke input position, and another candidate with the highest confidence value may be displayed in another area.

Further, in this embodiment, the code having the highest certainty factor is always output. However, when the certainty factor value is less than a predetermined value, the input stroke is displayed as it is and the certainty factor is displayed. It is also possible to display a candidate having a high price within a predetermined value in a different area.

[0042]

According to the present invention, the area information setting device (4) can be used to edit a handwritten stroke string even when a document in which characters and graphics are mixed is edited without setting the group type of the input area. Since the recognition target group can be automatically determined, it is not necessary to perform the input mode switching operation and the input area group type setting even when editing a document in which characters and figures are mixed.

Further, according to the present invention, when performing grouping,
Since the writing position situation of the stroke is taken into consideration together with the recognition score, it is possible to make a correct determination even in the case where it cannot be determined only by the shape of the stroke, which was difficult in the past.

[Brief description of drawings]

FIG. 1 is a block diagram showing a functional configuration of a handwriting information processing apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram for explaining the operation of this embodiment.

[Explanation of symbols]

 (1) Tablet with integrated display and input, (1a) Input section, (1b) Display section, (2) Document editing device, (3) Document memory, (4) Area information setting device, (5) Distribution information extracting device, (6) Confidence factor calculation / group determination device, (6a) Confidence factor calculation part, (6b) Group determination part, (7) Character recognition device, (8) Figure recognition device, (9) Editing symbol recognition device, (10 ) Writing position status determination device.

Claims (8)

[Claims] 1. At least for a coordinate input means (1a) for inputting handwriting coordinate data and a stroke set (31, 32, 33) consisting of handwriting coordinate data input by the coordinate input means (1a). , A recognition means (7, 8) for performing character recognition and graphic recognition processing and outputting a recognition result for each, and a document editing means for editing a document based on the recognition result recognized by the recognition means (7, 8) ( 2) and a handwriting information processing device comprising: a distribution information extracting means (5) for extracting information on the distribution of characters and figures from the input state of the document to be edited; and the distribution information extracting means (5 Character area (21,22,23,24) where the characters are entered using the distribution information extracted in).
An area information setting means (4) for setting a graphic area (25) in which a graphic is input, a position where the stroke set (31, 32, 33) is input, and the character area (21, 22, 23, 24) and the graphic area (25)
The writing position situation determining means (10) for determining the positional relationship with, and the stroke set (31, 32, 33) to which recognition target group of characters and graphics belongs to at least the writing A handwriting information processing device comprising: a judgment means (6) for judging from a judgment result of a position situation judgment means (10). 2. Character recognition for a coordinate input means (1a) for inputting handwriting coordinate data and a stroke set (31, 32, 33) consisting of handwriting coordinate data input by the coordinate input means (1a). Recognition means (7,8,9) for performing two or more recognition processes among the pattern recognition, the figure recognition, and the editing symbol recognition and outputting the recognition result for each, and the recognition recognized by the recognition means (7,8,9). A handwriting information processing device comprising: a document editing unit (2) for editing a document based on the result; and a distribution information extracting unit for extracting information on the distribution of characters and figures from the already input state of the document to be edited ( 5) and the distribution information extracted by the distribution information extraction means (5), the part in which the character is input is displayed in the character area (21, 22, 23,
24), the area where the graphic is input is defined as the graphic area (25), and the character area (21, 22, 23, 24) and the graphic area (2
Area information setting means to set the position and shape of 5) (4)
And the stroke set (31, 32, 33) based on the positions and shapes of the character areas (21, 22, 23, 24) and the graphic area (25) output from the area information setting means (4) A writing position situation determining means (10) for determining the positional relationship between the position where is input and the character area (21, 22, 23, 24) and the graphic area (25), and the writing position situation determining means (10 ) From the determination result of the stroke set (31, 32, 33) is a confidence factor calculation means (6a) for obtaining a confidence factor for determining which recognition target group of the characters, figures, and edit symbols belongs to, Group determination means (6b) for determining which recognition target group the stroke set belongs to by comparing the certainty factors obtained by the certainty factor calculation means (6), and handwriting characterized by the following: Information processing equipment. 3. Coordinate input means (1a) for inputting handwriting coordinate data, and character recognition, graphic recognition, edit symbol recognition for a stroke set consisting of handwriting coordinate data input by the coordinate input means (1a). A recognition means (7,8,9) for performing two or more recognition processes and outputting a recognition result and a recognition score for each, and a document editing means for editing a document based on the recognition result recognized by the recognition means. A handwritten information processing device comprising (2), comprising: a distribution information extracting means (5) for extracting information on the distribution of characters and figures from the already input state of a document to be edited; and the distribution information extracting means (5). Area information setting means for setting the position and shape of the character area and the graphic area by using the extracted distribution information as a character area for a character input area and a graphic area for a graphic input area (4) and Writing position for determining the positional relationship between the position where the stroke set is input and the character area and the graphic area based on the positions and shapes of the character area and the graphic area output from the rear information setting means (4) The situation determination means (10), the stroke position is a character from the determination result of the writing position situation determination means (10) and the recognition score output from the recognition means,
Confidence factor calculation means for obtaining the credibility factor to determine which recognition target group of the figure and the edit symbol belongs to (6
a) and group determination means (6b) for determining which recognition target group the stroke set belongs to by comparing the certainty factors obtained by the certainty factor calculation means, Handwriting information processing device. 4. The area information setting means (4) updates the positions and shapes of the character area and the graphic area each time a new document editing operation is performed. Handwriting information processing device. 5. The certainty factor calculating means (6a) obtains the certainty factor for each of the recognition target groups according to the density of characters already input at the input position of the stroke set. The handwritten information processing apparatus according to claim 2. 6. The certainty factor calculation means (6a) determines the certainty factor with respect to the figure when the writing position situation determination means (10) determines that the writing position is within the character area. The handwriting information processing apparatus according to claim 2, wherein the certainty factor with respect to the figure is relatively increased when it is determined that the figure is within the figure area. 7. The certainty factor calculating means (6a) determines that the writing position is outside the character area and outside the graphic area as a result of the judgment by the writing position situation judging means (10). The handwriting information processing apparatus according to claim 2, wherein the certainty factor relative to the edit symbol is relatively lowered. 8. The confidence factor calculation means (6a), as a result of the judgment by the writing position status judgment means (10), the writing stroke sequence (31, 32, 33) has a closed loop shape and the character area (2
4. The handwriting information processing apparatus according to claim 2, wherein the certainty factor for the character is relatively lowered when it is determined that it surrounds 0).