JPH04336684A - Method and device for document processing - Google Patents

Abstract

PURPOSE:To output a text obtained through character recognition after speech synthesis by operating a character recognition device and a speech synthesizer with a host computer. CONSTITUTION:A part of character recognition processing for one character image with the heaviest load in a series of character recognition operation is made independent as separate control and separate box body, and the other processing is separated so as to be performed on a main machine. The left part of dotted lines shows the personal computer main body and the right part shows each equipment connected to the personal computer. A CPU 1 controls and judges the entire system according to various programs stored in an external storage device 9. Especially, a character recognition device 21 connected to the personal computer and a speech synthesizer 22 can operate by one host computer. Thus, a speaker is generated by adding commercially marketed speech output device and soft ware to the text of the recognition result and the character recognition result can be confirmed through speech.

Description

[Detailed description of the invention]

0001

[Field of Industrial Application] The present invention relates to a document processing method and device that reads and recognizes character information on a document using an optical character reader (OCR) and outputs the text obtained as a recognition result as audio. be.

0002

[Prior Art] Conventionally, a character recognition device performs all processes such as reading an image on a sheet of paper, searching for lines, cutting out characters one by one, character recognition, evaluating and correcting the recognition results, and displaying them. In some cases, this was done using a single workstation, a personal computer, or special character recognition equipment.

Conventionally, there have been speech synthesis devices that input text, synthesize it into speech, and output it.

0004

However, in the conventional example described above, all the character recognition work is performed on one machine, which has the following drawbacks. (1) Since a large load is placed on the machine itself, a high-performance or expensive machine is required. (2) Even if a high-performance machine is used, the recognition speed is low. 1
Products with speeds of 0 to 20 characters/second are common. (3) Since the machine itself is essentially a specialized machine for character recognition, there is little freedom in each step involved in character recognition processing. In other words, it is virtually impossible for the user to improve the routine for reading character images of type on paper or the routine for cutting out single character images. (4) As in (3), since the machine itself is almost a specialized machine for character recognition, even if the user tries to add an audio output device for the character recognition results, for example, Practically not possible.

[0005] However, in the above-mentioned conventional example, since the program is designed to use the host computer only for character recognition processing, it is only possible to display the recognition results as they are on a display, etc., and it is impossible to convert data. .

[0006]

[Means for Solving the Problems (and Effects)] According to the present invention, among a series of character recognition tasks, the most
The character recognition process for only one character image is controlled separately.
By separating it into a separate chassis and performing all other processing on the main machine as before, (1) the main machine can be run on a low-cost personal computer or workstation, and (2) ) Recognition speed is also faster than conventional 2~
3 times the speed of 60 to 70 characters/second can be secured, and (3) the main machine-side processing is a program for reading type images on paper, a program for cutting out single character images, and evaluation and correction of recognition results. , text display programs, etc., users have a considerable degree of freedom by combining commercially available software or creating some themselves. In addition, a speaker is used to generate the sound so that character recognition results can be confirmed by voice.

[0007]

[Example] (Example 1) A first example will be described below with reference to the drawings.

FIG. 1 is a system configuration diagram of this embodiment. The left side of the dotted line shows the main body of a personal computer (hereinafter referred to as a personal computer), and the right side shows various devices connected to the personal computer. A central processing unit (hereinafter referred to as CPU) 1 controls and makes decisions about the entire system according to various programs stored in an external storage device 9. 2 is a CRT that displays data and text
A display, 3 a keyboard having various keys for inputting character strings, 4 a mouse as a pointing device, and 5 a memory. The memory 5 includes a read-only memory (hereinafter referred to as ROM), a random access memory (hereinafter referred to as RAM) 7, and a video RA.
M (hereinafter referred to as VRAM)8. R.O.
The M6 includes a part of an operating system (hereinafter referred to as OS), which is software that performs basic input/output control, and character fonts for display. RAM
7 is a place where various programs stored in the external storage device 9 are loaded and executed. For example, an image buffer used by the image reading program 10-2, a character cutting program 10-3, a sentence recognition command program 10-
4 and a character recognition buffer used by the post-processing program 10-5, and a speech synthesis buffer used by the speech synthesis program 10-6 are provided in the RAM 7. The VRAM 8 is a place where display data to be displayed on the CRT display 2 is stored.

The external storage device 9 includes various programs 10 for processing as shown in flowcharts shown in FIGS. 6 to 22, which will be explained later, data storage files 11 and 12, dictionary files 13 to 14, and the like. There is. To be more specific, 10 is character recognition/speech output software used in this implementation, and various programs are started according to user instructions under the control of a main program 10-1, which will be explained later using the flowchart of FIG. Reference numeral 10-2 is a program for capturing an image on a sheet into the image buffer 7-1 of the RAM 7 using the scanner 20, which will be described later using the flowchart of FIG. The character cutting program 10-3 is a program that cuts out an image for one character from the image in the image buffer 7-1, and will be described later using the flowchart of FIG. 10-
4 is a character recognition command program, and the extracted 1
This is a program that sends image information for a character to the character recognition device 21 and executes character recognition processing for one character, and will be described later using the flowchart of FIG. 10-5 is a post-processing program that evaluates the text after character recognition, and when there are multiple character candidates, selects the character candidate that is estimated to be more correct as the first candidate, etc. This will be described later using the flowchart in FIG. Reference numeral 10-6 is a speech synthesis program, which converts text completed by character recognition into kanji and kana characters into readings, adds accent and intonation information to the text, sends it to the speech synthesis device 22, and generates it from the speaker 23. This is a program that performs processing, and will be described later using the flowchart in FIG. At this time, the speech synthesis program 10-
6 uses the kanji-kana conversion dictionary file 13, the accent dictionary file 14, and the intonation regulation file 15 to convert kanji into readings, add accent information, and add intonation information. 10-7 is a storage program that saves the image data in the image buffer 7-1 to the image file 11, or saves the text data of the character recognition result in the character recognition buffer 7-2 to the text file 12. It has the role of storage, and will be described later using the flowchart of FIG. Also,
Conversely, it also has the role of reading and setting data in the image file 11 or text file 12 into the image buffer 7-1 or character recognition buffer 7-2. Furthermore, it also has the role of printing the image data in the image buffer 7-1 and the text data in the character recognition buffer 7-2 onto paper using the printer 24.

FIG. 2 is a system configuration diagram of the character recognition device 21. Although it is connected to a personal computer via a connection section 34, the entire device is controlled by a CPU 30. 31 is a memory, ROM32 and RAM33
The ROM 32 contains a character recognition program 32.
-1 and "line segment set data of all characters" 32-2 used in the character recognition processing routine are built-in. The RAM 33 stores image data 33-1 of an image of one character and recognition result data during character recognition work. The procedure is that when a recognition start command signal is received from the personal computer, the character recognition program 32-1 first captures an image for one character and stores the image data 33-1 on the RAM 33.
Set as . After various processes are performed, the character candidates are compared with the "line segment set data of all characters" 32-2, and the most probable character candidates are arranged with scores and set as recognition result data 33-2. Then, the recognition result data 33-2 is sent back to the personal computer. Details of the process will be explained with reference to a flowchart shown in FIG. 7, which will be described later.

(a) in FIG. 3 shows image data 33-
This is a format example diagram of No. 1, which shows that a character font that seems to be the character for "power" is set in a 16 x 16 dot image.

(b) in FIG. 3 shows image data 33-
This shows an example of recognition result data after subjecting 1 to character recognition processing. The leftmost item is the candidate ranking column, from 1st to 8th place.
In the second column, according to the JIS Kuten code,
Codes for one character are lined up, and the score for each candidate is listed in the last column. In this embodiment, the maximum score is 10 points, and the minimum score is 1 point. This score is determined by comparison with "line segment set data of all characters". The top 8 characters in order of highest score are recognized as recognition result data 33.
-2. By the way, it can be placed in Figure 3 (b
), the first place is the katakana ``ka'', the second place is the kanji ``chikara'', and the third place is the small katakana ``ka''.
, 8th place is the kanji ``katana''.

FIG. 4 is an example of a screen display of the character recognition/voice output software 10 on the CRT display 2. As shown in FIG. 40 is a screen frame, 41 is a title, and 42 is an end mark. When the user moves the mouse 4 to align the mouse cursor 55 with the end mark 42 and clicks the button on the mouse 4, the speech recognition/speech output software 10 ends. and return to the OS prompt display. Reference numeral 43 denotes a menu field, in which menus ``Load'', ``Recognition'', ``Audio'', and ``Save'' are lined up. To select the menu, click the end mark 42
As in the case of , all you have to do is align the mouse cursor 55 with the corresponding menu and click the button of the mouse 4. By clicking the button, the command currently indicated by the mouse cursor is sent to the CPU. "
"Load" menu is the image loading program 10-
2 is started and the image on the paper is taken into the personal computer via the scanner 20. At the same time, the read image is displayed on the read image display 44. The "recognition" menu includes the character extraction program 10-3 and the character recognition command program 10-3.
4. This is for activating the post-processing program 10-5 and starting character recognition work. As for the procedure, the user must first determine the Taisho region for character recognition on the read image 44 on the CRT 2. To do so, place the mouse cursor 55 on the upper left corner of the target region,
Press the mouse button, move the mouse cursor 55 while holding it down, and release the mouse button when it reaches the lower right corner of the target area. As a result, a frame 46 of the recognition target area is displayed. Then, when the user selects the "recognition" menu, the recognition result is displayed on the result display 45. This means that the character recognition work is finished.

The "voice" menu is for making the text 45 resulting from the recognition aloud using the voice synthesizer 22 and the speaker 23. As a procedure, the user first uses the mouse cursor 55 to determine the target area 47 for audio output. This is the same operation as when determining the target area 46 for character recognition. Then, when the user selects the "sound" menu, the speech synthesis program is activated, and the text within the output target area 47 is automatically uttered from the speaker 23.

When the "Save" menu is selected, the save program 10-7 is started, and a menu for more detailed selections appears. It is possible to select and execute the display set of the read image 44 by reading the image image, reading the text data from the text file 12 and displaying and setting the result text, or printing the image image and printing the result text.

[0016] 49 and 48 are next page display switching buttons when there is a lot of image data and it spans multiple pages;
This is a previous page display switching button. 50 is a display of the number of pages. Similar to 49 and 48, 52 and 51 are next page display switching buttons and previous page display switching buttons when the resulting text data is large and spans multiple pages, and 53 is a page number display.

Note that 56 is a character cursor. The user can also use the keyboard 3 to edit the resulting text. In other words, the character at the character cursor 56 position can be deleted, a character string inserted, moved, etc. If an error occurs in character recognition, it is possible to correct the text using this method.

FIG. 5 shows an example of the format of text as a result of character recognition on the character recognition buffer 7-2. The character codes are written in order from the beginning, but for the character “ka”, there were candidates with the same character recognition score, so the katakana character “ka” and the kanji character “chikara” were juxtaposed. There is. In order to distinguish in the text that two candidates are lined up, they are surrounded by a start mark code and an end mark code, separated by a delimiter mark code, and in between are the character code for katakana "ka" and the kanji " It has a code of "power" embedded in it. Note that what is displayed on the display as the result text 45 is the katakana character "ka" that appears first and is the first candidate in this case.

Next, the process flow of this embodiment will be explained using a flowchart.

FIG. 6 is a flowchart explaining the operation of the main program 10-1. Basically, this indicates that each processing program is activated by the user's menu selection. Since the outline of the operation has already been explained using FIG. 4, detailed explanation will be omitted here. The "load" menu selection in step S2, the "recognition" menu selection in step S6, the "audio" menu selection in step S8, and the "save" menu selection in step S12 only start each processing program, but in step S4 Area specification work, step S10 editing work, step S
The page change operation No. 14 is processed by the main program 10-1 itself.

FIG. 7 shows the ROM 32 in the character recognition device 21.
This is a flowchart illustrating the operation of the character recognition program 32-1 in FIG. This program starts up at the same time as the power of the speech recognition device 21 is turned on, and continues to operate until the power is turned off.

First, in step S20, it is checked whether the power is turned off, and if it is turned off, the process ends. If NO (if the power is ON), a check is made in step S21 to see if a command to start character recognition processing has come from the personal computer, and if NO, the process returns to immediately before step S20 to form a loop. If YES in step S21, step S2
Proceeding to step 2, an image for one character is fetched from the personal computer main body side and set on the RAM 33 as image data 33-1. Then, in step S23, from the image data 33-1, the outline of the black area corresponding to the font of the typeface,
That is, an outline is obtained, line segments, ie, vectors, are created along the outline in step S24, and set data of the line segments is obtained in step S25. Then, in step S26, the above line segment set data and the ROM 32
The line segment set data 32-2 of all the characters on the top are compared in terms of vector matching, and in step S27, the recognition result data 33 are selected from the first to eighth candidates in descending order of matching score. Create -2. Then, in step S28, the recognition result data 33-2 is transferred to the personal computer main body side, and the series of processing ends. Thereafter, the process returns to step S20, forming a loop and waiting again for a character recognition start command from the personal computer.

[0023] This character recognition processing method is a well-known technique in various information processing-related literature as the ``pseudo-Bayesian discriminant function method,'' so we will not go into further details. , the explanation will be omitted.

FIG. 8 shows the image reading program 10-2.
2 is a flowchart illustrating the operation of FIG. When started from the main program 10-1, first, step S30
Then, the image on the paper is read from the scanner 20, and in step S31, the image data is set in the image buffer 7-1 on the RAM 7, and in step S32, the CR
An image is displayed on the T-display 2 as shown at 44 in FIG. 4, the program returns, and control returns to the main program 10-1.

FIG. 9 shows the character recognition command program 10-4.
2 is a flowchart illustrating the operation of FIG. When started from the main program 10-1, it is first checked whether the target area for character recognition has been determined. If NO, return as is. If YES, the process moves to step S41 and starts the character cutting program 10-3, and when the processing of the character cutting program 10-3 is finished, then in step S42,
The post-processing program 10-5 is activated, waits for its processing to finish, returns, and returns control to the main program 10-1.

FIG. 10 shows the character extraction program 10-
3 is a flowchart illustrating the operation of step 3. When started by the character recognition command program 10-4, first, in step S50, the first character image of the recognition target area is cut out, and in step S51, the character recognition device 21
Sends a single character image and commands the start of recognition processing. Then, in step S52, the recognition result is received from the character recognition device 21, and in step S53, the first candidate character code is added to the character recognition buffer 7-2 on the recognition result data.
Add to. Then, in step S54, if there is a candidate with the same score, the character code of the second candidate is also added to the character recognition buffer 7-2. At this time, the character codes of the first and second candidates are set in a format embedded between the start mark, delimiter mark, and end mark, as explained with reference to FIG. Then, in step S55, the next character image is cut out in the recognition target area, and in step S56, it is checked whether the next character image exists. If YES, the process returns to immediately before step S51 and the above-mentioned process is performed. Repeat. If NO, the process returns and control returns to the character recognition command program 10-4.

FIG. 11 is a flowchart illustrating the operation of the post-processing program 10-5. When started by the character recognition command program 10-4, first, in step S60, the character recognition buffer is searched to see if there is a part where the first candidate and the second candidate are juxtaposed, and the result is sent to step S61. Determine if it is, and if NO,
Return from this program. If YES, the process moves to step S62, and if that part is a character surrounded by katakana characters at the front and back, and if there is a katakana character among the candidates, that part is selected as the first candidate. Next, step S
63, if that part is a character surrounded by hiragana characters on both sides, and if there is a hiragana character among the candidates, that part is selected as the first candidate. Next, proceed to step S64,
If that part is a character surrounded by kanji and hiragana on the front and back, and if there is a kanji as a candidate, that is selected as the first candidate. Then, in step S65, the character recognition buffer is searched again, and it is investigated whether there is a part where the next first candidate and second candidate are juxtaposed.The result is checked in step S66, and if YES, immediately before step S62. After that, the process of steps S62, S63, S64, and S65 is repeated. If NO in step S66, step S67
Proceed to , display the text with changed candidates on the screen, and return from the post-processing program.

FIG. 12 is a flowchart illustrating the operation of the speech synthesis program 10-6. This program is started from the main program 10-1, and first, in step S70, it is checked whether the audio output area has been determined. This is the frame 47 described in FIG. If NO, the program returns, but if YES, in step S71, the first sentence is extracted from the area determined in the speech synthesis buffer 7-3. And step S
72 performs kanji-kana conversion, converting the original sentence with kanji into a sentence with only kana reading separated into clauses. At this time, a kanji-kana conversion dictionary file 13 containing correspondence information between kanji words and kana readings is used. Furthermore, the separation of phrases is processed using the two-clause longest matching method of reading so that the length of reading for each phrase is as long as possible. Next step S7
In step 3, accent information is added to each word that makes up the sentence. At this time, an accent dictionary file 14 containing word-accent correspondence information is used. Next, in step S74, the intonation of this sentence is adjusted. Here, for example, if the sentence is a question, information such as raising the pitch at the end of the sentence is added. At this time, an intonation rule file 15 containing sentence structure and intonation rule information is used. Then, in step S75, it is checked whether or not the speech synthesizer 22 is producing a voice, and if it is producing a voice, it waits until it is finished. Then, the process moves to step S76, and the reading, accent, and intonation information obtained in steps S71, S72, and S73 is sent to the sound synthesis device 22,
Command vocalization. Then, in step S77, the next sentence is taken out and set in the phonetic synthesis buffer 7-3, but in step S78, it is checked whether there is "the next sentence", and if YES, it is checked. , returns to immediately before step S72, and from step S72 to step S
Repeat the process up to 77. If NO, the speech synthesis program 10-6 is returned.

FIG. 13 is a flow chart for explaining the operation of the storage program 10-7. When the save program 10-7 is started from the main program 10-1, a small window is opened in which various save menus are lined up for the user to select. FIG. 13 explains each process when the user makes a selection operation from the menu. If the user selects "end" in step S80, the small window with various save menus is closed and the save program is ended. If the user selects "text save" in step S81, the recognition result data 45 on the character recognition buffer 7-2 is saved in the text file 12 in step S82. If the user selects "save image" in step S83, the image buffer 7-1 is saved in step S84.
The read image data 44 at the top is converted to the image file 11.
Save to. If the user selects "read text" in step S85, the result data in the text file 12 is read in and set in the character recognition buffer 7-2 in step S86. If the user selects "image loading" in step S87, step S88
Then, the data in the image file 11 is read and set in the image buffer 7-1. In step S89, the user selects "
If you select "Text Print", in step S90,
The result data 45 in the character recognition buffer 7-2 is printed on the printer 24. The characters in a certain part are the first candidate and the second candidate.
If candidates are lined up, only the first candidate character is printed. If the user selects "image printing" in step S91, the data in the image buffer 7-1 is printed on the printer 24. Note that steps S82, S84, S86, S
After each process of 88, S90, and S92 is completed, the process returns to immediately before step S80 to form a loop, and waits for the menu selection by the user.

By using this embodiment, as shown in FIG. 4, the image before character recognition and the resulting text after character recognition are displayed side by side on one screen, so the user can check the recognition results. It's made easy to do.

[0031] Furthermore, since the character recognition results can be outputted audibly, it is easy to check the recognition results in this respect as well. Moreover, if there are any errors in the recognition results,
It can be modified directly using the keyboard.

[0032] Also, since multiple page processing is included,
It is possible to handle even large amounts of data.

(Example 2) Next, when performing character recognition,
An example will be described in which a plurality of candidate characters are derived with scores, and if the scores are low, the character extraction area is changed.

FIG. 14 is an explanatory diagram of character extraction processing. (a) in FIG. 14 will be explained using the hiragana character "ho". In the first embodiment, the character cutting program 10-3 is in charge of cutting out the image of one character from the image, but for example, suppose that the left half of the character "ho" is cut out by mistake. In the first embodiment, there was no choice but to cut out the right half of "ho" as the next character image and perform character recognition processing continuously. As a result, "1" and "ma" are output as the recognition results. Therefore, in the second embodiment, the score is checked at the stage when the character recognition of the left half of "ho" is output. The score in this case is the score shown in (b) in FIG. 3 of Example 1. For example, if this score is less than 8 points, it is assumed that the character image was cut out incorrectly.
Next, the image of the right half of "ho" is added to the left half, and the character recognition process is performed again as a single character image. As a recognition result, the correct character "ho" is obtained.

14(b) is also an example corresponding to the character ``kari'', similar to the character ``ho'' in FIG. 14(a). When I performed character recognition on the left half of the image of "Kari", the score was low, so I merged the right half of the image of "Kari" and performed character recognition processing again with "Kari" as a single character image. The character ``kari'' was obtained.

FIGS. 15 and 16 are flowcharts illustrating the operation of the character extraction program in this case. This embodiment is similar to the first embodiment in that the character recognition command program 10-4 starts the character segmentation program 10-3. In this embodiment, when started, first in step S100, a character image for the first "one character" in the target area for character recognition is cut out. Next, in step S101, an image for one character is sent to the character recognition device 21 to instruct the start of character recognition processing, and in step S102, the recognition result is received. Then, in step S103, the score of the recognition result is checked, and if it is higher than the standard, step S110
fly to In this case, the "rule" is, for example, "If the score is 8 or more on a scale of 10, the answer is YES." In step S110, the character code of the first candidate is added to the character recognition buffer 7-2, and if there is a candidate with the same score, the character code of the second candidate is also added to the character recognition buffer 7-2. Then, in step S112, the next character image is cut out from the recognition target area, and in step S113, the next character image is cut out from the recognition target area.
The presence or absence of a character image is checked, and if there is no character image, the character extraction program is returned. If "Yes", the process returns to immediately before step S101, forms a loop, and repeats the processes from step S101 to S112. Steps S100, S101, S102, S110, S
Steps 111, S112, and S113 are exactly the same as in the first embodiment, but the unique parts in this embodiment are steps S103 to S109.

If the score is low and the answer is NO in step S103, the process moves to step S104, where the next one-character image is added to the one-character image analyzed just before to form a new one-character image, and in step S105, this new one-character image is A single character image is sent to the character recognition processing device 21, command is given to start character recognition processing, and the recognition result is received in step S106. Then, in step S107, the score is compared with the score of the previous recognition result, and if the score is "high", the process directly proceeds to step S110 and subsequent steps. If it's not expensive,
In step S108, the added single character image is canceled and redefined as an unanalyzed image, and in step S109, the previously received recognition result is made the official recognition result. After that, the process moves to step S110.

The processes in steps S108 and S109 are provided for the case where the combined character image is not necessarily the correct character.

In both the first embodiment and the second embodiment, the data format in the character recognition buffer is as shown in FIG. In other words, there was always only one character candidate. On the other hand, in FIG. 17, considering the difference in character extraction methods,
This is an example of a format that includes two results: one where the left half image and the right half image of a certain character are recognized separately, and the other when the left half and right half are combined. be. In this case, the candidates analyzed as "me" and "ri" and the candidates analyzed as "kari" are juxtaposed. In addition, candidates for "woman" and "child" and candidates for "girl" are also juxtaposed. In other words, the character candidate is not necessarily limited to one character, but may be two characters. This point differs from Example 1 and Example 2.

(Embodiment 3) In Embodiment 1, an example was described in which kanji-kana conversion is used to convert a sentence containing kanji and kana in the character recognition result text to a reading-only sentence when outputting voice. Here, we will discuss an example used to replace character candidates in recognition result text that includes character candidates.

FIG. 18 is a flow chart for explaining a character candidate replacement processing program using this method. Note that this program operates by being inserted immediately after step S42 "Start post-processing program" in the character recognition command program described in FIG. 9 of the first embodiment.

[0042] When started, first in step S120,
Perform kanji-kana conversion to obtain reading-only sentences, step S
At 121, the reading is divided into clauses using the longest matching method. In step S122, a search is made for a continuous one-character Chinese character clause, and if the result is "none" in step S123, the program returns. If “Yes”,
In step S124, it is checked whether there is a character candidate as a result of character recognition in that part, and if NO, step S12
The process jumps to step S123, searches for the next continuous one-character phrase, and returns to immediately before step S123 to form a loop. If YES in step S124, step S1
In step S125, the second candidate for the recognition character is changed to the first candidate, and in step S126, the several clauses before and after that are again divided into clauses using the two-clause longest matching method. The result is sent to step S12.
In step S7, it is checked whether there are no more consecutive one-character phrases of kanji, and if YES, the process jumps to step S129. If NO, in step S128, the second character recognition candidate is returned to the first candidate, and thereafter the process moves to step S129, but the rest is as described above.

[0043] To summarize, it is assumed that ``character recognition has failed in parts where one-character clauses of kanji are consecutive'', the character candidates are replaced, and then analyzed again. When there are no more consecutive kanji character candidates, the candidate is officially determined.

(Embodiment 4) In this embodiment, another example of post-processing for text as a result of character recognition similar to that in Embodiment 3 will be described.

[0045] In this embodiment, the relationship between the subject and predicate, and the relationship between object and predicate in the text is determined, and if it is determined that there is an incorrect connection, that part is underlined on the display. , is what is said. Although it can be said to be a kind of simple semantic processing of Japanese sentences, it is a general technology used in word processors of various companies on the market to determine homophones for kana-kanji conversion. Usually, "AI conversion",
It is advertised as "example conversion".

FIG. 19 is a flowchart for explaining this semantic processing program. This program is
Character recognition command program 10 explained in FIG. 9 of Example 1
It works by being inserted just before the -4 return. First, in step S130, the connection between subjects and predicates is searched between clauses, and in step S131, if there is an invalid part in the combination rule of subjects and predicates, that part is displayed on the image display 44,
The result is displayed underlined at 45 on the display. Then, in step S132, the connection between the object and predicate is searched between the clauses, and in step S133, if there is an invalid part in the combination rule of the object and predicate, that part is displayed as 44 on the image display.
45 is underlined on the result display, and the program returns.

FIG. 20 is an example of a sentence determined to be invalid by the semantic processing program of FIG. 19.

(Example 5) In this example, when the user who is listening to the audio output thinks that "this is an incorrect sentence" and presses the ESC key on the keyboard, the corresponding text displayed on the display will be displayed. Let's take an example where a sentence is displayed underlined. Figures 21 and 22
shows a speech synthesis program 10-6 modified for this embodiment. For other parts, Example 1
It is fine to be the same as. First, in step S140, it is checked whether the audio output area has been determined. If NO, return from this program; if YES, in step S141, the first sentence of the audio output area is taken out to the audio synthesis buffer 7-3, and in step S142 In step S143, accent information is added to each word using an accent dictionary, and in step S144, the intonation of the entire sentence is adjusted according to intonation rules. Then, step S145
Then, it is checked whether the speech synthesizer 22 is producing a voice, and if the speech has been completed, the process moves to step S146, and the reading, accent, and intonation information is sent to the speech synthesizer 22 to instruct the speech synthesizer 22 to produce a voice. At that time, step S14
In step S7, it is checked whether the user has pushed the ESC key from the keyboard, and if YES, the sentence being uttered is displayed underlined on the image display 44 and result display 45 in step S151. After that, in step S152, the next sentence is taken out to the speech synthesis buffer 7-3, and in step S152, the next sentence is extracted into the speech synthesis buffer 7-3.
At step 153, it is checked whether there is a "next sentence", and if NO, the program returns directly. YES
If so, return to immediately before step S142 and proceed to step S14.
Repeat steps 2 and below. If NO in step S147, it is checked in step S148 whether the speech synthesizer 22 is producing a voice, and if YES, the process returns to immediately before step S147 and the ESC key check is repeated. Step S148
If NO in step S149, the process waits for 2 seconds, and in step S150, the ESC key input from the user is checked. If YES here, an underline display process is performed in step S151, and the process proceeds as described above. If NO in step S150, the process moves to step S152, where the "next sentence" is set, and the rest of the process proceeds as described above.

In step S149, the meaning of waiting 2 seconds is to wait at least 2 seconds between the utterance of one sentence and the utterance of the next sentence, and during that time, if the user determines that the sentence is ``inauthentic,''
If you press the ESC key, the last sentence uttered will be marked as "invalid".
This is to display an underline indicating the . Of course, even if the user is uttering a sentence, if it is determined that the sentence is "incorrect," the user may press the ESC key immediately. In step S147, this determination is made.

[0050]

[Effects of the Invention] As explained above, in a series of character recognition operations, the character recognition process for only one character image, which is the most burdensome part, is controlled separately and handled independently as a separate housing.
By separating all other processes so that they are performed on the main machine as before, (1) the main machine can be executed on a low-cost personal computer or workstation. (2) Recognition speed can also be secured at 60 to 70 characters/second, two to three times faster than conventional technology. (3) In addition, the main machine-side processing, such as a program for reading the type image on paper, a program for cutting out single character images, a program for evaluating and correcting recognition results, a program for displaying text, etc., are also performed by the user. , you can combine commercially available software, or create some of your own, giving you a lot of freedom. (4) By adding a commercially available audio output device and software to the text that is the recognition result and making it speak through a speaker, the character recognition result can be confirmed audibly. It has the effect of saying.

[Brief explanation of drawings]

FIG. 1 is a system configuration diagram of a first embodiment.

FIG. 2 is a system configuration diagram of a character recognition device.

FIG. 3 is a diagram illustrating a format of image data and recognition result data.

FIG. 4 is an exemplary diagram of a screen display.

FIG. 5 is a diagram illustrating a format of result text.

FIG. 6 is a flowchart of the main program.

FIG. 7 is a flowchart of a character recognition program.

FIG. 8 is a flowchart of an image reading program.

FIG. 9 is a flowchart of a character recognition command program.

FIG. 10 is a flowchart of a character extraction program.

FIG. 11 is a flowchart of a post-processing program.

FIG. 12 is a flowchart of a speech synthesis program.

FIG. 13 is a flowchart of a storage program.

FIG. 14 is an explanatory diagram of character cutting processing according to the second embodiment.

FIG. 15 is a first diagram of a flowchart of a character segmentation program according to the second embodiment.

FIG. 16 is a second diagram of a flowchart of a character segmentation program according to the second embodiment.

FIG. 17 is a second exemplary diagram of the resulting text format.

FIG. 18 is a flowchart of a character candidate replacement processing program according to the third embodiment.

FIG. 19 is a flowchart of a semantic processing program according to the fourth embodiment.

FIG. 20 is a diagram illustrating a sentence that is determined to be invalid through semantic processing.

FIG. 21 is a first diagram of a flowchart of a speech synthesis program according to the fifth embodiment.

FIG. 22 is a first diagram of a flowchart of a speech synthesis program according to the fifth embodiment.

Claims (10)

[Claims] 1. An input means for inputting image information, a cutting means for cutting out a character region from the image information, a recognition means for recognizing the image information of the cut out region and deriving it as character information, and a recognition result. A document processing device comprising: a display means for displaying character information; and an audio output means for outputting the character information as sound. 2. The document processing apparatus according to claim 1, further comprising editing means for editing the displayed character information. 3. The document processing apparatus according to claim 1, wherein the cutting means derives a plurality of pieces of character information from the image information to be cut out, together with a frequency representing certainty. 4. The method according to claim 1, further comprising control means for controlling to change the region to be cut out when the frequency representing the certainty of the plurality of derived character information is low. Document processing device. 5. The document processing device includes means for storing the derived plurality of character information, means for converting the character information stored in the storage means into kanji and kana characters, and a means for converting the character information stored in the storage means into kanji and kana characters, 2. The document processing apparatus according to claim 1, further comprising selection means for selecting character information from which a phrase can be obtained. 6. The document processing device includes an instruction means for instructing a change in the recognition result, and when instructed by the instruction means, the document processing device changes character information corresponding to the information outputted by the voice output means. 2. The document processing apparatus according to claim 1, further comprising display control means for adding identification information and controlling the display on said display means. 7. Inputting image information, cutting out a character region from the image information, recognizing the image information of the cut out region, deriving it as character information, and displaying the character information that is the recognition result, A document processing method characterized by outputting the character information as audio. 8. The document processing method according to claim 7, further comprising editing the displayed character information. 9. The document processing method according to claim 7, further comprising deriving a plurality of pieces of character information along with a frequency representing certainty from the extracted image information. 10. The document processing method according to claim 7, further comprising controlling to change the region to be cut out if the frequency representing the certainty of the plurality of derived character information is low.