JP2005352735A - Document file creation support device, document file creation support method, and program thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a document file creation support device that facilitates a check on the result of recognition of character recognition processing. <P>SOLUTION: The document file creation support device 2 identifies a character displayed in a manuscript image based on the manuscript image of the raster data form, and creates an image pattern of a character image based on the identification result. The document file creation support device 2, by applying the created image pattern to the character image of which accuracy of the identification result is a standard value or higher, and applying the character image captured from the manuscript image to the character image of which accuracy of the identification result is below the standard value, creates a character string reproduction image that reproduces a character string contained in the manuscript image, and displays it. In this way, a user can input a character code for the character of which accuracy of the identification result is low while checking the reproduction image of the character string. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a document file creation support apparatus that supports creation of a document file using character recognition processing.

For example, Patent Literature 1 displays both an input image and a character recognition result, and when one image is scrolled, a character recognition device that automatically scrolls the other image to correspond to one image. Is disclosed. Patent Document 2 discloses a character recognition system that displays a document image sequence and a character recognition result sequence at positions facing each other across a reference line. Patent Document 3 discloses an optical character reader that prints a recognition result in the vicinity of a recognition target character. Similarly, Patent Document 4 discloses a character recognition display device that displays a character recognition result and image information based on the character recognition.
Japanese Examined Patent Publication No. 7-079023 Japanese Patent Laid-Open No. 10-021326 JP-A-7-098746 Japanese Patent Laid-Open No. 7-121654

  The present invention has been made from the above-described background, and an object thereof is to provide a document file creation support apparatus that facilitates confirmation of a recognition result of character recognition processing.

[Document file creation support device]
In order to achieve the above object, a document file creation support apparatus according to the present invention includes a character discrimination unit that discriminates characters displayed in an original image based on an original image in a raster data format, and a discrimination by the character discrimination unit. A character string included in the document image using an image pattern generation unit that generates an image pattern of the character image based on the result, a character image cut out from the document image, and an image pattern generated by the character determination unit Reproduction image creating means for creating a character string reproduction image for reproducing the image, and user interface means for displaying the character string reproduction image created by the reproduction image creation means.

  Preferably, the user interface means displays the character string reproduction image in a display mode capable of distinguishing between the character image included in the document image and the character image generated by the image pattern generation means.

  Preferably, the character discriminating unit further discriminates the accuracy of the character discrimination result, and the reproduction image creating unit is configured such that the accuracy of the discrimination result discriminated for any one of the character images by the character discriminating unit is a reference value. In this case, when the image pattern generated by the image pattern generation unit is applied to the portion corresponding to the character image, and the accuracy of the determination result of the character image is smaller than the reference value, the character image The character image cut out from the document image is applied to the portion corresponding to.

  Preferably, the character discriminating unit discriminates at least character identification information of a character displayed in the document image, and the user interface unit performs character identification information on the character image included in the displayed character string reproduction image. The character identification information file corresponding to the character string included in the document image is created based on the character identification information determined by the character determination means and the character identification information received by the user interface means. And a character string file creating means.

  Preferably, the user interface means skips a cursor position indicating an input target of character identification information in the character string reproduction image, depending on whether or not the character image is cut out from the document image.

[Document file creation support method]
The document file creation support method according to the present invention discriminates characters displayed in a manuscript image based on the manuscript image in the raster data format, generates an image pattern of the character image based on the discrimination result, and A character string reproduction image that reproduces the character string included in the document image is created using the character image cut out from the image and the generated image pattern, and the created character string reproduction image is displayed.

[program]
According to another aspect of the invention, there is provided a program file creation support apparatus including a computer for determining a character displayed in an original image based on an original image in a raster data format, and for determining a character image based on the determination result. A step of generating an image pattern, a step of generating a character string reproduction image that reproduces a character string included in the document image using the character image cut out from the document and the generated image pattern, and Causing the computer of the document file creation support apparatus to execute the step of displaying the character string reproduction image.

  According to the document file creation support apparatus of the present invention, the recognition result of the character recognition process can be easily confirmed.

First, in order to help understanding of the present invention, its background and outline will be described.
Like OCR (Optical Character Reader) or the like, image data (original image) is optically read from a document, and characters are recognized by extracting characters from image data (raster data format) of the read document image. Technology has been proposed. This character recognition process is performed by pattern matching or the like, but has not yet achieved 100% accuracy. Therefore, it is necessary for the user to confirm and calibrate the recognition result of the character recognition process.

FIG. 1 is a diagram illustrating a technique that facilitates confirmation of a recognition result of character recognition processing.
In order to facilitate confirmation of the recognition result in the character recognition process, as illustrated in FIG. 1A, a document image read from a document and a character image recognized based on the document image (recognition result) Can be displayed side by side and scrolled simultaneously. In this case, the user performs confirmation and proofreading while comparing the document image and the recognition result, but the operation of comparing these is not easy. Also, when the recognition accuracy of the character recognition process is extremely low, it may be faster for the user to input directly while looking at the document.
In addition, as illustrated in FIG. 1B, a method of displaying a line of a character string included in a document image and a recognition result corresponding to the line at a position facing each other across a reference line is also conceivable. In this case, the user can easily recognize the row to be compared, but cannot view other regions (for example, the next row) of the document image.
Further, as illustrated in FIG. 1C, a method of inserting and displaying the recognition result in the vicinity of the character image included in the document image is also conceivable. In this case, an area where the recognition result can be inserted does not always exist on the document image.

  As described above, when the document image and the recognition result are displayed side by side, it is not easy to compare the document image and the recognition result, and when the recognition accuracy of the character recognition process is extremely low, the user can It may be faster to input directly while watching. Here, the recognition accuracy of the character recognition processing is information indicating the accuracy of the character recognition processing, and is, for example, the degree of matching (number of matching pixels, distance, etc.) in pattern matching.

Therefore, the document file creation support apparatus 2 according to the present embodiment applies a character image cut out from the document image or a character image generated based on the recognition result according to the recognition accuracy of the character recognition process. Reproduce the character string included in the original image.
FIG. 2 is a diagram illustrating a reproduced image displayed by the document file creation support apparatus 2.
As illustrated in FIG. 2, the document file creation support apparatus 2 uses a character image generated based on the recognition result (an image pattern or font described later) for a character image whose recognition accuracy of the character recognition process is equal to or higher than a reference value. For a character image whose recognition accuracy is less than the reference value, a character image cut out from the document image is applied to display a reproduced image in which a character string included in the document image is reproduced.
The user can check whether there is an error in the recognition result based on the relationship with the preceding and succeeding characters for the portion with high recognition accuracy, and the character image itself cut out from the original image for the portion with low recognition accuracy. You can enter the character code directly while watching. That is, the user refers to the reproduced image displayed on the document file creation support apparatus 2, thereby confirming the recognition result and complementing the recognition result without comparing the original image with the recognition result ( Calibration work).

[Hardware configuration]
Next, the hardware configuration of the document file creation support apparatus 2 will be described.
FIG. 3 is a diagram illustrating a hardware configuration of the document file creation support apparatus 2 to which the document file creation support method according to the present invention is applied, centering on the control apparatus 20.
As illustrated in FIG. 3, the document file creation support 2 includes a control device 20 including a CPU 202 and a memory 204, a communication device 22, a recording device 24 such as an HDD / CD device, an LCD display device or a CRT display device, and the like. It comprises a user interface device (UI device) 26 including a keyboard / touch panel.
The document file creation support device 2 is, for example, a general-purpose computer in which a document file creation program 5 (described later) is installed, and acquires image data (raster data format) of a document image via the communication device 22 or the recording device 24. Then, character recognition processing is performed based on the acquired image data of the document image, and a document file including the recognition result (character code or the like) of the character recognition processing is created. For example, the document file creation support apparatus 2 is connected to a multifunction device 10 having a printer function, a scanner function, and the like. From this multifunction device 10, image data of a document image optically read from a document by a scanner function. And a document file composed of character codes and the like is created based on the acquired image data of the original image.

[Document file creation program]
FIG. 4 is a diagram illustrating a functional configuration of the document file creation program 5 that is executed by the control device 20 (FIG. 3) and implements the document file creation support method according to the present invention.
As illustrated in FIG. 4, the document file creation program 5 includes a character recognition unit 40, an image dictionary creation unit 50, an encoding unit 60, a reproduction image creation unit 70, a user interface control unit (UI control unit) 80, and a code file. A creation unit 90 is included.
In the document file creation program 5, the character recognition unit 40 includes image data of a document image read by the scanner function of the multifunction machine 10 or image data of a document image acquired via the communication device 22 or the recording device 24. (Raster data format) is acquired, and character recognition processing is performed on the image data of the acquired document image. For example, the character recognizing unit 40 performs pattern matching by comparing a template image of a character prepared in advance with an image (partial image) included in a document image, and character identification information of the most matching character. The font information, the character area information of this character, and the recognition accuracy of the character recognition process are determined, and the determination result is output to the image dictionary creation unit 50. Here, the character identification information is information for identifying a character, for example, a versatile character code (such as an ASCII code or a shift JIS code). The character area information is information indicating the area of the character image in the document image, and is, for example, character layout information including the position, size, range, or combination of the character images. The font information is information that defines the shape, size, color, etc. of the font image, and the font type (gothic, italic, mincho, etc.), font size (number of points), and font The color etc. are included.
The character recognition unit 40 may perform character recognition processing based on the image pattern generated by the image dictionary creation unit 50 (described later). For example, the character recognition unit 40 performs pattern matching using an image pattern generated based on a document image, and recognizes a character included in the document image.

  The image dictionary creation unit 50 creates an image pattern of characters that typically appear in the document image based on the result of the character recognition process input from the character recognition unit 40 and the image data of the document image. For example, the image dictionary creation unit 50 cuts out a character image from the document image based on the character identification information and the character region information input from the character recognition unit 40, and creates an image pattern based on the cut out character image. Then, an index is assigned to the created image pattern (character image), and the image pattern and the index are output to the encoding unit 60 as an image dictionary.

The encoding unit 60 compresses the image data of the document image based on the image dictionary input from the image dictionary creation unit 50, and stores the compressed image data of the document image and the image dictionary in the recording device 24 (FIG. 3) or The data is output to the multifunction machine 10 (FIG. 3). More specifically, the encoding unit 60 compares the image pattern registered in the image dictionary with a character image included in the document image (a character image whose character recognition processing recognition accuracy is equal to or higher than a reference value). Then, the character image data matching any one of the image patterns is replaced with an index corresponding to the image pattern and the position information of the character image, and compressed. Here, the term “match” does not mean a complete match, but includes a case where a partial match is made within a predetermined allowable range. Furthermore, the encoding unit 60 may encode the index and position information replaced with the character image, the image dictionary, and the like by entropy encoding (Huffman encoding, arithmetic encoding, LZ encoding, or the like).
The encoding unit 60 compresses the image data of the character image as it is by using another encoding method (MH, MMR, etc.) suitable for the raster data for the character image whose recognition accuracy of the character recognition processing is less than the reference value. To do. The encoding unit 60 also encodes images that are not subject to character recognition processing (for example, photographic images, CG images, etc.) by applying other encoding methods (JPEG, MH, MMR, etc.).

The reproduction image creation unit 70 creates a reproduction image corresponding to the document image using the character image cut out from the document image and the image pattern generated by the image dictionary creation unit 50. Specifically, the reproduction image creation unit 70 uses the image data of the document image input from the encoding unit 60 and replaces the portion replaced with the index and position information (that is, the character whose recognition accuracy is equal to or higher than the reference value). The image is reproduced with an image pattern corresponding to this index, and the portion that is not replaced with the index and position information (that is, a character image whose recognition accuracy is less than the reference value and compressed by another encoding method) 1) is reproduced by decoding the image data of this portion. In other words, the reproduced image creation unit 70 applies the most approximate image pattern created by the image dictionary creation unit 50 to a character image whose recognition accuracy in the character recognition process is greater than or equal to the reference value, and performs the character recognition process. For a character image whose recognition accuracy is less than the reference value, a character image itself cut out from the document image is applied to generate image data of a reproduced image that reproduces the document image.
The reproduced image creating unit 70 also applies an image cut out from the document image to an image that is not subject to character recognition processing.

The UI control unit 80 controls the UI device 26 (FIG. 3) to display the reproduced image created by the reproduced image creating unit 70.
In addition, the UI control unit 80 receives a calibration operation of character identification information for the reproduced image from the user via the UI device 26, and causes the reproduced image creation unit 70 and the code file creation unit 90 to respond to the calibration operation. Instruct. The reproduced image creating unit 70 changes the reproduced image in response to this instruction.

  Based on the recognition result of the character recognition process by the character recognition unit 40 and the user's proofreading operation input to the UI control unit 80, the code file creation unit 90 generates a document file (character code, font) corresponding to the document image. Code file consisting of information etc.). Specifically, the code file creation unit 90 uses the calibration results input to the UI control unit 80 based on all recognition results (all character codes, etc.) of the character recognition processing by the character recognition unit 40 regardless of the recognition accuracy. The recognition result is corrected (replacement, deletion, addition, etc.) according to the operation.

FIG. 5 is a diagram for explaining the function of the image dictionary creation unit 50 in more detail.
As shown in FIG. 5, the image dictionary creation unit 50 includes a storage unit 500, a character image extraction unit 510, a match determination unit 520, a dictionary determination unit 530, a position correction unit 540, and an index assignment unit 550.
The storage unit 500 controls the memory 204 (FIG. 3) and the recording device 24 (FIG. 3) to store the document image, character identification information, and character area information input from the character recognition unit 40. Hereinafter, a character code will be described as a specific example of character identification information, and character position information will be described as a specific example of character region information.

  Character image extraction unit 510 cuts out a character image from a document image based on character position information. That is, the character image extraction unit 510 extracts an area indicated by the character area information from the document image as a character image. The extracted character image is an area determined by the character recognition unit 40 to be a character image. Note that, when the character image is cut out from the document image in the character recognition process, the image dictionary creation unit 50 may apply the character image cut out in the character recognition process as it is.

The coincidence determination unit 520 compares the character image cut out from the document image with the image pattern registered in the image dictionary, and determines the degree of coincidence thereof. Here, the degree of coincidence is information indicating the degree to which a plurality of images coincide with each other. For example, when binary images are compared, the number of pixels that overlap each other when the two images are overlaid (hereinafter referred to as the number of pixels). , The number of matching pixels), the matching pixel ratio obtained by normalizing the number of matching pixels (for example, the number of matching pixels divided by the total number of pixels), or the pixel distribution (histogram) when a plurality of images are superimposed is there.
The coincidence determination unit 520 determines the degree of coincidence by comparing the character image cut out from the document image and the image pattern registered in the image dictionary at a plurality of relative positions. That is, the coincidence determination unit 520 compares the clipped character image and the image pattern registered in the image dictionary while changing (shifting) their relative positions in order to calculate the maximum degree of coincidence. .
For example, the matching determination unit 520 shifts the matching pixel rate while shifting the character image cut out from the document image and the image pattern in which the character image and the character code (or the combination of the character code and the font information) match each other. The maximum value of the coincidence pixel ratio and the shift vector when the maximum is obtained are output to the storage unit 500.

  The dictionary determination unit 530 determines an image pattern to be registered in the image dictionary based on the character image cut out by the character image extraction unit 510 and the matching degree determined by the matching determination unit 520. For example, the dictionary determination unit 530 selects a plurality of character images having a matching degree equal to or higher than a reference value, and sets a common shape of these character images as an image pattern. In other words, the dictionary determination unit 530 associates character images having similar shapes with each other via the image pattern.

The position correction unit 540 corrects the position information of the character image based on the shift vector output from the match determination unit 520. That is, the position correction unit 540 corrects the position information of the character image input from the character recognition unit 40 so that the degree of coincidence between the character image and the image pattern is maximized.
The index assigning unit 550 assigns indexes for identifying these image patterns to the image patterns registered in the image dictionary, and stores the assigned indexes, image patterns, and character codes in association with each other as an image dictionary. To the unit 500.

FIG. 6 is a diagram illustrating an image dictionary 902 created by the image dictionary creation unit 50. For convenience, the case where the image data of the document image is binary data will be described below as a specific example.
As illustrated in FIG. 6, the image dictionary creation unit 50 includes an image pattern generated by the dictionary determination unit 530 (FIG. 5), a character code (character identification information) corresponding to the image pattern, and the image pattern. The assigned index is associated with each other to form an image dictionary 902. The character code is determined by the character recognition unit 40, and the image pattern is generated based on the character image classified by the character code.
In this example, a plurality of image patterns (“file 014” and “file 031”) are associated with the same character code (“0x42”). This is because the dictionary determination unit 530 corresponds to each character image when the character code is matched but the shape of each character image is too different (for example, when the font type or font size is different). This is because an image pattern to be generated is generated.

FIG. 7 is a diagram for explaining the function of the encoding unit 60 in more detail.
As illustrated in FIG. 7, the encoding unit 60 includes a pattern determination unit 610, a position information encoding unit 620, an index encoding unit 630, an image encoding unit 640, a dictionary encoding unit 650, a selection unit 660, and a code output unit. 670.
The pattern determination unit 610 compares each image pattern registered in the image dictionary with a partial image included in the document image, and determines an image pattern (same or similar image pattern) corresponding to the partial image. Specifically, the pattern determination unit 610 superimposes the partial image (corrected by the position correction unit 540) cut out from the original image in units of character images and the image pattern, and matches the determination unit 520 ( The degree of coincidence is calculated by the same method as in FIG. 5), and it is determined whether or not it is compatible based on whether or not the calculated degree of coincidence is greater than or equal to an allowable value.
When the corresponding image pattern is found, the pattern determination unit 610 outputs the position information of the partial image to the position information encoding unit 620, and the index of the image pattern is output to the index encoding unit 630. Output. Further, the pattern determination unit 610 outputs the image data of this partial image to the image encoding unit 640 regardless of whether or not a corresponding image pattern is found.
Note that the pattern determination unit 610 in this embodiment creates an image dictionary based on the index of the image pattern that matches the character image cut out from the document image and the position information of the character image (corrected by the position correction unit 540). For the partial image cut out as a character image for acquisition from the unit 50, the index and position information input from the image dictionary creation unit 50 are output to the index encoding unit 630 and the position information encoding unit 620, respectively.

The position information encoding unit 620 encodes the position information input from the pattern determination unit 610 (that is, position information of the partial image (character image) corrected by the position correction unit 540) and outputs the encoded position information to the selection unit 660. To do. For example, the position information encoding unit 620 encodes position information by applying LZ encoding or arithmetic encoding.
The index encoding unit 630 encodes the index input from the pattern determination unit 610 and outputs the encoded index to the selection unit 660. For example, the index encoding unit 630 assigns a code having a different code length to each index according to the appearance frequency of the index.
The image encoding unit 640 encodes the partial image input from the pattern determination unit 610 by applying an encoding method (JPEG, MH, MMR, etc.) suitable for the image (rasterized image data), and selects the selection unit To 660.
The dictionary encoding unit 650 encodes at least a part of the image dictionary input from the image dictionary creation unit 50 (FIGS. 4 and 5) and outputs the encoded image dictionary to the code output unit 670. For example, the dictionary encoding unit 650 encodes an image pattern (raster data) included in the image dictionary 902 by an encoding method suitable for the image.

The selection unit 660 selects either the code data input from the position information encoding unit 620 and the index encoding unit 630 or the code data input from the image encoding unit 640 according to the recognition accuracy of the character recognition process. One is selected and the selected code data is output to the code output unit 670. Specifically, for the character image whose recognition accuracy is equal to or higher than the reference value, the selection unit 660 includes the code data of the position information input from the position information encoding unit 620 and the index input from the index encoding unit 630. And the code data of the partial image encoded by the image encoding unit 640 is output as a code for a character image whose recognition accuracy is less than the reference value. Output to the unit 670.
The code output unit 670 includes code data (position information, index, and partial image code data) input from the selection unit 660, code data (image dictionary code data) input from the dictionary encoding unit 650, characters The correspondence table 904 is associated with each other and output to the reproduced image creation unit 70 and the recording device 24 (FIG. 3).

FIG. 8 is a diagram illustrating a reproduction image 260 displayed on the UI device 26.
As illustrated in FIG. 8, the UI control unit 80 (FIG. 4) displays a reproduced image 260 obtained by combining an image cut out from an original image and an image pattern on the monitor of the UI device 26 (FIG. 3). . Further, the reproduced image 260 displays an underline 262 for distinguishing between an image cut out from the document image and an image pattern, and a cursor 264 indicating a user input position.
The underline 262 in this example is displayed in the vicinity of the character image cut out from the document image, and has a function of notifying the user of a character image whose recognition accuracy is less than the reference value. In this example, a character image cut out from the document image by the underline 262 (that is, a character image whose recognition accuracy is less than the reference value) and an image pattern (that is, a character image whose recognition accuracy is greater than or equal to the reference value). However, the present invention is not limited to this. For example, the color (density) of the character image may be different, or the color around the character image may be different.
In addition, the cursor 264 of this example is displayed at a position (specifically, near the lower part) associated with the character image to be input, and when the user performs an operation to change the input target, the recognition accuracy is the reference. A character image (that is, an image pattern) that is greater than or equal to the value is skipped, and only an area corresponding to a character image (that is, a character image cut out from the document image) whose recognition accuracy is less than the reference value is moved.
Thereby, the user can input a character code etc. only with respect to the character image with low recognition accuracy of character recognition processing.

[Reproduced image display operation]
Next, the creation processing and display processing of the reproduction image 260 (FIG. 8) by the document file creation support apparatus 2 will be described.
FIG. 9 is a flowchart showing the overall operation of the reproduction image display process (S10). For convenience of explanation, a case where binary image data is input is taken as a specific example.
As shown in FIG. 9, in step 100 (S100), the multi function device 10 (FIG. 3) optically reads a document image from a document, and supports document file creation using image data (raster data) of the read document image. Transmit to device 2 (FIG. 3).
In step 102 (S102), the character recognition unit 40 (FIG. 4) of the document file creation support apparatus 2 receives image data (binary raster data) of a document image from the multifunction machine 10 (FIG. 3). Character recognition processing is performed on the input image data, the character code of the character image included in the document image, position information, and recognition accuracy of the character recognition processing are determined for each character, and the determined character code and position are determined. Information and recognition accuracy are output to the image dictionary creation unit 50.

In step 104 (S104), the storage unit 500 (FIG. 5) of the image dictionary creation unit 50 stores the document image (binary), character code, position information, and recognition accuracy input from the character recognition unit 40 in the memory 204 (FIG. 5). 3) Remember.
The character image extraction unit 510 (FIG. 5) cuts out a character image from the document image based on the input position information and outputs it to the storage unit 500.

  In step 106 (S106), the dictionary determining unit 530 reads out the image pattern and the index of the image pattern from the image dictionary 902 (FIG. 6) based on the character code of the character image cut out sequentially from the document image. Specifically, the storage unit 500 associates the already determined image pattern, the index assigned to the image pattern, and the character code of the character image corresponding to the image pattern as an image dictionary 902. I remember it. The dictionary determination unit 530 acquires the character code of the character image newly cut out by the character image extraction unit 510 from the character recognition unit 40, and reads out the image pattern and index corresponding to the acquired character code from the image dictionary 902. When the processing target is the first character image cut out from the document image (that is, when no image pattern is registered in the image dictionary 902), this character image is registered in the image dictionary 902 as it is. become.

In step 108 (S108), the coincidence determination unit 520 compares the character images sequentially extracted by the character image extraction unit 510 with the image patterns registered in the image dictionary 902, and determines the degree of coincidence at a plurality of relative positions. judge. Specifically, the coincidence determination unit 520 calculates the coincidence pixel count K of black pixels while shifting the image pattern read by the dictionary determination unit 530 and the extracted character image from each other.
The coincidence pixel number K is a position vector indicating a relative position in the image, x is a black pixel distribution of the image pattern, S (x), a character image number cut out in order is i (1 to N), When the distribution of black pixels is P (i, x) and the shift vector of the character image i is vi, it is calculated by the following formula.
(Number of matched pixels K) = Σ {S (x) * P (i, x−vi)}
Note that “Σ” indicates the total sum for the variable x.
Next, the coincidence determination unit 520 normalizes the calculated coincidence pixel number K to calculate a coincidence pixel rate K ′.
The coincidence pixel rate K ′ is calculated by the following equation, where M is the number of pixels constituting the character image.
(Matched pixel rate K ′) = K / M

In step 110 (S110), the dictionary determination unit 530 converts the image pattern determined by the match determination unit 520 and the character image (newly cut out) into a newly cut out character image. Based on this, it is determined whether or not to register an image pattern. Specifically, when the determined matching pixel ratio K ′ is equal to or greater than the reference value, the dictionary determination unit 530 determines that the index of the image pattern having the largest matching pixel ratio K ′ and the cut character image are mutually connected. Correspondingly, the process proceeds to S114, and when the determined matching pixel ratio K ′ is smaller than the reference value, the process proceeds to S112.
That is, if the matching degree is equal to or greater than the reference value, the dictionary determining unit 530 prohibits registration of a new image pattern based on the character image, associates the character image with the registered image pattern, and matches the character image. When the degree is smaller than the reference value, this character image is newly registered in the image dictionary 902 as an image pattern.

In step 112 (S112), the dictionary determining unit 530 registers the extracted character image as an image pattern in the image dictionary 902, and associates the character image with the image pattern.
The index assigning unit 550 assigns identification information (index) for uniquely identifying the image pattern to the image pattern determined by the dictionary determining unit 530. Then, the index assigning unit 550 registers the assigned index in the image dictionary 902 in association with the image pattern. The assigned index uniquely identifies each image pattern in at least the original image.
The index assigned to the character image and the position information of the character image are sequentially output to the encoding unit 60 as encoding target data.

In step 114 (S114), the position correction unit 540 matches the degree of matching calculated by the matching determination unit 520 at a plurality of relative positions for the character image cut out from the document image and the image pattern registered in the image dictionary 902. Based on the matching pixel count K or matching pixel rate K ′), the correction vector of the position information input from the character recognition unit 410 is determined. Specifically, in S108, the position correction unit 540 sets the shift vector vi when the number of coincident pixels K calculated for the extracted character image and image pattern is the maximum as the correction vector.
That is, the image dictionary creation unit 50 sets the cutout position (character image position information) of the character image to be processed so that the character image to be processed and the image pattern corresponding to the character image most closely match. to correct.
The position information of the character image corrected by the correction vector is sequentially output to the encoding unit 60 as encoding target data together with the index of the image pattern corresponding to the character image.

In step 116 (S116), the encoding unit 60 applies an encoding method based on the image dictionary 902 to a character image whose recognition accuracy is equal to or higher than a reference value, performs an encoding process, and the recognition accuracy is the reference accuracy. Encoding processing is performed by applying another encoding method that does not use the image dictionary 902 to a character image that is less than the value. Specifically, when the recognition accuracy of the input character image is equal to or higher than the reference value, the encoding unit 60 uses the index corresponding to the character image and the position of the character image instead of the image data of the character image. When the information (that is, the encoding target data) is encoded and the recognition accuracy of the input character image is less than the reference value, the image data of the character image is encoded.
The encoded image data of the document image is output together with the image dictionary 902 to the reproduction image creation unit 70 (FIG. 4) and the recording device 24 (FIG. 3).

In step 118 (S118), the document file creation program 5 determines whether or not the processing from S102 to S116 has been completed for all character images included in the document image, and has been completed for all character images. If not, the process proceeds to S120. Otherwise, the process returns to S102, the next character image is cut out, and the processes from S104 to S116 are repeated.
As a result, the character image included in the document image is associated with one of the image patterns registered in the image dictionary 902. In other words, character images whose shapes are approximated are associated with each other via an image pattern (index).

  In step 120 (S120), the reproduction image creation unit 70 creates a reproduction image 260 by decoding the code data of the document image input from the encoding unit 60. That is, the reproduction image creating unit 70 replaces a character image whose recognition accuracy is equal to or higher than a reference value in the image data of the document image with an image pattern corresponding to the character image (an image pattern having the maximum number of matching pixels). Thus, a reproduced image 260 (FIG. 8) is created. This reference value is desirably a value that can be used to determine that the result of the character recognition process is reliable, and is set by a user, for example.

In step 122 (S122), the UI control unit 80 displays the reproduced image 260 created by the reproduced image creating unit 70 on the monitor of the UI device 26 (FIG. 3). At that time, the UI control unit 80 displays an underline 262 (FIG. 7) so that the character image cut out from the document image and the image pattern can be distinguished from each other, and a cursor 264 (FIG. 7) indicating the input position of the user. FIG. 7) is displayed in the vicinity of the character image cut out from the document image.
The code file creation unit 90 sets the result of the character recognition process (that is, the character code group) by the character recognition unit 40 as a provisional code file.

[Proofreading]
Next, the proofreading process of the code file (character recognition process result) will be described.
FIG. 10 is a flowchart showing the proofreading process (S20) of the character recognition result.
As shown in FIG. 10, in step 200 (S200), the UI control unit 80 (FIG. 4) displays the reproduced image 260 created by the reproduced image creating unit 70 on the monitor of the UI device 26 (FIG. 3). . The reproduction image 260 includes a character image cut out from a document image (corresponding to a character image with low recognition accuracy), an image pattern (corresponding to a character image with high recognition accuracy), and a character image cut out from the document image. An underline 262 (FIG. 7) arranged in the vicinity is included.
At the present stage, the result of the character recognition processing by the character recognition unit 40 (that is, the character code group) is held as it is as a temporary code file.

In step 202 (S202), the UI control unit 80 sequentially searches the character images included in the reproduced image 260 for character images whose recognition accuracy is less than the reference value (that is, character images cut out from the document image). To do.
In step 204 (S204), the document file creation program 5 proceeds to the process of S206 when a character image whose recognition accuracy is less than the reference value is found, and when a character image less than the reference value is not found. Then, the fact that the calibration process is completed is displayed, and the process proceeds to S220.

In step 206 (S206), the UI control unit 80 displays a cursor 264 (FIG. 7) in the vicinity of the found character image (that is, a character image whose recognition accuracy is less than the reference value), and this character image ( Hereinafter, input of a character code for the input target character) is accepted.
In step 208 (S208), the document file creation program 5 proceeds to the process of S210 when the character code is input, and proceeds to the process of S216 when the character code is not input.

In step 210 (S210), when a character code is input from the user via the UI device 26 (FIG. 3), the UI control unit 80 converts the character code into an input target corresponding to the display position of the cursor 264. The image is output to the encoding unit 60 via the reproduction image creation unit 70 in association with the character.
The encoding unit 60 specifies an index corresponding to the input target character, and specifies another character image associated with the specified index. Then, the encoding unit 60 updates the code data of the document image by replacing the image data of the other character image having the same input target character and index with the position information of the index and each character image. That is, when a character code is input for any character image (with low recognition accuracy), the encoding unit 60 receives this character image and another character image corresponding to the same image pattern as this character image. The compression processing based on the image dictionary 902 is applied on the assumption that the recognition accuracy is equal to or higher than a reference value (100%) for a group of character images that are very similar to each other. In the image dictionary 902, the character code corresponding to this index is replaced with the input character code.

  In step 212 (S212), the code file creation unit 90 updates the code file so that the input character code is applied to the input target character and another character image whose index matches the input target character.

  In step 214 (S214), the reproduction image creation unit 70 creates the reproduction image 260 again based on the code data of the document image updated by the encoding unit 60. The reproduced image 260 to be created is an image in which an input target character and another character image whose index matches this are replaced with an image pattern. That is, a character image to which a character code is input and a character image (a character image group having a matching index) that is very similar in shape to the character image are represented by a corresponding image pattern in the reproduced image 260. .

In step 216 (S216), the UI control unit 80 receives a movement operation of the cursor 264 from the user via the UI device 26 (FIG. 3).
When the UI control unit 80 accepts the movement operation of the cursor 264, the UI control unit 80 proceeds to the processing of S202, searches for a character image whose recognition accuracy is less than the reference value, and moves the cursor 264 to the vicinity of the next input target character. In other cases, the process proceeds to S218.

In step 218 (S218), the UI control unit 80 accepts a calibration processing end operation from the user via the UI device 26 (FIG. 3).
When accepting the end operation, the UI control unit 80 proceeds to the process of S220, and otherwise returns to the process of S208 and waits for the character code input operation (S208) or the cursor movement operation (S216). .

In step 220 (S220), the code file creation unit 90 stores the code file updated according to the user's input in the recording device 24 (FIG. 3) or the like, and the document file creation program 5 ends the calibration process. To do.
Note that the document file creation support apparatus 2 of this example, when a character code is input by a user for any character image, compresses the image data of this character image by replacing it with an index and position information. However, the present invention is not limited to this. For example, the image data of the character image may be replaced with the input character code itself to improve the compression rate.

As described above, the document file creation support apparatus 2 according to the present embodiment compares character images included in a document image with each other, classifies these character images based on the degree of matching of the shapes, and classifies the characters. An image pattern is generated based on the image. Then, the document file creation support apparatus 2 applies an image pattern corresponding to these character images to a character image whose recognition accuracy of the character recognition process is greater than or equal to the reference value, and the character image whose recognition accuracy is less than the reference value. For the above, the reproduction image 260 is created by applying the character image itself included in the document image. Thus, the user can directly input a character code by visually recognizing the character image included in the original image for a portion with low recognition accuracy of the character recognition result, and replace the portion with high recognition accuracy with an image pattern. Thus, the image data of the document image can be encoded at a high compression rate.
In the reproduced image 260, since the image pattern and the character image cut out from the document image are displayed in a distinguishable display form, the user can easily distinguish between characters with high recognition accuracy and characters with low recognition accuracy. can do.
In the reproduced image 260, the cursor 264 moves only at a position corresponding to a character whose recognition accuracy is less than the reference value, so that the user can easily input a character code for a character with low recognition accuracy.
Also, for character image groups whose recognition accuracy is less than the reference value, character images whose shapes are very similar to each other are associated through an index (image pattern), so a character code is input for any character image. Then, the character code is calibrated together with another character image having a very similar shape.

[Modification]
Next, a modification of the above embodiment will be described. In the above embodiment, the pattern applied in the character recognition process by the character recognition unit 40 is independent of the image pattern created by the image dictionary creation unit 50, but may be a form dependent on each other. For example, the image dictionary creation unit 50 registers, in the image dictionary 902, as an image pattern, a template image (for example, a font image) applied by the character recognition unit 40 that is closest to the character image included in the document image. May be. In addition, the character recognition unit 40 may use the image pattern created by the image dictionary creation unit 50 as a template image in the character recognition process.
When an image pattern registered in the image dictionary 902 is used as a template image for character recognition processing, it is desirable that each image pattern is associated not only with a character code but also with various font information.
FIG. 11 is a diagram illustrating an image dictionary 904 in which character codes and font information are registered.
As illustrated in FIG. 11, the image dictionary 904 in the present modified example has character codes and font information of character images corresponding to the image patterns in association with the image patterns and indexes. In this example, the font information is the type of font and the font size, but may include the color of the font. In this case, the code file may include not only the character code but also font information.
Further, when this image pattern is applied as a template image for character recognition processing, the font information of the character image can be determined by specifying the font information corresponding to the image pattern. Furthermore, by further feeding back the image pattern registered in the image dictionary 904 to the character recognition process, a further improvement in the compression rate can be expected.

  In the above embodiment, the image data of the document image is compressed before the reproduction image is created. However, the present invention is not limited to this. For example, the document file creation support apparatus 2 performs the compression process (that is, If the character images whose shapes are similar to each other are associated with each other via the image pattern (or index) without performing the replacement of the character image with the index and the position information), a reproduction image using the image pattern is created. In addition, it is possible to realize batch calibration of approximate character image groups.

  In the above embodiment, the document file creation support apparatus 2 includes a character image (image pattern) having a recognition accuracy equal to or higher than a reference value, and a character image (cut out from a document image) having a recognition accuracy lower than the reference value. However, the present invention is not limited to this. For example, the recognition accuracy is divided into three stages, and the recognition accuracy corresponds to the highest classification. The image is displayed as an image pattern and excluded from the object of the proofreading process. The character image whose recognition accuracy corresponds to the lowest classification is displayed as a character image cut out from the original image and is used as the object of the proofreading process. The character image corresponding to the middle category other than the above is calibrated by displaying the character image cut out from the original image and the font image corresponding to the character recognition result. Receiving an input of a character code as the management of the target. When the recognition accuracy is classified into three levels, the document file creation support apparatus 2 displays the character image corresponding to each classification in a color corresponding to the classification and identifies the recognition accuracy of each character image. It may be possible. Further, the boundary value (reference value) of the classification of the recognition accuracy may be changeable according to a user input.

Next, a modified example regarding the method of creating the image dictionary 902 will be described.
In the above embodiment, the image dictionary creation unit 50 sequentially cuts out character images from the document image, and sequentially creates an image dictionary based on the cut out character images. However, the present invention is not limited to this. For example, the image dictionary may be created based on the entire original image such as one page or one document. In addition, the image dictionary creation unit 50 may select a plurality of character images having similar shapes in the document image, and create an image pattern to be registered in the image dictionary based on the selected plurality of character images.
Therefore, the image dictionary creation device 50 according to the present modification classifies the character image cut out from the document image based on the character code or the combination of the character code and the font information, and the classified character image according to the appearance frequency. An image pattern to be registered in the image dictionary is created by integration. A plurality of image patterns may be created from character images classified by the same character code.
As a result, the image dictionary creation unit 50 can create an image dictionary in consideration of the appearance frequency of character images and the like, so that a high compression rate can be realized.

FIG. 12 is a diagram schematically illustrating the first image pattern creation process in the modification. In this figure, an example in which a binary document image is input and character images cut out from the document image are classified by character codes will be described as a specific example.
As shown in FIG. 12, the image dictionary creation unit 50 classifies the character images included in the input image into a plurality of character image groups based on the character codes, and the black pixel distribution probability Q ′ for each character image group. (X) is calculated. The calculated distribution probability Q ′ (x) indicates a different numerical value depending on the pixel position x, as illustrated in FIG. This is because character images having different shapes are mixed with different appearance frequencies in the classified character image group.
The distribution probability Q ′ (x) is calculated by the following equation.
Q (x) = P (1, x) + P (2, x−v2) +... + P (i−1, x−v (i−1))
Q ′ (x) = Q (x) / N
Q (x): Pixel distribution of the classified character image group, P (i, x): Black pixel distribution of each character image, x: Position vector, i: Each character image belonging to the character image group (1 to N: N is the number of character images belonging to the character image group)
When i = 1, Q (x) = P (1, x).

Next, the image dictionary creation unit 50 performs threshold processing on the distribution probability Q ′ (x) to extract a typical shape (sum coupling pattern Q ″ (x)) having a high appearance frequency. The probability Q ′ (x) is subjected to threshold processing with the threshold B, so that the difference shape of the character image with a low appearance frequency (difference from a typical shape with a high appearance frequency), the noise portion, and the like are excluded, Only typical shapes are extracted.
The sum coupling pattern Q ″ (x) is calculated by the following conditional expression.
When Q ′ (x)> threshold B, Q ″ (x) = 1
In other cases, Q ″ (x) = 0

  Subsequently, the image dictionary creation unit 50 extracts a common part between the extracted sum combination pattern Q ″ (x) and each character image belonging to the character image group as an image pattern to be registered in the image dictionary. The image dictionary creation unit 50 multiplies the sum coupling pattern Q ″ (x) by the pixel distribution P (i, x−vi) of each character image. As a result, the image pattern # 1 and the image pattern # 2 of the typical character images (high appearance frequency) existing in the character image group are extracted.

Next, the second image pattern creation process will be described.
In the second image pattern creation process, the image dictionary creation unit 50 classifies the character images cut out from the document image based on the character code or a combination of the character code and the font information, and classifies the character images and their character images. Based on the appearance frequency, common shapes and differential shapes in the classified character image group are extracted, and the extracted common shapes and differential shapes are hierarchized and registered in the image dictionary. Here, the common shape is a shape that exists in common in the character image group classified by the character code or the like, and the difference shape is a difference between each character image belonging to this character image group and the common shape. That is, the appearance frequency is equal to or higher than a reference value.

FIG. 13 is a diagram schematically illustrating the second image pattern creation process. In this figure, a specific example will be described in which a binary document image is input, and character images cut out from the document image are classified by character codes.
First, as described above, the image dictionary creation unit 50 classifies the character images included in the document image into a plurality of character image groups based on the character codes, and the black pixel distribution probability Q ′ (x ) Is calculated. The calculated distribution probability Q ′ (x) shows a different value depending on the pixel position x, as shown in FIG. Of these, the portion with the highest distribution probability is considered to have a shape common to character images belonging to this character image group (that is, a common shape). The other area where the distribution probability exists is considered to have a shape corresponding to the difference between each character image and the common shape (that is, a difference shape).

The image dictionary creation unit 50 has a first level threshold value for extracting a common shape (an area where the distribution probability is almost maximum) and a second level threshold value for extracting a difference shape having a high appearance frequency. Yes.
First, the image dictionary creation unit 50 performs threshold processing on the distribution probability Q ′ (x) using the first level threshold, and extracts a portion corresponding to the first level pattern (common shape). Next, the portion corresponding to the extracted first level pattern is removed from the distribution probability Q ′ (x) and converted to “1” or “0” with the second level threshold as a reference, thereby A two-level sum coupling pattern Q1 ″ (x) is generated.
The image dictionary creation unit 50 performs a product operation on the second level sum coupling pattern Q1 ″ (x) and the pixel distribution P (i, x) of each character image i, thereby obtaining a second portion which is a common part of these. In this example, the common part between the second level sum combined pattern Q1 ″ (x) and “character image # 1” is the second level pattern a, and the second level sum combined pattern Q1 ″ ( The common part between x) and “character image # 2” is the second level pattern b.
Thereby, “character image # 1” can be replaced with the respective indexes as the sum of the first level pattern and the second level pattern a, and “character image # 2” can be replaced with the first level pattern and the second level pattern b. Can be replaced with an index as the sum of
In this case, a plurality of image patterns (first level pattern and second level pattern) are associated with one character image. Therefore, one character image is associated with a plurality of indexes. However, the document file creation program 5 calibrates a combination of these indexes as a character image having a very similar shape (FIG. 10). I do.

It is a figure explaining the technique which makes easy confirmation of the recognition result of a character recognition process. It is a figure which illustrates the reproduction image displayed by the document file creation assistance apparatus. It is a figure which illustrates the hardware constitutions of the document file creation assistance apparatus 2 to which the document file creation assistance method concerning this invention is applied centering on the control apparatus 20. FIG. It is a figure which illustrates the functional structure of the document file creation program 5 which is performed by the control apparatus 20 (FIG. 3) and implement | achieves the document file creation assistance method concerning this invention. It is a figure explaining the function of the image dictionary preparation part 50 in detail. It is a figure which illustrates image dictionary 902 created by image dictionary creation part 50. It is a figure explaining the function of the encoding part 60 in detail. 6 is a diagram illustrating a reproduction image 260 displayed on the UI device 26. FIG. It is a flowchart which shows the whole operation | movement of reproduction image display processing (S10). It is a flowchart which shows the proofreading process (S20) of a character recognition result. It is a figure which illustrates the image dictionary 904 in which the character code and the font information were registered. It is a figure which illustrates typically the 1st image pattern creation processing in a modification. It is a figure which illustrates typically the 2nd image pattern creation processing in a modification.

Explanation of symbols

2 ... Document file creation support device 26 ... User interface device 260 ... Reproduced image 262 ... Underline 264 ... Cursor 5 ... Document file creation program 40 ... Image input unit 40 ... Character recognition unit 50 ... Image dictionary creation unit 510 ... Character image extraction unit 520 ... Match determination unit 530 ... Dictionary determination unit 540 ... Position correction unit 550 ... Index assignment unit 60 ..Encoding unit 610 ... Pattern determination unit 620 ... Position information encoding unit 630 ... Index encoding unit 640 ... Image encoding unit 650 ... Dictionary encoding unit 660 ... Selection 670: Code output unit 70: Reproduced image creation unit 80 ... User interface control unit 90 ... Code file creation unit 902, 904 ... Image dictionary

Claims (7)

A character discriminating means for discriminating characters displayed in the original image based on the original image in the raster data format;
Image pattern generation means for generating an image pattern of a character image based on a determination result by the character determination means;
Reproduction image creation means for creating a character string reproduction image that reproduces a character string included in the document image using the character image cut out from the document image and the image pattern generated by the character discrimination means;
A document file creation support apparatus comprising user interface means for displaying a character string reproduction image created by the reproduction image creation means. The document file creation according to claim 1, wherein the user interface unit displays a character string reproduction image in a display mode capable of distinguishing between a character image included in the document image and a character image generated by the image pattern generation unit. Support device. The character discrimination means further determines the accuracy of the character discrimination result,
The reproduction image creation means generates the portion corresponding to the character image by the image pattern generation means when the accuracy of the determination result determined for any one of the character images by the character determination means is a reference value or more. The character image cut out from the document image is applied to a portion corresponding to the character image when the accuracy of the determination result of the character image is smaller than a reference value. Document file creation support device. The character determining means determines at least character identification information of a character displayed in the document image;
The user interface means accepts input of character identification information for a character image included in the displayed character string reproduction image,
Character string file creation for creating a file of character identification information corresponding to the character string included in the document image based on the character identification information determined by the character determination means and the character identification information received by the user interface means The document file creation support apparatus according to claim 1, further comprising: means. The document file creation according to claim 4, wherein the user interface unit skips a cursor position indicating an input target of character identification information in a character string reproduction image according to whether or not the character image is cut out from a document image. Support device. Based on the raster data format original image, the characters displayed in the original image are determined,
Generate an image pattern of the character image based on the discrimination result,
Create a character string reproduction image that reproduces the character string included in the document image using the character image cut out from the document image and the generated image pattern,
A document file creation support method that displays the created character string reproduction image. In a document file creation support apparatus including a computer,
Determining a character displayed in the original image based on the original image in the raster data format;
Generating an image pattern of the character image based on the discrimination result;
Creating a character string reproduction image that reproduces a character string included in the document image using the character image cut out from the document and the generated image pattern;
A program for causing the computer of the document file creation support apparatus to execute a step of displaying the created character string reproduction image.

Images