JP6888299B2 - Image processing equipment and image processing program - Google Patents

Description

The present invention relates to an image processing apparatus and an image processing program.

There are many opportunities to handwrite and submit printed forms (forms, questionnaires, etc.) such as documents to be submitted to government offices and various questionnaires. There is a demand that the side that collects the filled-in forms wants to automatically read the handwritten characters written on those forms and the characters (including numbers) selected by the mark such as ○ mark and total them.

In response to the request, Patent Document 1 discloses a technique of scanning a mark sheet on which an answer is written with a scanner or the like and totaling the form for a type of form to be filled and answered, such as a mark sheet.

Japanese Unexamined Patent Publication No. 2013-45309

However, in the case of the technique disclosed in Patent Document 1 described above, the position of the mark written on the mark sheet can be detected, but what the mark written at that position means is separately determined. It is necessary to set in advance as the information of.

"The present invention provides an image processing device and an image processing program that recognize character information corresponding to an entered or selected position without requiring a work of associating a character with a character position in advance. The purpose is.

Claim 1 is
An image acquisition unit that acquires a first image data representing a first image and a second image data representing a second image in which additional recording is performed on the first image data.
An additional recorded image extraction unit that extracts an additional recorded image, which is an image additionally recorded with respect to the first image, from the second image.
A character recognition area for executing character recognition processing is set in the area of the first image corresponding to the additional recorded image, and the character recognition processing is repeatedly executed for the character recognition area while changing the area of the character recognition area. Equipped with a character recognition processing execution unit
The character recognition processing execution unit executes character recognition processing including calculation of recognition certainty, and recognizes characters with certainty exceeding a predetermined threshold in the area corresponding to the additional recorded image. If not, the position of the character recognition area set in the area corresponding to the additional recorded image is shifted to a position determined by referring to the position of the additional recorded image of the other second image, and the character is displayed. It is a character recognition device characterized by executing recognition processing.
Claim 2
The character recognition processing execution unit shifts the position of the character recognition area set in the area corresponding to the additional recording image, and has a predetermined threshold value corresponding to the additional recording image of the other second image. The first aspect of claim 1, wherein a character recognition area having an area corresponding to the area of the character recognition area when the certainty exceeding the above is obtained is set, and the character recognition process is executed for the character recognition area. It is a character recognition device.

Claim 3
Even if the additional recorded image extraction unit constitutes the additional recorded image and is separated from each other, the plurality of figures are the same for the plurality of figures that are close to each other within a predetermined threshold distance. The character recognition device according to claim 1 or 2 , wherein the character recognition device is considered to belong to the additional recorded image of the above.

Claim 4
The character recognition processing execution unit sets the character recognition area on the first image, and makes the character on the first image a recognition target, according to any one of claims 1 to 3 . The character recognition device according to any one of the items.

Claim 5
The character recognition processing execution unit sets the character recognition area on the second image or on the image of the difference between the first image and the second image, and on the second image or the difference. The character recognition device according to any one of claims 1 to 4 , wherein the characters on the image of the above are to be recognized.

Claim 6
When the region corresponding to the additional recorded image on the first image is a blank region, the character recognition processing execution unit is on the second image or on the first image and the second image. The character recognition area is set on the image of the difference from the image of the above, the character on the second image or the image of the difference is targeted for recognition, and corresponds to the additional recorded image on the first image. Claim 1 is characterized in that when the area is not a blank area, the character recognition area is set on the first image and the characters on the first image are to be recognized. The character recognition device according to any one of 5 to 5.

Claim 7
The information processing device is executed in the information processing device that executes the program.
An image acquisition unit that acquires a first image data representing a first image and a second image data representing a second image in which additional recording is performed on the first image data.
An additional recorded image extraction unit that extracts an additional recorded image, which is an image additionally recorded with respect to the first image, from the second image.
A character recognition processing execution unit that sets a character recognition area for executing character recognition processing in an area corresponding to the additional recorded image, and executes character recognition processing for the character recognition area while changing the area of the character recognition area. equipped with a,
The character recognition processing execution unit executes character recognition processing including calculation of recognition certainty, and recognizes characters with certainty exceeding a predetermined threshold in an area corresponding to the additional recorded image. If not, the position of the character recognition area set in the area corresponding to the additional recorded image is shifted to a position determined by referring to the position of the additional recorded image of the other second image, and the character is displayed. It is a character recognition program characterized by operating as a character recognition device that executes recognition processing.

According to the character recognition device of claim 1 and the character recognition program of claim 7 , it is necessary to set the character information corresponding to the entered or selected position in advance by associating the character with the character position. Can be recognized without.
Further, according to the character recognition device of claim 1 and the character recognition program of claim 7, the possibility of correct recognition is increased as compared with the case where the position of the additional recorded image of the other second image is not referred to.

According to the character recognition device of claim 2 , the time until the end of character recognition may be shortened as compared with the case where the area of the character recognition area at the shifted position is restarted from the initial area and the character recognition process is repeated. Will increase.
According to the character recognition device of claim 3, there is an increased possibility that each physically separated figure is correctly recognized as compared with the case where it is regarded as an additional recorded image.

According to the character recognition device of claim 4, when the character to be recognized exists in both the first image and the second image, as compared with the case where the character recognition area is set on the second image, as compared with the case where the character recognition area is set on the second image. It is more likely to be recognized correctly.

According to the character recognition device of claim 5, when the character to be recognized does not exist in the first image, the character recognition is performed by setting the character recognition area on the second image or the difference image. Can be done.

According to the character recognition device of claim 6 , a wide range of recognition is possible as compared with the case where the image for which the character recognition area is set is fixed to any one of the first image, the second image, or the difference image. ..

It is an external view of a character recognition system. It is a functional block diagram of a character recognition device realized by executing a character recognition program in a notebook PC. It is a figure which showed the flowchart of the character recognition program as one Embodiment of this invention. It is a figure which showed an example of the unfilled manuscript and the filled-out manuscript. FIG. 3 is a diagram showing a detailed flow of the first example of the character recognition process shown in one step (step S08). It is a figure which showed an example of the difference image. It is a schematic diagram which showed the process for recognizing the figure separated into two as one individual additional recorded image. It is explanatory drawing of the character recognition area setting method. It is a figure which showed the setting method of the character recognition area. FIG. 3 is a diagram showing a detailed flow of the second example of the character recognition process shown in one step (step S08). It is a schematic diagram which showed the character recognition area when ○ mark as an answer is written in the position shifted from the character to recognize. FIG. 3 is a diagram showing a detailed flow of the first example of the re-recognition process shown as one step in FIG. It is a figure which showed the detailed flow about the 2nd example of the re-recognition processing shown as one step in FIG. It is a figure which superimposes one individual additional recording image on the target difference image, and corresponding one individual addition recording image on the non-target difference image on the unfilled manuscript.

Hereinafter, embodiments of the present invention will be described.

FIG. 1 is an external view of a character recognition system.

The character recognition system 10 shown here includes a scanner 20 and a notebook personal computer (hereinafter, abbreviated as "notebook PC") 30. The scanner 20 and the notebook PC 30 are connected by a communication cable 40.

The scanner 20 is a device that reads an image recorded on a document and generates image data. When a document is placed on the document tray 21 of the scanner 20 and the start button (not shown) is pressed, or when an instruction is given from the notebook PC, one document is sent into the scanner 20. The scanner 20 is provided with a sensor (not shown) that photoelectrically reads the image on the document, and the image recorded on the document is photoelectrically read from the document sent into the scanner 20. Image data is generated. After the recorded image is read, the original is ejected onto the output tray 22. A plurality of originals can be stacked and placed on the original tray 21, and the scanner 20 sequentially feeds the plurality of originals on the original tray 21 into the scanner 20 one by one, and the fed originals are fed. The above image is read and discharged onto the output tray 22.

Further, the scanner 20 can lift the upper lid 23 with a hinge (not shown) extending to the left and right provided on the back side as a center of rotation. It is also possible to lift the upper lid 23, place one document under it, close the upper lid 23, and read the placed document.

The image data obtained by reading with the scanner 20 is input to the notebook PC 30 via the communication cable 40.

The notebook PC 30 is provided with a display screen 31 and a keyboard 32, and is provided with equipment such as a CPU and a memory for executing a program inside the notebook PC 30. A program is executed in the notebook PC 30, and processing is performed according to the executed program. Corresponding to this embodiment, the character recognition program described below is executed in this notebook PC. The character recognition program executed in the notebook PC 30 corresponds to an example of the character recognition program of the present invention. Then, the notebook PC 30 operates as a character recognition device as an embodiment of the present invention by executing this character recognition program.

FIG. 2 is a functional block diagram of a character recognition device realized by executing a character recognition program in a notebook PC.

The character recognition device 50 of the present embodiment includes an image acquisition unit 51, an additional recording image extraction unit 52, an additional recording image separation unit 53, and a character recognition processing execution unit 54. Examples of specific embodiments will be postponed, and here, each part 51 to 54 will be described in detail. It should be noted that the image on the data is dealt with here, and therefore, unless it is necessary to distinguish it, it should be clearly stated that the image on the data is an image on the data. Instead, it may be simply referred to as an "image" or "manuscript."

The image acquisition unit 51 captures an image of an unfilled manuscript in which characters as a question of a questionnaire are recorded and an answer to the question is not filled in, and an image of a completed manuscript in which an answer is additionally recorded in the unfilled manuscript. get. Although there is one unfilled manuscript, there are usually a plurality of filled-in manuscripts, and the image acquisition unit 51 acquires all the images. These unfilled manuscripts and filled-in manuscripts correspond to examples of the first image and the second image, respectively, as referred to in the present invention.

Further, the additional recorded image extraction unit 52 calculates an image of the difference between the recorded manuscript and the unfilled manuscript, and adds an image of the answer additionally recorded to the unfilled manuscript from the recorded manuscripts. Extract the recorded image.

Further, the additional recording image separation unit 53 separates the extracted additional recording image into an additional recording image for each individual recording (here, the additional recording image for each individual recording is referred to as an "individual additional recording image"). To do. Here, the additional recorded image separation unit 53 in the present embodiment determines in advance when separating the additional recorded image into each individual additional recorded image, even if it is a plurality of mutually separated figures constituting the additional recorded image. For a plurality of figures that approach each other within the specified threshold distance, the plurality of figures are regarded as belonging to the same individual additional recorded image, and the additional recorded image is separated into each individual additional recorded image.

Further, the character recognition processing execution unit 54 sets a character recognition area for executing the character recognition processing in each area corresponding to each individual additional recorded image, and changes the area of the character recognition area for the character recognition area. Repeat the character recognition process. Here, when the area corresponding to the individually added recorded image on the unfilled manuscript is a blank area, the character recognition processing execution unit 54 in the present embodiment writes as an filled-in manuscript or an unfilled manuscript. A character recognition area is set on the image of the difference from the completed manuscript to recognize the characters on the filled-in manuscript or the difference image, and the area corresponding to the individually added recorded image on the unfilled manuscript is not a blank area. If this is the case, a character recognition area is set on the unfilled manuscript to recognize the characters on the unfilled manuscript.

Further, the character recognition processing execution unit 54 executes character recognition processing including calculation of the certainty of recognition, and while changing the area of the character recognition area, a predetermined recognition processing stop condition is set. The execution of the character recognition process is repeated until it is satisfied. As the recognition processing stop condition, it is adopted that the certainty exceeds a predetermined threshold value, the certainty reaches a maximum value, or the execution of the character recognition processing is repeated a predetermined number of times. ..

Here, in the present embodiment, when the area on the unfilled manuscript corresponding to the individually added recorded image is not a blank area, the character recognition area is set on the unfilled manuscript and the characters on the unfilled manuscript are set. However, when the character recognition processing execution unit 54 in the present embodiment does not recognize a character with a certainty exceeding a predetermined threshold value in the character recognition area set on the unfilled manuscript, The character recognition process is executed by shifting the position of the character recognition area. In shifting the position of the character recognition area, the character recognition processing execution unit 54 refers to the position of the individually added recorded image recorded in another filled-in manuscript different from the filled-in manuscript currently being processed. To determine the position to shift the character recognition area. Further, the character recognition processing execution unit 54 shifts the position of the character recognition area set in the area on the unfilled manuscript corresponding to the individually added recorded image, and also shifts the position of the character recognition area on the other written manuscript described above. A character recognition area having an area corresponding to the area of the character recognition area when the certainty equal to or higher than the threshold value is obtained is set, and the character recognition process is executed for the character recognition area of that area. Specific examples will be described later.

FIG. 3 is a diagram showing a flowchart of a character recognition program as an embodiment of the present invention.

The image on the document is read by the scanner 20 shown in FIG. 1 to generate image data, and the generated image data is input to the notebook PC 30 via the communication cable 40. Then, the character recognition program shown in FIG. 3 is activated, and the image data input to the notebook PC 30 via the communication cable 40 is acquired (step S01). As described above, here, unless it is particularly necessary, even if it is an image on the data, the "data" is omitted and referred to as an "image" or a "manuscript".

When the image is acquired in step S01, it is determined whether the image acquired this time is the first image or the second and subsequent images (step S02).

In the present embodiment, a rule is set in which the scanner 20 is made to read the unfilled manuscript on the first sheet, and then sequentially read the filled-in manuscript on the second and subsequent sheets. Therefore, when the acquired image is the first image, this character recognition program temporarily saves the image as an unfilled manuscript (step S03). Image acquisition is repeated for the second and subsequent sheets (step S05), and all the images acquired for the second and subsequent sheets are temporarily saved as completed manuscripts (step S04).

FIG. 4 is a diagram showing an example of an unfilled manuscript and a filled-in manuscript.
FIG. 4A shows a questionnaire form before filling in, that is, an unfilled manuscript 51. Here, there are three questions (1) to (3) as the contents of the questionnaire, and of these three questions, the answer to the questions (1) and (2) is any of the numbers 1 to 5. This is a method in which one number is selected by enclosing it with a circle. Question (3) is a question in the form of having the answer freely entered in the blank answer column 511.

Further, FIG. 4B shows a completed manuscript 52 in which the respondent entered the answer on the questionnaire form having the same format as the questionnaire form shown in FIG. 4A. The completed manuscript is not limited to one, and each of the second and subsequent manuscripts out of the plurality of manuscripts sequentially read by the scanner 20 is treated as a filled-in manuscript.

In the one completed manuscript shown in FIG. 4 (B), the number "3" is surrounded by a circle 521 for the question (1). Regarding the question (2), the number "1" is surrounded by a circle 522. Further, regarding the question (3), the character string 523 entered by the respondent is lined up in the blank answer column.

The explanation will be continued by returning to FIG.

When the series of image acquisition is completed (step S05), then one of the completed documents temporarily saved in step S04 is taken out (step S06). However, the completed manuscript that has undergone the character recognition process in step S08 is excluded from the extraction target. Then, when there is an unprocessed written manuscript, that is, when the unprocessed written manuscript can be taken out (step S07), the character recognition process is performed on the one unprocessed written manuscript taken out. Is executed (step S08). The details of the character recognition process will be described later.

When the unprocessed completed manuscript cannot be taken out, that is, when the character recognition process (step S08) for all the filled-in manuscripts is completed (step S07), then for each character under certain conditions. (Step S09), the re-recognition process is executed (step S10). The condition of step S09 and the re-recognition process (step S10) will be described later. When the character satisfying the condition of step S09 does not exist, or when the character satisfying the condition of step S09 does not exist due to the re-recognition process (step S10), the current character recognition routine is terminated.

FIG. 5 is a diagram showing a detailed flow of the first example of the character recognition process shown in one step (step S08) in FIG.

Here, first, an image of the difference between the one filled-in manuscript taken out in step S06 of FIG. 3 and the unfilled manuscript temporarily saved in step S03 is generated (step S21).

FIG. 6 is a diagram showing an example of a difference image.

The difference image 53 shown in FIG. 6 is between the unfilled manuscript 51 shown in FIG. 4 (A) and the top one of the filled-in manuscripts 52 shown in FIG. 4 (B). It is a difference image of. An "additional recorded image" composed of two circles 521 and 522 and a character string 523 entered by the respondent is extracted from the difference image 53. The additional recorded image appearing on the difference image corresponds to an example of the additional recorded image referred to in the present invention.

The explanation will be continued by returning to FIG.

When a difference image as illustrated in FIG. 6 is generated (step S21), the additional recorded image appearing on the difference image is then separated into "individual additional recorded images" which are images for each individual recording (step S21). Step S22). Here, the "individual additional recorded image" refers to each of the images collected to the extent that the respondent recognizes it as one character or one figure. Specifically, in the example shown in FIG. 6, each of the two circles 521 and 522 and each character constituting the character string 523 are each individual additional recorded image. Therefore, even if the figure is separated into a plurality of figures or the characters are separated into a plurality of parts due to blurring at the time of writing, the figure separated into a plurality of parts or the characters separated into a plurality of parts should be recognized as one individual additional recorded image. There are also scenes.

FIG. 7 is a schematic diagram showing a process for recognizing a figure separated into two as one individual additional recorded image.

FIG. 7A is one of the circles drawn by the respondents that appeared on the difference image. The circles shown in FIG. 7 (A) are figures that are separated into two by being blurred in the middle.

Here, by filling the periphery of each pixel constituting the ○ mark shown in FIG. 7 (A) as a pixel constituting the ○ mark over a predetermined range, it is shown in FIG. 7 (B). As shown, thicken the lines that make up the circle.

FIG. 7C is a schematic diagram in which pixels are represented by squares. One square represents one pixel. The central pixel P is a pixel showing only one on behalf of a large number of pixels on the line forming the circle in FIG. 7 (A).

Specifically, in thickening the line, as shown in FIG. 7 (C), when one pixel P constituting the circle is present, it is predetermined around the pixel P. Pixels within the range (in the example shown here, within the range of 5 pixels × 5 pixels) are filled as pixels constituting the circles. Here, one pixel P is typically shown, but the same processing is performed for all the pixels constituting the circles to generate thick circles as shown in FIG. 7 (B). .. As a result of thickening the line in this way, the connected figures are recognized as one individual additional recorded image. In the present embodiment, by such processing, the additional recorded image appearing on the difference image is separated into each individual additional recorded image. In the example of the difference image 53 shown in FIG. 6, by the above processing, each of the two circles 521 and 522 and one of the characters constituting the character string 523 are separated, and each of them is separated. Is recognized as each individual additional recorded image.

In the present embodiment, a process of thickening a line to make a figure connected to each other as an individual additional recorded image is adopted, but in this process, figures separated from each other are close to each other within a predetermined distance. This is one of the processes for determining whether or not there is a presence. That is, here, when the figures separated from each other are close to each other within a predetermined distance, they are recognized as one individual additional recorded image.

The explanation will be continued by returning to FIG.

After the additional recorded image appearing on the difference image is separated into individual additional recorded images as described above (step S22), the difference image is inspected from the upper left to the lower right (step S23). When the individual additional recorded image is found, one of the found individual additional recorded images is taken out (step S24). Then, the following processing is repeated until there is no unprocessed individual additional recorded image on the difference image targeted this time (step S25).

Here, first, it is determined whether or not the area corresponding to the one individually added recorded image taken out this time on the unfilled manuscript is blank (step S26). In the case of the first example described here, as a method of determining whether or not it is blank, binarization processing is performed, and the white side is defined as blank. Another example of the method of determining whether or not it is blank will be described later.
When the area on the unfilled manuscript corresponding to the one individually added recorded image taken out this time is not blank (step S26), then, is the image additionally recorded in that area an image having dimensions equal to or larger than the threshold value? Whether or not it is determined (step S27). Then, when an image having a dimension equal to or larger than the threshold value is recorded in that area, a character recognition area is set on the unfilled document (step S28). On the other hand, when the area on the unfilled document corresponding to the individually added recorded image is blank (step S26), in the present embodiment, the character recognition area is set on the difference image (step S29). Further, even when the image on the unfilled manuscript corresponding to the individually added recorded image is an image having dimensions that are not blank but does not meet the threshold value, that is, a noise image (step S27), the character recognition area is formed on the difference image. Set (step S29). If the area on the unfilled manuscript corresponding to the individually added recorded image is blank or an image with dimensions less than the threshold value, character recognition is performed not on the difference image but on the filled-in manuscript being processed this time. The area may be set.

FIG. 8 is an explanatory diagram of a character recognition area setting method. Here, it is assumed that the unfilled manuscript and the difference image are overlapped with each other, and the character recognition area on the unfilled manuscript and the character recognition area on the difference image will be described without distinction.

FIG. 8A shows one individual additional recorded image (marked with a circle as an example). Here, considering a rectangle inscribed by the individually added recorded image as shown in FIG. 8B, a character recognition area D having a predetermined minimum area is set at the center of the rectangle. However, instead of considering the rectangle, another method of finding the substantially center of the individually added recorded image, such as setting the character recognition area at the center of gravity of the individually added recorded image, may be adopted.

In this way, the character recognition area is set (steps S28 and S29), and the character recognition process is executed in the character recognition area (step S30). The character recognition process itself is an existing technique, and the description thereof is omitted here. In this character recognition process, the "certainty" of the recognized character is also calculated.

Then, this character recognition process (step S30) is repeated while resetting the character recognition area (step S32) until the stop condition is satisfied (step S31).

FIG. 9 is a diagram showing a method of resetting the character recognition area.

Initially, as shown in FIG. 8C, the character recognition area D having the smallest area is set (FIG. 5, step S30), and the area of the character recognition area is set to D1 → D2 → D3 → ... The character recognition process is executed (step S30) until the stop condition is satisfied (step S31) while gradually expanding as in (step S32).

Here, as a stop condition in step S31,
(A) The certainty of character recognition has exceeded a predetermined threshold value (b) The certainty of character recognition has reached the maximum value (c) Character recognition while gradually expanding the area of the character recognition area It is adopted that the process is repeated a predetermined number of times. Here, even when the above (a) or (b) is adopted as the stop condition, it is desirable to use the above (c) together so that the character recognition process does not continue indefinitely.

Here, it has been explained that the character recognition process is repeated while gradually expanding the area of the character recognition area, but as the initial character recognition area, for example, a large size similar to the rectangular frame shown in FIG. 8 (B). The character recognition process may be repeated while setting the character recognition area of the area and gradually reducing the area of the character recognition area.

When the character recognition stop condition is satisfied (step S31), the character recognized this time is saved together with the "certainty" (step S33).

When the above processing is executed for each of the individually added recorded images on one difference image (steps S23 and S24), and the processing for all the individually added recorded images on the one difference image is completed. (Step S25), the process shown in FIG. 5 for the one difference image, that is, the character recognition process shown as step S08 in FIG. 3 is completed, and the process proceeds to the character recognition process for the next unprocessed completed manuscript. (Step S06). Then, when the character recognition process for all the completed manuscripts is completed (step S07), the process proceeds to the re-recognition process (step S10).
Here, a second example of the character recognition process in step S08 will be described before moving on to the description of the re-recognition process (step S10).
FIG. 10 is a diagram showing a detailed flow of the second example of the character recognition process as the second example shown in one step (step S08) in FIG. The character recognition process as the second example shown in FIG. 10 is a character recognition process that can be adopted in place of the character recognition process as the first example shown in FIG.
The characters recognition process steps S21 to S25 shown in FIG. 10 are the same processes as the same steps S21 to S25 of the character recognition process shown in FIG. 5, and duplicate description thereof will be omitted here.
When it is determined in step S25 that there is an individual additional recorded image, in the case of the first example shown in FIG. 5, it is determined whether or not it is blank by the binarization process (FIG. 5, step S26), and it is not blank. If so, it is determined whether or not the image recorded there is an image having dimensions equal to or greater than the threshold value (step S27). On the other hand, in the case of the second example shown in FIG. 10, those steps S27 and S28 do not exist, and suddenly a character recognition area is set on the unfilled manuscript (step S211), and the character recognition area is set as a character. The recognition process is executed (step S212). Then, the character recognition process (step S212) is reset as shown in FIG. 9 as D1 → D2 → D3 → ... Until the stop condition is satisfied (step S213) (step S214). ),repeat. These steps S212 to S214 are the same processes as steps S30 to S32 of FIG. 5, except that only unfilled documents are processed.
Then, when the stop condition is satisfied (step S213), it is determined whether or not the character is recognized from the character recognition area set (reset) on the unfilled document (step S215). The determination here as to whether or not the character is recognized is a determination for distinguishing from mere noise, and the certainty as a threshold value is set to a considerably low level (for example, 20% certainty). Then, when it is determined that the character is recognized (step S215), the recognized information is saved (step S216).
On the other hand, in step S215, when it is determined that the character is not recognized or the certainty is equal to or less than the threshold value, the character recognition area is set on the difference image (step S217), and the character recognition process is executed (step S217). S218). The processing of steps S217 to S221 is the same as that of steps S30 to S33 of FIG. 5, except that only the difference document is processed.
In the case of the first example shown in FIG. 5, it is directly determined whether or not the target area on the blank manuscript is blank (FIG. 5, step S26), but instead, the process shown in FIG. 10 is performed. Assuming that characters exist on the unfilled manuscript, it tries to recognize the characters on the unfilled manuscript, and if the characters cannot be recognized, it is left blank, or if it can be recognized only with low certainty, it is noise. As a result, the character recognition process from the difference image may be started.
Next, the re-recognition process (FIG. 3, step S10) will be described.

FIG. 11 is a schematic diagram showing a character recognition area when a circle as an answer is written at a position deviated from the character to be recognized.

When the ○ mark as an answer is entered at a position deviated from the character to be recognized, the center point of the character recognition area is deviated from the character to be recognized (here, the number “3”) as shown in FIG. It is possible that the ruled line set at the position and recorded under the number "3" is erroneously recognized as a character, and the character recognition process for that character is completed while the certainty remains less than the threshold value.

The re-recognition process of step S10 in FIG. 3 is a process for saving the low recognition rate in such a situation.

This re-recognition process is executed for each character that meets the re-recognition process condition of step S09. The re-recognition processing condition in step S09 is that all of the following (d) to (f) are satisfied.

(D) This re-recognition process is intended for recognizing characters on an unfilled manuscript. Since the characters on the difference image or the filled-in manuscript cannot be expected to greatly contribute to the improvement of the certainty even if they are the targets of the following re-recognition processing, here, the characters on the difference image or the filled-in manuscript are referred to. It is not subject to re-recognition processing.

(E) Further, this re-recognition process targets characters whose certainty is less than a predetermined threshold value (for example, 80% certainty) in the character recognition process (step S08). ..

(F) Further, this re-recognition process targets characters for which the re-recognition process has not yet been executed, among the characters with less than 80% certainty in character recognition on the unfilled manuscript.

Here, when there are characters satisfying the above re-recognition processing conditions (d) to (f) (step S09), the re-recognition process (step S10) is performed for each character satisfying the re-recognition processing conditions. ) Is executed.

FIG. 12 is a diagram showing a detailed flow of the first example of the re-recognition process shown as one step in FIG.
Here, refer to FIG. 9 while moving the center point of the character recognition process (the character recognition area D having the smallest area shown in FIG. 11) within a predetermined area (for example, within an area of 4 pixels each in the vertical and horizontal directions). The character recognition process described above is repeated. Specifically, the flow is as shown in FIG.
Here, first, a character recognition area having the smallest area is set at a certain offset position within a predetermined area (for example, within an area of 4 pixels each on the top, bottom, left, and right) (step S41), and the character recognition area is set as a character. The recognition process is executed (step S42). Then, in this character recognition process, the character recognition area is reset as shown in FIG. 9 (that is, D1 → D2 → D3 → ...) Until the stop condition is satisfied (step S43). While gradually expanding) (step S44), repeat.
When the stop condition is satisfied (step S43), is the character recognition process performed by shifting the center point (character recognition area D having the smallest area) in all of the predetermined areas (for example, in the areas of 4 pixels each in the vertical and horizontal directions)? If it is determined whether or not (step S45) and the position where the center point should be shifted still remains in the area, the process returns to step S41 and one of the remaining positions where the center point should be shifted remains. The process of steps S42 to S44 is executed by shifting the center point to. On the other hand, in step S45, when it is determined that the center point has been shifted to all the positions in the region where the center point should be shifted, this time, as a result of this series of character recognition processing, has the certainty improved? Whether or not it is determined (step S46), and when the certainty is improved, the recognition result of the same target saved up to that point is replaced with the current re-recognition result (step S47).
The above re-recognition process is executed for each character satisfying the above conditions (d) to (f) (FIG. 3, steps S09 and S10), and the execution of this character recognition process routine ends.
Next, a second example of the re-recognition process will be described.
FIG. 13 is a diagram showing a detailed flow of the second example of the re-recognition process shown as one step in FIG. The re-recognition process of the second example is a process that can be adopted in place of the re-recognition process as the first example described with reference to FIG.

In the re-recognition process of the second example, first, the position where the character recognition area for recognizing the character to be recognized is shifted and the area of the character recognition area are determined (step S411). For this reason, a difference image different from the difference image used in recognizing this character (here, this difference image is referred to as a "target difference image") (here, this difference image is referred to as "this difference image". When the target difference image and the non-target difference image are superimposed with reference to the non-target difference image), the center point of the individually added recorded image on which the recognition of this character is based (FIG. 8 (Fig. 8). Compared to C)), it is the basis for setting the character recognition area when the center points are within a predetermined distance and a character recognition result with a certainty of 80% or more is obtained for a difference image other than the target. Find individual additional recorded images.

FIG. 14 is a diagram showing one individually added recorded image on the target difference image and the corresponding one individually added recorded image on the non-target difference image superimposed on the unfilled manuscript.

The individually added recorded image 551 on the target difference image is located at a position slightly deviated from the character to be recognized (here, the number “3”). Then, it is assumed that only the recognition result with a certainty of less than 80% can be obtained from the character recognition area set based on the individually added recorded image 551.

On the other hand, the individually added recorded image 552 on the difference image other than the target is drawn so as to neatly surround the character to be recognized (here, the number "3"). Then, it is assumed that a recognition result having a certainty of more than 80% is obtained from the character recognition area set based on the individually added recorded image 552.

In this case, when executing the re-recognition process for a character for which a recognition result of less than 80% certainty is obtained, a recognition result having a certainty of more than 80% is set in the character recognition area for executing the re-recognition process. Is shifted to the same position as the character recognition area set based on the individually added recorded image 552 on the difference image other than the target obtained. Further, regarding the area of the character recognition area, it is a character recognition area set based on the individually added recorded image 552 on the difference image other than the target, and the area when the certainty exceeds 80% is set. For example, when the character recognition area of the area Dx shown in FIG. 9 is the character recognition area when the certainty of more than 80% is obtained for the difference image other than the target, the certainty is less than 80%. In the re-recognition process for a character for which only the recognition result has been obtained, the character recognition area having the area Dx is adopted.

In step S411 of FIG. 13, the shift position of the character recognition area and the area of the character recognition area are determined as described above, and the character recognition process is executed for the determined character recognition area (step S412).

Then, as a result of the character recognition process (step S412), it is determined whether or not the certainty is improved (step S413), and when the certainty is improved, the recognition result of the same target saved up to that point is displayed. , It is replaced with the result of this re-recognition (step S414).

The above re-recognition process is executed for each character satisfying the above conditions (d) to (f) (FIG. 3, steps S09 and S10), and the execution of this character recognition process routine ends.

As described above, according to the present embodiment, there is no need to pre-set information such as what the mark at that position means for each position of the mark on the mark sheet. , Can recognize the respondent's answer.

Here, in the case of the present embodiment, a plurality of originals are continuously read by the scanner 20, the first original among the plurality of originals is regarded as an unfilled original, and the second and subsequent originals have been filled. There is a rule that it should be a manuscript. In this case, the image data of the unfilled manuscript can be easily and surely acquired. However, in the present invention, the rule of arranging the unfilled manuscript at a specific position such as the first sheet is not always necessary. The unfilled manuscript may be sandwiched in the middle of the filled-in manuscripts in which a plurality of sheets are stacked, for example. In that case, a process of finding an unfilled manuscript from a plurality of manuscripts may be performed in the image acquisition unit. As an example of the process of finding the unfilled manuscript, the process of extracting the difference between the first manuscript and the second and subsequent manuscripts is sequentially performed, and the manuscript in which the difference appears only in the first manuscript is the unfilled manuscript. And it is sufficient. In addition, an image obtained by extracting the common parts of a plurality of manuscripts on the data obtained by scanning regardless of whether the manuscript is an unfilled manuscript or a filled manuscript is created, and the created image and each read manuscript are created. The manuscript with the highest degree of matching may be regarded as an unfilled manuscript by performing pattern matching with.

Alternatively, even if the completed manuscript is read multiple times, the unfilled manuscript for the same type of manuscript is read only once, the unfilled manuscript once read is stored, and the filled-in manuscript read this time is completed. The unfilled manuscript corresponding to the filled-in manuscript read this time may be specified by pattern matching with the manuscript, feature point extraction, or form recognition using the degree of coincidence of the area surrounded by the straight line.

Furthermore, in the present invention, it is not always necessary to even read an unfilled manuscript. For example, an unfilled manuscript may be created on the data by extracting the common portion of the plurality of filled-in manuscripts from the plurality of filled-in manuscripts. In this case, the unfilled manuscript on the data created by extracting the common portion corresponds to the first image data representing the first image.

Further, as shown in FIG. 1, the character recognition device including the notebook PC 30 connected to the scanner 20 by the communication cable 40 has been described here, but the character recognition device in the present invention does not necessarily have to be in this form. For example, the function of the character recognition device of the present invention may be incorporated into a copier in which a scanner and a printer are combined, or a multifunction device having more functions. Further, the function of the character recognition device of the present invention may be incorporated into a portable terminal having a camera function. In that case, the image obtained by shooting the original with the camera function is the target of character recognition.

10 Character recognition system 11 Image acquisition unit 12 Additional recording image extraction unit 13 Additional recording image separation unit 14 Character recognition processing execution unit 20 Scanner 21 Document tray 22 Paper ejection tray 23 Top lid 30 Notebook type personal computer (notebook PC)
31 Display screen 32 Keyboard 51 Unfilled manuscript 52 Filled manuscript 53 Difference image 511 Answer column 521,522 ○ mark 523 Character string 551,552 Individual additional recorded image

Claims (7)

An image acquisition unit that acquires a first image data representing a first image and a second image data representing a second image in which additional recording is performed on the first image data.
An additional recorded image extraction unit that extracts an additional recorded image, which is an image additionally recorded with respect to the first image, from the second image.
A character recognition area for executing character recognition processing is set in the area of the first image corresponding to the additional recorded image, and the character recognition processing is repeatedly executed for the character recognition area while changing the area of the character recognition area. Equipped with a character recognition processing execution unit
The character recognition processing execution unit executes character recognition processing including calculation of recognition certainty, and recognizes characters with certainty exceeding a predetermined threshold in the area corresponding to the additional recorded image. If not, the position of the character recognition area set in the area corresponding to the additional recorded image is shifted to a position determined by referring to the position of the additional recorded image of the other second image, and the character is displayed. A character recognition device characterized by executing recognition processing. The character recognition processing execution unit shifts the position of the character recognition area set in the area corresponding to the additional recording image, and has a predetermined threshold value corresponding to the additional recording image of the other second image. The first aspect of claim 1 , wherein a character recognition area having an area corresponding to the area of the character recognition area when the certainty exceeding the above is obtained is set, and the character recognition process is executed for the character recognition area. Character recognition device. Even if the additional recorded image extraction unit constitutes the additional recorded image and is separated from each other, the plurality of figures are the same for the plurality of figures that are close to each other within a predetermined threshold distance. The character recognition device according to claim 1 or 2, wherein the character recognition device is considered to belong to the additional recorded image of the above. The character recognition processing execution unit sets the character recognition area on the first image, and makes the character on the first image a recognition target, according to any one of claims 1 to 3. The character recognition device according to any one of the items. The character recognition processing execution unit sets the character recognition area on the second image or on the image of the difference between the first image and the second image, and on the second image or the difference. The character recognition device according to any one of claims 1 to 4, wherein the characters on the image of the above are to be recognized. When the region corresponding to the additional recorded image on the first image is a blank region, the character recognition processing execution unit is on the second image or on the first image and the second image. The character recognition area is set on the image of the difference from the image of the above, the character on the second image or the image of the difference is targeted for recognition, and corresponds to the additional recorded image on the first image. Claim 1 is characterized in that when the area is not a blank area, the character recognition area is set on the first image and the characters on the first image are to be recognized. The character recognition device according to any one of 5 to 5. The information processing device is executed in the information processing device that executes the program.
An image acquisition unit that acquires a first image data representing a first image and a second image data representing a second image in which additional recording is performed on the first image data.
An additional recorded image extraction unit that extracts an additional recorded image, which is an image additionally recorded with respect to the first image, from the second image.
A character recognition processing execution unit that sets a character recognition area for executing character recognition processing in an area corresponding to the additional recorded image, and executes character recognition processing for the character recognition area while changing the area of the character recognition area. With
The character recognition processing execution unit executes character recognition processing including calculation of recognition certainty, and recognizes characters with certainty exceeding a predetermined threshold in an area corresponding to the additional recorded image. If not, the position of the character recognition area set in the area corresponding to the additional recorded image is shifted to a position determined by referring to the position of the additional recorded image of the other second image, and the character is displayed. A character recognition program characterized by operating as a character recognition device that executes recognition processing.

Images