JP2018124854A - Image processing apparatus and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To recognize character information corresponding to a selected position, without requiring a work for setting in advance a character and the position of the character while associating them.SOLUTION: Image data of an unwritten document 51 and image data of an already written document 52 including an additional recording on the unwritten document 51 are acquired. Characters on the unwritten document 51 are recognized. Additional recorded images (circle marks 521, 522, and 523), which are images additionally recorded on the unwritten document 51, are extracted from the written document 52. Characters recorded in areas corresponding to the additional recorded images (circle marks 521, 522, and 523) are identified from the characters recognized on the unwritten document 51.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an image processing apparatus and an image processing program.

  There are many opportunities to fill out printed forms (forms, questionnaires, etc.) by hand, such as filling out documents submitted to government offices and various questionnaires. There is a demand for the side that collects the completed forms to automatically read and complete the completed questionnaire forms.

  In response to such a request, Patent Document 1 discloses a technique for reading and counting a mark sheet on which a reply is written with a scanner or the like for a type of form that is filled and answered like a mark sheet.

Japanese Unexamined Patent Publication No. 2013-45309

  However, in the case of the technique disclosed in the above-mentioned Patent Document 1, the position of the mark written on the mark sheet can be detected, but what the mark written at the position means separately It is necessary to set in advance as information.

  An object of the present invention is to provide an image processing apparatus and an image processing program for recognizing character information corresponding to a specified position without requiring an operation for setting characters and character positions in association with each other in advance. And

Claim 1
An image acquisition unit configured to acquire first image data representing a first image and second image data representing a second image additionally recorded on the first image data;
A character string recognition unit for recognizing a character string including one character from the first image;
An additional recorded image extraction unit that extracts an additional recorded image that is an image additionally recorded with respect to the first image from the second image;
An image processing comprising: a character string specifying unit that specifies a character string corresponding to the additional recording image on the first image from the character strings recognized by the character string recognition unit. Device.

Claim 2
In addition to recognizing the character string from the first image, the character string recognizing unit records one or more points on which the character string is recorded on the first image for each recognized character string. A first region including the region represented by the coordinates of
The additional recorded image extraction unit records the individual additional recorded image on the second image for each individual additional recorded image that constitutes the additional recorded image. Extracting a second region including the region represented by the coordinates of the point;
2. The character string specifying unit specifies a character string associated with a first area having a predetermined first positional relationship with respect to the second area. This is an image processing apparatus.

Claim 3
A plurality of characters in which the character string recognition unit recognizes each of the characters including a region represented by one or a plurality of coordinates in a predetermined second positional relationship. Is recognized as one character string, and the first image includes a region including the region represented by the coordinates of one point or a plurality of points on which the character string is recorded. The image processing apparatus according to claim 2, wherein the image processing apparatus is associated with the character string as an area of the image.

Claim 4
The character string specifying unit does not specify a character string for the second area when the first area having the first positional relationship with the second area does not exist. The image processing apparatus according to claim 2.

Claim 5
The character string recognition unit recognizes a character string for each area surrounded by the ruled line when the first image is an image including a ruled line. The image processing apparatus according to any one of the above.

Claim 6
When the character string specifying unit specifies the same character string associated with the same first region corresponding to the plurality of second regions, a plurality of the same character strings The image processing according to any one of claims 2 to 5, wherein the same character string specified in the remaining specification other than one specification among the times specified is ignored. Device.

Claim 7
The information processing apparatus is executed in an information processing apparatus that executes a program.
An image acquisition unit configured to acquire first image data representing a first image and second image data representing a second image additionally recorded on the first image data;
A character string recognition unit for recognizing a character string including one character from the first image;
An additional recorded image extraction unit that extracts an additional recorded image that is an image additionally recorded with respect to the first image from the second image;
Operate as an image processing apparatus including a character string identifying unit that identifies the character string corresponding to the additional recorded image on the first image from among character strings recognized by the character string recognizing unit. This is an image processing program.

  According to the image processing apparatus of claim 1 and the image processing program of claim 7, the character information corresponding to the specified position is not required to be set in association with the character and the character position in advance. Can be recognized.

  According to the image processing apparatus of the second aspect, it is possible to specify the character string more accurately than in the case where the character string is specified without the concept of the predetermined first positional relationship.

  According to the image processing apparatus of claim 3, even if the character string corresponding to one second region is a character string composed of a plurality of characters, the character string composed of the plurality of characters can be specified.

  According to the image processing apparatus of the fourth aspect, erroneous recognition is suppressed as compared with the case where character strings are specified for all the second regions.

  According to the image processing apparatus of the fifth aspect, it is possible to recognize the character string more accurately as compared with the case where the coordinates are recognized without using the ruled line even if the ruled line is recorded.

  According to the image processing apparatus of claim 6, even if a figure or the like that should be originally recognized as one second area is divided into a plurality of parts by drowning or the like and recognized as a plurality of second areas, The column can be identified correctly.

It is a schematic diagram of a character recognition system. It is a functional block diagram of an image processing device realized by executing an image processing program in a notebook PC. It is the figure which showed the flowchart of the image processing program as one Embodiment of this invention. It is the figure which showed the 1st example of the unfilled original and the completed original. It is the figure which showed the 2nd example of the unfilled original and the completed original. It is the figure which showed the 3rd example of the unfilled original and the completed original. It is the figure which showed the 4th example of the unfilled original and the completed original. It is the figure which showed the flowchart of the character string and area | region recognition process on an unwritten manuscript. It is the figure which showed the calculation method of the 1st area | region matched with the recognized character. FIG. 6 is a diagram illustrating a part of a document on which ruled lines are drawn. It is the figure which showed the example of a connection of the 1st area | regions using a ruled line. FIG. 4 is a diagram showing a detailed flow of character recognition processing shown in one step (step S09) in FIG. It is the figure which showed an example of the difference image. It is the figure which showed the detailed flow of the character string specific process. It is the figure which showed the example with which the 2nd area | region and the 1st area | region have overlapped. It is the figure which showed the example in which the 1st area | region exists on the right side of a 2nd area | region.

  Embodiments of the present invention will be described below.

  FIG. 1 is a schematic diagram of a character recognition system.

  The character recognition system 10 shown here includes a scanner 20 and a notebook personal computer (hereinafter abbreviated as “notebook PC”) 30. The scanner 20 and the notebook PC 30 are connected by a communication cable 40.

  The scanner 20 is an apparatus that reads an image recorded on a document and generates image data. When a document is placed on the document tray 21 of the scanner 20 and a start button (not shown) is pressed or an instruction is given from a notebook PC, one document is fed into the scanner 20. The scanner 20 is provided with a sensor (not shown) for photoelectrically reading an image on the document, and the image recorded on the document is photoelectrically read from the document sent into the scanner 20. Image data is generated. The original after the recorded image is read is discharged onto the paper discharge tray 22. A plurality of documents can be stacked and placed on the document tray 21, and the scanner 20 sequentially feeds the plurality of documents on the document tray 21 one by one into the scanner 20, and the fed documents The upper image is read and discharged onto the paper discharge tray 22.

  Further, the scanner 20 can lift the upper lid 23 around a hinge (not shown) provided on the back side and extending left and right. It is also possible to lift the upper lid 23 and place one document underneath, close the upper lid 23 and read the placed document.

  Image data obtained by reading with the scanner 20 is input to the notebook PC 30 via the communication cable 40.

  The notebook PC 30 includes a display screen 31 and a keyboard 32, and also includes equipment such as a CPU and a memory for executing a program therein. In the notebook PC 30, a program is executed, and processing according to the executed program is performed. In correspondence with the present embodiment, an image processing program to be described later is executed on this notebook PC. The image processing program executed in the notebook PC 30 corresponds to an example of the image processing program of the present invention. The notebook PC 30 operates as an image processing apparatus as an embodiment of the present invention by executing the image processing program.

  FIG. 2 is a functional block diagram of the image processing apparatus realized by executing the image processing program in the notebook PC.

  The image processing apparatus 60 of the present embodiment includes an image acquisition unit 61, a character string recognition unit 62, an additional recorded image extraction unit 63, and a character string specifying unit 64. A specific embodiment will be illustrated later, and here, each of the parts 61 to 64 will be described in general. Note that here, images on data are dealt with. Therefore, unless there is a particular need for distinction, it is specifically stated that even images on data are images on data. Instead, it may be simply referred to as “image” or “original”.

  The image acquisition unit 61 includes an image of an unfilled manuscript in which characters as questions of a questionnaire are recorded and an answer to the question is unfilled, and an image of a filled manuscript in which an answer is additionally recorded on the unfilled manuscript. get. Although there is one unwritten document, there are usually a plurality of completed documents, and the image acquisition unit 61 acquires all of these images. These unfilled manuscript and filled manuscript correspond to examples of the first image and the second image, respectively, according to the present invention.

Further, the character string recognition unit 62 recognizes a character string including one character from an unfilled document. The “character string” here is a concept including not only a character string composed of a plurality of characters but also a character string composed of only one character.
Here, the character string recognizing unit 62 of the present embodiment recognizes the character string and, for each recognized character string, the coordinates of one or more points on the unfilled document on which the character string is recorded. The first area including the area expressed by is associated with the character string. The “first region” associated with the character string may be represented by the coordinates of one point or the coordinates of the four corners of the region.
In addition, the character string recognition unit 62 of the present embodiment further has a second positional relationship in which regions including the region represented by the coordinates of one point or a plurality of points of each recognized character are determined in advance. A plurality of characters in (the first positional relationship will be described later) is recognized as one character string. In that case, an area including the area represented by the coordinates of one point or a plurality of points where the character string is recorded on the unfilled document is set as the first area and associated with the character string. As this “second positional relationship”, for example, a positional relationship in which they are arranged side by side within a predetermined second threshold distance is employed.
Furthermore, when a ruled line is drawn on the unfilled document, the character string recognition unit 62 of this embodiment recognizes the character string for each area surrounded by the ruled line. This is because when a ruled line is drawn, the recognition rate is expected to be improved by using the ruled line for character string recognition.
Further, the additional recorded image extraction unit 63 extracts an additional recorded image that is an image additionally recorded with respect to the unwritten original from the completed original. Specifically, for example, by calculating an image of the difference between the recorded document and the unfilled document, an additional recorded image that is an additionally recorded answer image is extracted.

  Here, the additional image extraction unit 63 according to the present embodiment is an entry in which the individual additional recording image on the completed document is recorded for each individual additional recording image for each individual recording constituting the additional recording image. An area on the finished document including an area represented by coordinates of one point or a plurality of points is extracted. Here, the extracted area on the completed manuscript is distinguished from the first area and is referred to as a “second area”. This second area corresponds to an example of the second area referred to in the present invention. The “second region” is preferably extracted with the same coordinate system after aligning the unfilled document and the filled document. Further, the “second region” may be expressed as, for example, a coordinate of one point or may be expressed as a collection of coordinates of four points.

Further, the character string specifying unit 64 specifies a character string recorded in an area corresponding to the additional recording image on the unfilled document from the character strings recognized by the character string recognition unit 62.
Here, the character string specifying unit 64 in the present embodiment specifies a first area that has a predetermined first positional relationship with respect to the second area, and associates the first area with the first area. Identify the character string Here, as an example, “if there is a first region overlapping the second region, or even if there is no first region overlapping the second region, the second region If there is a first area that is within a first threshold distance that is predetermined and that is aligned to the right of the second area, the first area is “predetermined for the second area. Specified as the first region ”.
In addition, the character string specifying unit 64 in the present embodiment corresponds to the plurality of second areas, and when the same character string associated with the same first area is specified, the same first string is specified. The same character string specified in the remaining specification excluding one specification among a plurality of times of specification for the same character string associated with the region is ignored. For example, when the same character string recorded in the same first area is specified a plurality of times, the character string specifying unit 64 in the present embodiment corresponds to the plurality of second areas. The same character string specified after the second time is ignored except the first time.
Further, the character string specifying unit 64 in the present embodiment, when there is no first area in the first positional relationship with respect to the second area, the character string corresponding to the second area Is not specified. This is because misrecognition increases if it is forcibly specified.

  FIG. 3 is a diagram showing a flowchart of an image processing program as an embodiment of the present invention.

  The scanner 20 shown in FIG. 1 reads the image on the document to generate image data, and the generated image data is input to the notebook PC 30 via the communication cable 40. Then, the image processing program shown in FIG. 3 is activated, and the image data input to the notebook PC 30 via the communication cable 40 is acquired (step S01). Note that, as described above, unless otherwise necessary, “data” may be omitted and referred to as “image” or “original” even for images on data.

  When the image is acquired in step S01, it is determined whether the image acquired this time is the first image or the second and subsequent images (step S02).

  In the present embodiment, a rule is set such that the scanner 20 reads an unfilled original on the first sheet and then sequentially reads the completed original on the second and subsequent sheets. Therefore, when the acquired image is the first image, the image processing program temporarily stores the image as an unfilled document (step S03). Image acquisition is repeated for the second and subsequent sheets (step S05), and all images acquired for the second and subsequent sheets are temporarily stored as completed documents (step S04).

FIG. 4 is a diagram showing a first example of an unfilled document and a filled document.
Here, FIG. 4A shows a questionnaire sheet before filling, that is, an unfilled manuscript 51A. Here, there are three questions (1) to (3) as the contents of the questionnaire, and the answer to these three questions is that number by enclosing one of the numbers 1 to 5 with a circle. It is a method of selecting.

  FIG. 4B shows a completed manuscript 52A in which a respondent enters an answer on a questionnaire sheet in the same format as the questionnaire sheet shown in FIG. The completed manuscript is not limited to one, but each of the second and subsequent manuscripts among the plurality of manuscripts sequentially read by the scanner 20 is treated as a completed manuscript.

  In one completed manuscript 52A shown in FIG. 4B, the number “3” is surrounded by a circle 521 for the question (1). Regarding the question (2), the numeral “1” is surrounded by a circle 522. Further, regarding the question (3), the numeral “5” is surrounded by a circle 523.

FIG. 5 is a diagram showing a second example of a blank document and a blank document.
As in the case of FIG. 4, FIG. 5 (A) shows a questionnaire sheet before entry, that is, an unfilled manuscript 51B. FIG. 5B shows a completed manuscript 52B in which the respondent enters an answer on a questionnaire sheet having the same format as the questionnaire sheet shown in FIG. The completed manuscript is not limited to one, but each of the second and subsequent manuscripts among the plurality of manuscripts sequentially read by the scanner 20 is treated as a completed manuscript.

  Here, there are four questions (1) to (4) as the contents of the questionnaire, and the answers to (1) to (3) of those questions are “very good”, “good”, “normal”, By putting a mark such as ○ on top of either “bad” or “very bad”, and for the question (4), “I want to introduce it”, “Well to introduce”, “ This is a method of selecting the contents with the mark by putting a mark such as a circle over the item “I do not want to introduce”.

FIG. 6 is a diagram showing a third example of a blank document and a blank document.
As in the case of FIG. 4 and FIG. 5, FIG. 6 (A) shows a questionnaire sheet before entry, that is, an unfilled manuscript 51C. FIG. 6B shows a completed manuscript 52C in which the respondent enters an answer on a questionnaire sheet in the same format as the questionnaire sheet shown in FIG. 6A. The completed manuscript is not limited to one, but each of the second and subsequent manuscripts among the plurality of manuscripts sequentially read by the scanner 20 is treated as a completed manuscript.

  Here, there are two questions (1) and (2) as the contents of the questionnaire, and the answers to these two questions are made by entering a check mark in the square as shown in FIG. 6 (B). Is called. When a check mark is entered in the □ mark, the contents represented by the character string recorded on the right side of the □ mark in which the check mark is written are answered.

FIG. 7 is a diagram showing a fourth example of an unfilled document and a filled document.
As in the case of FIGS. 4 to 6, FIG. 7A shows a questionnaire sheet before filling, that is, an unfilled manuscript 51D. FIG. 7B shows a completed manuscript 52D in which the respondent enters an answer on a questionnaire sheet in the same format as the questionnaire sheet shown in FIG. 7A. The completed manuscript is not limited to one, but each of the second and subsequent manuscripts among the plurality of manuscripts sequentially read by the scanner 20 is treated as a completed manuscript.

  Here, there are two questions (1) and (2) having the same contents as those in FIG. 6 as the contents of the questionnaire. However, here, the character strings as answer candidates are recorded in a frame surrounded by ruled lines. As shown in FIG. 7B, the answer to this question is made by entering a check mark in a frame surrounded by a ruled line that is arranged to the left of the character string to be answered. When a check mark is entered in the frame, the content represented by the character string recorded in the frame arranged immediately to the right of the check mark frame is answered.

  Returning to FIG. 3, the description will be continued.

  When the series of image acquisition is completed (step S05), the character string recorded in the unfilled document and the area in which the character string is recorded are recognized (step S06). In the first example shown in FIG. 4, it is sufficient to recognize only numbers, but since the recognition target in this embodiment covers a wide variety of questionnaire forms as shown in FIGS. 4 to 7, the character types of characters to be recognized are limited. Not.

  FIG. 8 is a diagram showing a flowchart of character string and area recognition processing on an unfilled document. In step S06 of FIG. 3, the process shown in FIG. 8 is executed.

  Here, first, for each character recorded on the unfilled document, the character and the region (first region) where the character is recorded are recognized (step S61). The recognition of the characters and the first area is performed over the entire surface of the unwritten document.

  FIG. 9 is a diagram illustrating a method for calculating the first region associated with the recognized character.

  Here, as shown in FIG. 9, it is assumed that the number “3” is recognized. At this time, a rectangle R circumscribing the numeral “3” is calculated, and the rectangle R is recognized as the first region corresponding to the recognized numeral “3”, and the rectangle R is recognized here. The first area corresponding to the number “3” is associated with the number “3”. However, the first region associated here is not necessarily a two-dimensionally spread region, for example, a set of coordinates C1 to C4 at the four corners of the rectangle R, or 1 at the center of the rectangle R. The coordinates C0 of the point may be used.

  Returning to FIG.

  When the individual characters on the unwritten manuscript and the individual first areas corresponding to the individual characters are recognized (step S61), the ruled line on the unwritten manuscript is then recognized (step S62). . Here, as shown in FIG. 4 to FIG. 6, questionnaire papers without ruled lines and questionnaire papers with ruled lines as shown in FIG. 7 are also processed. For this reason, a ruled line may be drawn depending on the questionnaire sheet. Therefore, here, the presence or absence of a ruled line on the unfilled document and the position and length of the ruled line when the ruled line is drawn are recognized.

  When the recognition of individual characters and individual first areas (step S61) and ruled line recognition (step S62) are completed, the unprinted document is inspected sequentially from upper left to lower right ( Step S63) When the recognized character is found, the found one character and the first area corresponding to the character are taken out (Step S64). The following processing is repeated until there are no more characters to be extracted (step S65).

  Here, first, it is determined whether or not one character extracted this time is the first character on the left side of one line (step S66). The following process cannot be executed with only the first character on the left side of one line. If one character extracted this time is the first character on the left side of one line, the process returns to step S63, and the next character And a first area corresponding to the character is extracted (step S64).

  Next, it is determined whether there is a ruled line surrounding the previously extracted character (step S67).

  Here, first, a case where there is no ruled line (ruled line surrounding the previously extracted character) will be described. If there is no ruled line that satisfies the above condition, then whether or not the join condition for joining the two first areas corresponding to the two characters of the character extracted first and the character extracted this time is satisfied. Determination is made (step S68). Here, as this combination condition, the first area corresponding to the character extracted this time is right next to the first area corresponding to the previously extracted character and is within a predetermined threshold distance. The condition that it exists is adopted.

  FIG. 10 is an explanatory diagram of a coupling condition for coupling the first regions.

  Here are the letters “Please answer the following questions”. Then, it is assumed that the previously extracted character is “I” and the character extracted this time is “below”. Here, the first area surrounding the character “lower” corresponding to the character “lower” extracted this time (herein referred to as “the first area of this time”) is extracted first. Corresponding to the character “and”, located right next to the first region surrounding the character “and” (herein referred to as the “first region”), and It exists within a predetermined threshold distance. If this combination condition is satisfied, the two first regions are combined so that they become one first region corresponding to the character string “below” consisting of two characters “below” and “below”. (FIG. 8, step S69).

  The determination as to whether or not the current first region is within a predetermined threshold distance with respect to the previous first region is not limited to a specific determination method, but for example, the following determination method: Is adopted.

  For example, as shown in FIG. 10A, the right side of the previous first area (the area surrounding the character “following”) and the left side of the current first area (the area surrounding the character “below”) A distance between the two is calculated, and it is determined whether or not the distance is within a threshold distance.

  Alternatively, as shown in FIG. 10 (A), each of the coordinates of the four corners of the first region (region surrounding the character “以”) and the current first region (character “below”) It is also possible to calculate the distance between each of the four corner coordinates of the (enclosing region) and determine whether or not there is a distance within the threshold distance among these distances.

  Alternatively, as shown in FIG. 10B, the distance between the center coordinates of the previous first area and the center coordinates of the current first area is calculated, and whether the distance is within a threshold distance or not. You may judge by.

  However, a threshold distance suitable for the determination method is employed according to these different determination methods. Or you may use these several determination methods together.

  When it is determined that the combination condition is satisfied by such a determination method, the first regions are combined (step S69). Then, the above processing is repeated until extraction of all characters on the unfilled document is completed (step S65). In the case of the character string shown in FIG. 10, by repeating this process, as shown in FIG. 10C, one first region corresponding to the entire character string “Please answer the following question” is generated.

  Next, a case where there is a ruled line surrounding the extracted character will be described.

  In this case, if it is determined in step S67 in FIG. 8 that there is a ruled line surrounding the previously extracted character, the process proceeds to step S71. It is determined whether or not it exists in the same ruled line area as the enclosed frame. If it is determined that these characters exist in the same ruled line area (in the same frame surrounded by ruled lines), the two first areas corresponding to these characters are combined (step S72).

  FIG. 11 is a diagram illustrating an example of combining the first areas using ruled lines.

  Here, each character string “Ver7.0”, “Ver7.1”, “Ver8.0”, “Ver8.02”, “Ver8.05” is surrounded by one ruled line area (ruled line). In the same frame). Therefore, here, for example, “Ver 7.0”, the first areas of the individual characters “V”, “e”, “r”, “7”, “.”, And “0” are combined, and the character string “ One first area corresponding to “Ver 7.0” is generated. The same applies to other character strings.

  In this way, the first region for each character string is generated by executing the processing shown in FIG. Here, when the character extracted this time is separated from the previously extracted character, and the next extracted character is also separated, a character string composed of only one character is formed.

  When the first region corresponding to each character string including only one character is generated by the process shown in FIG. 8, that is, the process of step S06 of FIG. 3, next, step S07 of FIG. Proceed to Here, one of the completed manuscripts temporarily stored in step S04 is taken out (step S07). However, the completed manuscript for which the character recognition processing in step S09 has been completed is not taken out. When there is an unprocessed completed document, that is, when an unprocessed completed document can be taken out (step S08), a character recognition process is performed on the one unprocessed completed document that has been taken out. Is executed (step S09). Details of the character recognition process will be described later.

When the unprocessed completed manuscript cannot be taken out, that is, when the character recognition process (step S09) is completed for all the completed manuscripts (step S08), the current character recognition routine is terminated.
FIG. 12 is a diagram showing a detailed flow of the character recognition process shown in one step (step S09) in FIG.

Here, first, an image of a difference between one completed original document taken out in step S07 in FIG. 3 and an unwritten original document temporarily stored in step S03 is generated (step S21).
FIG. 13 is a diagram illustrating an example of the difference image.
The difference image 53A shown in FIG. 13 is between the unfilled document 51A shown in FIG. 4A and the uppermost one of the filled documents 52A shown in FIG. 4B. It is a difference image. In this difference image 53A, three circles 521, 522, and 523 entered by the respondent are extracted. The additional recording image that appears on the difference image corresponds to an example of the additional recording image referred to in the present invention. Here, each image constituting the additional recording image is referred to as an individual additional recording image. Here, the ◯ mark 521 is divided into two parts 521a and 521b due to a wrinkle at the time of entry. For this reason, each of the two parts 521a and 521b becomes an individual additional recording image.
Returning to FIG. 12, the description will be continued.
When the difference image as illustrated in FIG. 13 is generated (step S21), the difference image is then inspected from the upper left to the lower right (step S22). One individually added recorded image is taken out (step S23). Then, the following processing is repeated until there is no unprocessed individual additional recording image on the current difference image (step S24).
Here, first, an area calculation process is performed (step S25). This area calculation process is a process for calculating the area (second area) on the completed document of one individual additional recording image taken out in step S23. In the present embodiment, in calculating the second area, the same calculation method as that of the first area on the unfilled document shown in FIG. 9 is employed. That is, here, a rectangle R circumscribing one individual additional recording image taken out in step S23 is calculated, and the rectangle R corresponds to the individual additional recording image as a second area corresponding to the individual additional recording image. Attached. Alternatively, as in the case of the first area, the coordinates of the four corners of the rectangle R or the center coordinates of the rectangle R may be set as the second area.
When the second area corresponding to one individually added recording image is calculated (step S25), a character string specifying process is performed (step S26).

  FIG. 14 is a diagram showing a detailed flow of the character string specifying process.

  Here, it is determined whether or not the second area calculated this time in step S25 of FIG. 12 overlaps any of the first areas (step S261).

  FIG. 15 is a diagram illustrating an example in which the second region and the first region overlap.

  In any of FIGS. 15A to 15C, the first area (area surrounding the character string) overlaps with the second area (area surrounding the circle). When calculating the distance between the center coordinates to determine whether or not they overlap, the center coordinates of the second area among the center coordinates of the second area and the center coordinates of each character constituting the character string It may be determined whether or not the distance from the center coordinate of the character closest to the coordinates is within a threshold distance.

  Returning to FIG. 14, the description will be continued.

  As illustrated in FIG. 15, when there is a first region that overlaps the second region, the character string corresponding to the first region that overlaps the second region, that is, the current region. It is specified as a character string corresponding to the individually added recording image (step S262).

  When there is no first area overlapping the second area (step S261), it is determined whether the first area exists on the right side of the second area and within a threshold distance. (Step S263). As this determination method, various determination methods similar to the above-described determination method for determining whether or not to combine the first regions can be employed. However, when the determination is made based on the distance between the center coordinates of the characters, the center coordinates of the leftmost character in the character string corresponding to the first region to be determined is adopted.

  FIG. 16 is a diagram illustrating an example in which the first region exists on the right side of the second region.

  Here, there is a first area that encloses the character string “Ver7.0” on the right side of the second area that encloses the check mark written in the square and within the threshold distance. Therefore, “Ver7.0”, which is a character string corresponding to the first area surrounding the character string of “Ver7.0”, corresponds to the second area, that is, the check mark which is the individual additional recording image this time. Is specified as a character string (FIG. 14, step S264). When there are a plurality of first regions on the right side of the second region and within the threshold distance, the first of the plurality of first regions having the shortest distance from the second region A character string corresponding to the area is specified as a character string corresponding to the second area.

  When there is no first area overlapping the second area, and there is no first area within the threshold distance on the right side of the second area, the second area of this time, That is, no character string is specified for the individual additional recording image this time (step S265).

  Returning to FIG. 12, the description will be continued.

When the character string specifying process shown in FIG. 14 corresponding to this one second area, that is, the character string specifying process in step S26 of FIG. 12 corresponding to this one second area is completed, The first character string corresponding to the character string identified earlier in step S26, in which the character string identified in this way passes a plurality of times while the individual additional recorded images are taken out and processed one by one in step S22. It is determined whether or not the character string of the first area is the same as the area (step S27).
For example, a ◯ mark 521 shown in FIG. 13 is divided into two parts 521a and 521b due to a crease at the time of filling in the ◯ mark. For this reason, each of the portions 521a and 521b may be recognized as one individual additional recording image. In this case, the same character string (here, the numeral “3” shown in FIG. 4) having the same coordinates is specified in the two portions 521a and 521b. In such a case, step S27 in FIG. 12 does not require the same character string in the same first area specified after the second time, so the same first area specified after the second time is the same. The string of is ignored.
In step S27, when a character string in the first area different from the previous one is specified, the specified character string is stored (step S28).
Here, the above processing is executed for each individual additional recording image on one difference image (steps S22 and S23), and processing for all the individual additional recording images on that one difference image. Is completed (step S24), the processing shown in FIG. 12 for the one difference image, that is, the character recognition processing shown as step S09 in FIG. 3 ends, and character recognition relating to the unprocessed next completed manuscript is completed. The process proceeds (step S07 in FIG. 3). When the character recognition processing for all completed originals is completed (step S08 in FIG. 3), the current execution of the image processing routine is terminated.
As described above, according to the present embodiment, for each position of the mark on the mark sheet, the respondent without any prior setting such as inputting in advance information on what the mark at that position means. Can be recognized.

  Here, in the case of the present embodiment, a plurality of originals are continuously read by the scanner 20, the first original of the plurality of originals is set as an unfilled original, and the second and subsequent originals are already entered. There is a rule that it is a manuscript. In this case, the image data of the unfilled document can be acquired easily and reliably. However, in the present invention, the rule that an unwritten document is arranged at a specific position such as the first sheet is not necessarily required. For example, a plurality of unwritten manuscripts may be sandwiched between half-finished manuscripts stacked. In that case, a process of finding an unfilled document from a plurality of documents in the image acquisition unit may be performed. As an example of the process for finding an unfilled document, a process for extracting the difference between the first document and the second and subsequent documents is sequentially performed, and a document with a difference only in the first document is unfilled. And it is sufficient. Also, an image is created by extracting the common part of multiple originals on the data obtained by reading regardless of whether it is an unfilled manuscript or a filled manuscript, and the created image and each read manuscript A document having the highest degree of matching may be used as an unfilled document.

  Alternatively, even if the completed manuscript is read multiple times, the unwritten manuscript is read only once for the same type of manuscript. An unfilled manuscript corresponding to the filled manuscript read this time may be specified by pattern matching with the manuscript, feature point extraction, or form recognition using the degree of coincidence of the areas surrounded by straight lines.

  Furthermore, in the present invention, it is not always necessary to read a blank document. For example, an unfilled manuscript may be created on data by extracting a common part of the plurality of filled manuscripts from a plurality of manuscripts. In this case, the unfilled document on the data created by extracting the common part corresponds to the first image data representing the first image.

  Here, as shown in FIG. 1, the image processing apparatus including the notebook PC 30 connected to the scanner 20 via the communication cable 40 has been described. However, the image processing apparatus according to the present invention is not necessarily in this form. For example, the function of the image processing apparatus according to the present invention may be incorporated into a copier in which a scanner and a printer are combined, or a multi-function machine having more functions. Furthermore, the function of the image processing apparatus of the present invention may be incorporated in a portable terminal having a camera function. In this case, an image obtained by photographing a document with the camera function is a character recognition target.

DESCRIPTION OF SYMBOLS 10 Character recognition system 20 Scanner 21 Document tray 22 Discharge tray 23 Top cover 30 Notebook personal computer (notebook PC)
31 Display Screen 32 Keyboard 51A, 51B, 51C, 51D Unfilled Document 52A, 52B, 52C, 52D Filled Document 53A Difference Image 521, 522, 523 ○ Mark 551, 552 Individual Additional Recorded Image 60 Image Processing Unit 61 Image Acquisition Unit 62 Character string recognition unit 63 Additional recorded image extraction unit 64 Character string identification unit

Claims (7)

An image acquisition unit configured to acquire first image data representing a first image and second image data representing a second image additionally recorded on the first image data;
A character string recognition unit for recognizing a character string including one character from the first image;
An additional recorded image extraction unit that extracts an additional recorded image that is an image additionally recorded with respect to the first image from the second image;
An image processing comprising: a character string specifying unit that specifies a character string corresponding to the additional recording image on the first image from the character strings recognized by the character string recognition unit. apparatus. In addition to recognizing the character string from the first image, the character string recognizing unit records one or more points on which the character string is recorded on the first image for each recognized character string. A first region including the region represented by the coordinates of
The additional recorded image extraction unit records the individual additional recorded image on the second image for each individual additional recorded image that constitutes the additional recorded image. Extracting a second region including the region represented by the coordinates of the point;
2. The character string specifying unit specifies a character string associated with a first area having a predetermined first positional relationship with respect to the second area. Image processing apparatus.
A plurality of characters in which the character string recognition unit recognizes each of the characters including a region represented by one or a plurality of coordinates in a predetermined second positional relationship. Are recognized as one character string, and the first image includes a region including the region represented by the coordinates of one point or a plurality of points on which the character string is recorded. The image processing apparatus according to claim 2, wherein the image processing apparatus is associated with the character string as an area of the image.
The character string specifying unit does not specify a character string corresponding to the second area when the first area is not in the first positional relationship with respect to the second area. The image processing apparatus according to claim 2, wherein the image processing apparatus is an image processing apparatus.
  The character string recognition unit recognizes a character string for each area surrounded by the ruled line when the first image is an image including a ruled line. The image processing apparatus according to any one of the above.   When the character string specifying unit specifies the same character string corresponding to the same first region in correspondence with the plurality of second regions, the character string specifying unit performs a plurality of times for the same character string. 6. The image processing apparatus according to claim 2, wherein the same character string that is specified in the remaining specifications other than one specification among the specifics is ignored. . The information processing apparatus is executed in an information processing apparatus that executes a program.
An image acquisition unit configured to acquire first image data representing a first image and second image data representing a second image additionally recorded on the first image data;
A character string recognition unit for recognizing a character string including one character from the first image;
An additional recorded image extraction unit that extracts an additional recorded image that is an image additionally recorded with respect to the first image from the second image;
Operating as an image processing apparatus including a character string specifying unit that specifies a character string corresponding to the additional recording image on the first image from among character strings recognized by the character string recognition unit. An image processing program characterized by the above.

Images