JP2019008775A - Image processing device, image processing system, image processing method, program - Google Patents

Abstract

To record any recording object character in any document corresponding to the type of a format even when the number of format types of documents is large.SOLUTION: An image processing device includes: a feature quantity extraction unit for extracting, on the basis of the result of a character recognition processing of a plurality of form image data, the feature quantity indicating the feature of a character string included in each of the form image data; and a reading object feature quantity generation unit for generating a first feature quantity as the feature quantity of the recording character string using the feature quantity corresponding to the form image data.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus, an image processing system, an image processing method, and a program.

  There is an image processing apparatus that performs OCR (Optical Character Recognition) processing on an image of a document acquired using a scanner or the like and reads the document. When there are various document formats, such an image processing apparatus stores information on the format definition of the document in advance and specifies the reading target area to improve the accuracy of the OCR processing. A technique related to this is disclosed in Patent Document 1.

JP 2016-48444 A

  However, when there are many types of document formats as described above, it is laborious for the operator to store all the format definition information in the image processing apparatus.

  Therefore, an object of the present invention is to provide an image processing apparatus, an image processing system, an image processing method, and a program that solve the above-described problems.

  According to the first aspect of the present invention, the image processing apparatus obtains a feature amount indicating a character string feature included in the form image data based on a result of character recognition processing on a plurality of form image data. A feature amount extraction unit that extracts each data, and a read target feature amount generation unit that generates a first feature amount that is a feature amount of the recorded character string using the feature amount corresponding to the form image data. .

  According to the second aspect of the present invention, in the image processing method, based on a result of character recognition processing on a plurality of form image data, a feature amount indicating a feature of a character string included in the form image data is calculated. Extracting for each data, and generating a first feature quantity that is a feature quantity of the recorded character string using the feature quantity corresponding to the form image data.

  According to the third aspect of the present invention, the program causes the computer to display a feature amount indicating a feature of a character string included in the form image data based on a result of character recognition processing of a plurality of form image data. It is a program for executing a step of extracting for each image data and a step of generating a first feature amount that is a feature amount of the recorded character string using the feature amount corresponding to the form image data.

  According to the present invention, even when there are many document format types, it is possible to record any character to be recorded in any document according to the format type.

It is a figure which shows the outline | summary of the image processing system containing an image processing apparatus. It is a figure which shows the hardware constitutions of an image processing apparatus. It is a functional block diagram of an image processing apparatus. It is a figure which shows an example of a document form. It is a figure which shows the outline | summary of the data table which a database memorize | stores. It is a 1st figure which shows the processing flow of the image processing apparatus by 1st embodiment. It is a 2nd figure which shows the processing flow of the image processing apparatus by 1st embodiment. It is a functional block diagram of the image processing apparatus by 2nd embodiment. It is a 1st figure which shows the processing flow of the image processing apparatus by 2nd embodiment. It is a 2nd figure which shows the processing flow of the image processing apparatus by 2nd embodiment. It is a figure which shows the processing flow of the image processing apparatus by 6th embodiment. It is a figure which shows the example of a connection of the image processing system by 7th embodiment, and the existing OCR system. It is a figure which shows the processing flow of the image processing apparatus by 6th embodiment. It is a figure which shows the outline | summary of the image processing system 200 by 8th embodiment. It is a figure which shows the processing flow of the terminal device by 8th embodiment. It is a figure which shows the processing flow of the server apparatus by 8th embodiment. It is a figure which shows the minimum structure of an image processing apparatus.

Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram showing an outline of an image processing system including an image processing apparatus according to the present embodiment.
As shown in FIG. 1, the image processing system 100 includes an image processing apparatus 1, an image reading apparatus 2, a recording apparatus 3, and a database 4.
The image processing apparatus 1 is connected to the image reading apparatus 2 via a communication cable. The image reading device 2 optically acquires image data such as a document form and outputs it to the image processing device 1. The image processing apparatus 1 recognizes characters by performing OCR processing on image data of a document form. The image processing apparatus 1 outputs the character recognition result to the recording device 3, and the recording device 3 records the character recognition result in the database. The database 4 is connected to the image processing apparatus 1 and the recording apparatus 3. The database 4 stores a correspondence relationship between image data of a plurality of document forms registered in the past from the recording device 3 and a recorded character string indicating a character string to be recorded among character strings included in the image data. The character string indicated by the recorded character string is an important character string to be recorded and stored in the database 4 among the character strings described in the document form. An operator who uses the image processing system 100 registers in advance in the database 4 the image data of a plurality of document forms registered in the past using the recording device 3 and the recorded character strings among the character strings included in the image data. Keep it.

  The recording device 3 has sufficient correspondence between the image data of the document form and the information of the recorded character string indicating the character string to be recorded among the information of the character string included in the image data for many document forms. It shall be recorded. In such a state, the image processing apparatus 1 performs processing.

FIG. 2 is a diagram illustrating a hardware configuration of the image processing apparatus.
As shown in FIG. 2, the image processing apparatus 1 includes a CPU (Central Processing Unit) 11, an IF (Interface) 12, a communication module 13, a ROM (Read Only Memory) 14, a RAM (Random Access Memory) 15, an HDD (Hard Disk). Drive) 16 or the like. The communication module 13 may perform wireless communication with the image reading device 2, the recording device 3, and the database 4, or may perform wired communication, and may have these two functions. Good.

FIG. 3 is a functional block diagram of the image processing apparatus.
The CPU 11 of the image processing apparatus 1 has the functions of a control unit 101, an acquisition unit 102, a feature amount extraction unit 103, a read target feature amount generation unit 104, and a recording unit 105 by executing a stored program.

The control unit 101 controls other functional units.
The acquisition unit 102 acquires image data of a document form.
The feature amount extraction unit 103 extracts, for each image data of the document form, a feature amount indicating the feature of the recorded character string included in the image data of the document form based on the character recognition processing result of the image data of the plurality of document forms. .
The reading target feature value generation unit 104 generates a first feature value of the recorded character string using a feature value corresponding to the image data of the document form.
The recording unit 105 uses the first feature amount to extract and record a recorded character string from character string information read from image data of a new document form.

  By such processing, the image processing apparatus 1 reduces the labor of recording character string information to be recorded included in the image data of the new document form.

FIG. 4 shows an example of a document form.
As shown in this figure, in the document form, the mark of the company that created the document, the creation date, the person in charge of the creation, and the document content are described in a format specific to the document form. For example, if the document form is an order form, the document content indicates one or a plurality of information sets such as the name of the ordered product and the number of orders. Based on one document form, the operator records a specific record character string to be recorded among the character strings described in the document form in the database 4 using the recording device 3. Specifically, the operator inputs a specific recording character string to be recorded in the database 4 by the recording device 3 while looking at the document form. Further, the operator causes the image reading device 2 to read the image data of the document form. The document form is read by the image reading device 2 and output to the image processing device 1 based on the operation of the operator. Based on the operator's operation and the control of the image processing apparatus 1, the recording device 3 stores the image data for one document form and a specific recording character to be recorded among the character strings described in the document form. The columns are associated and recorded in the database 4. In the example of FIG. 4, the date 51, the supplier 52, the product name 53, the quantity 54, and the amount 55 are specific recording character strings to be recorded. Other information such as a non-recorded character string that is not recorded by the operator is also printed on the document form 5. The information includes, for example, the name 501 of the orderer who issued the document form, the emblem image 502 of the orderer, the title 503 of the document form, the greeting 504, and the like.

FIG. 5 is a diagram showing an outline of a data table stored in the database.
As shown in FIG. 5, the database 4 associates image data of a document form with a specific recording character string to be recorded among character strings described in the document form, and stores them in a recording table.

<First embodiment>
FIG. 6 is a first diagram illustrating a processing flow of the image processing apparatus according to the first embodiment.
Next, the processing flow of the image processing apparatus 1 will be described in order.
First, in the database 4, a combination of image data of a certain document form and a specific recording character string described in the document form are recorded for the same document form. For example, it is assumed that a plurality of specific recording character string information is recorded for the document form 5 shown in FIG. In this state, the worker activates the image processing apparatus 1 and instructs the image processing apparatus 1 to start processing.

  The acquisition unit 102 of the image processing apparatus 1 reads the image data of the document form 5 and the information of the recorded character string corresponding to the image data from the database 4 (step S601). The acquisition unit 102 outputs the image data and the recorded character string to the feature amount extraction unit 103. The feature amount extraction unit 103 performs OCR processing on the image data, and detects coordinates in the image data indicating all character strings in the image data and the range of the character strings (step S602). The character string is a group of characters composed of a plurality of characters. The feature amount extraction unit 103 analyzes the range of one group based on an interval with other characters, extracts one or a plurality of characters included in the range as a character string, and character strings in the image data The coordinates indicating the range of are detected. Characters included as a character string may include symbols such as ideograms and phonetic characters, marks, icon images, and the like.

  The feature amount extraction unit 103 compares the character string extracted from the image data by OCR processing with the recorded character string read from the database 4 together with the image data. Of the character strings extracted from the image data by OCR processing, the feature amount extraction unit 103 includes a character string in the image data that matches the character information of the recorded character string, a character attribute included in the character string, and a range of the character string. The coordinates are specified (step S603). Character attributes are information represented by numbers, alphabets, hiragana, kanji, number of characters, character height, font, and the like. The coordinates of the range of the character string are information indicating the coordinates of the first character and the end character included in the character string. The feature amount extraction unit 103 generates feature amounts including the specified information for a plurality of the same document forms 5 (step S604).

The feature amount is generated for each recorded character string in the document form 5 by the feature amount extraction unit 103. The read target feature amount generation unit 104 acquires the feature amounts in the plurality of the same document forms 5, and analyzes the character attributes included in each feature amount corresponding to the recorded character string and the coordinates indicating the character string range. One feature amount is generated for each. This feature amount analysis is performed by a process such as machine learning. Machine learning is also simply referred to as learning.
The feature amount generated by the reading target feature amount generation unit 104 through analysis such as machine learning is referred to as a first feature amount. That is, the reading target feature value generation unit 104 generates a first feature value for each of the recorded character strings in the document form 5 using a plurality of the same document form 5 (step S605). The first feature amount is a feature amount for recognizing a recorded character string, and includes a character attribute and coordinates indicating a character string range. The reading target feature value generation unit 104 records the first feature value of each of the one or more recorded character strings in the document form 5 in the database 4 in association with the identifier of the document form 5 (step S606).

  For example, the read target feature amount generation unit 104 includes the character attribute and the character string range coordinates of the date 51, the ordering party 52, the product name 53, the quantity 54, and the amount 55 that are the recorded character strings included in the document form 5. Etc. are recorded in the database 4 in association with the identifier of the document form 5.

  With the above processing, the image processing apparatus 1 can generate information (first feature amount) used to reduce the labor of recording the recorded character string of the worker by machine learning or the like and store it in the database 4. . As a result, the image processing apparatus 1 can automatically record the recorded character string in the database 4 based on the image data of the new document form. Hereinafter, the processing will be described.

FIG. 7 is a second diagram showing a processing flow of the image processing apparatus according to the first embodiment.
The operator performs an operation for causing the image reading apparatus 2 to read a new document form. As a result, the image reading device 2 generates image data of the document form and outputs it to the image processing device 1. The acquisition unit 102 of the image processing apparatus 1 acquires image data (step S701). The acquisition unit 102 outputs the image data to the feature amount extraction unit 103. The feature amount extraction unit 103 performs OCR processing on the image data to detect a character string, a character feature included in the character string, and coordinates in the image data within the range of the character string (step S702). The feature amount extraction unit 103 generates a third feature amount including the detected information for each character string in the image data (step S703). That is, the third feature amount is information indicating the feature of the character string included in the document form of the newly read image data. Thereafter, the group specifying unit 107 reads out one or more first feature amounts for the document form of the processing target image data from the database 4 (step S704). The group specifying unit 107 outputs the third feature value and the first feature value to the recording unit 105.

  The recording unit 105 acquires a third feature amount and one or more first feature amounts for one or more character strings in the image data. The recording unit 105 uses the coordinates indicating the range of the character string included in each first feature value to determine whether all the third feature values having coordinates corresponding to the coordinates indicated by the first feature values exist. (Step S705). When all the third feature values having coordinates corresponding to the coordinates of the first feature values exist, the description of characters exists in all the description items in the document form corresponding to the recorded character string. On the other hand, when there is no third feature amount having coordinates corresponding to the coordinates of each first feature amount, there is no description of characters in any description items in the document form.

  If YES in step S705, the recording unit 105 determines whether or not the character attribute included in the first feature amount matches the character attribute included in the corresponding third feature amount specified based on the coordinates. (Step S706).

  When the determination result in step S706 is YES and the character attributes match, the recording unit 105 forms a rectangular frame within the range of the recorded character string based on the coordinates indicated by one or more third feature amounts in the currently processed image data. Generate a confirmation screen displaying. The recording unit 105 outputs the confirmation screen to the monitor (step S707). The operator can confirm the rectangular area displayed on the confirmation screen and confirm the recorded character string that the image processing apparatus 1 intends to record. Thereby, the operator can confirm whether there is a shortage in the recorded character string. An icon image of either OK or NG button is displayed on the confirmation screen. By selecting an OK button among the icon images of this button, the operator can instruct that there is no shortage of selection as a recorded character string. On the other hand, by selecting an NG button among the icon images of the buttons, the operator can instruct that there is a lack in selection as a recorded character string.

  The reason for outputting the confirmation screen to the monitor will be described with reference to FIG. In FIG. 4, six product names 53 are entered in the recorded character string. In the past document form, when the entry of the six product names 53 is the maximum, the product name 53 is automatically determined as a recorded character string within a range of 1 to 6 with respect to a new document form. Therefore, for example, if seven product names 53 are described in a new form, steps S705 and S706 are both YES for the first to sixth parts, so that the image processing apparatus 1 is the seventh. Exit without recording the string. In order to improve such an event, the image processing apparatus 1 may display a confirmation screen before recording the recorded character string in step S707, and record and end the confirmation for the operator. Confirm.

  The recording unit 105 determines whether there is a lack of selection of the recording character string in response to the pressing of the icon image of the operator's button (step S708). If there is no shortage, the recording unit 105 records the character string included in the third feature amount in the recording table in association with the identification information of the document form (step S709).

  For example, it is assumed that the third feature value a3, the third feature value b3, the third feature value c3, and the third feature value d3 can be acquired from the image data of the document form. The third feature quantity a3 is recorded in advance in the database, the first feature quantity a1, the third feature quantity b3 is the first feature quantity b1, the third feature quantity c3 is the first feature quantity c1, and the third feature. It is assumed that the amount d3 matches the first feature amount d1 with the feature amount. In this case, the recording unit 105 records the character strings included in the third feature value a3, the third feature value b3, the third feature value c3, and the third feature value d3 in the record table of the document form.

  In the case of NO in step S705 described above, in the case of NO in step S706, or in the case of NO in step S708, the recording unit 105 has a third feature amount having a coordinate corresponding to the coordinate indicated by the first feature amount. If not, perform the process. Specifically, the recording unit 105 generates image data for input of a form image in which an input field is provided in the range of the coordinates of the first feature value in which the third feature value of the corresponding coordinate in the image data does not exist. Is output to the monitor (step S710). The input image data may be data described in a markup language such as HTML or XML. While viewing the input image data, the operator operates an input device such as a keyboard of the image processing apparatus 1 to input a recorded character string in the input field in the input image data displayed on the monitor. A save button is displayed on the input image data. When the save button is pressed, the recording unit 105 inputs a new feature to the input field of the input image data in addition to the third feature amount already acquired for the document form. A third feature amount including the character string thus generated is generated (step S711). The recording unit 105 records the identifier of the form image data and the character string input in the input field in association with each other in the database 4. The image processing apparatus 1 updates the first feature amount by performing the processing flow shown in FIG. 6 again, and can extend the range of character strings that can be automatically recorded. As a result, the next time the same document form is processed, the character string can be automatically recorded, thereby saving the operator from having to input the character string. The recording unit 105 records the character string included in each of the third feature amounts in the document form recording table (step S712).

According to such processing, the image processing apparatus 1 automatically uses the document form image data and the recorded character string previously recorded by the operator to record the recorded character string in the newly entered document form image data. Can be recorded. Therefore, the image processing apparatus 1 can reduce the labor of the operator who records the recorded character string in the document form.
Further, even when the recorded character string is not described in the document form, the image processing apparatus 1 outputs the input image data when the description items corresponding to the recorded character string that should originally be described are not described. . This makes it possible to find an error that has not been entered for a description item to be described in the document form, and to easily record a recorded character string indicated by the description item.

<Second embodiment>
FIG. 8 is a functional block diagram of the image processing apparatus according to the second embodiment.
As shown in this figure, the image processing apparatus 1 according to the second embodiment has functions of a group classification unit 106 and a group identification unit 107 in addition to the functional units shown in FIG. The hardware configuration of the image processing apparatus 1 is the same as the configuration shown in FIG.

FIG. 9 is a first diagram illustrating a processing flow of the image processing apparatus according to the second embodiment.
Next, the processing flow of the image processing apparatus 1 according to the second embodiment will be described in order.
In the database 4, a large number of combinations of image data for a plurality of different document forms and specific recording character strings described in each document form are recorded for each document form. For example, assume that a plurality of pieces of specific recorded character string information are recorded for each of the different document forms 5 shown in FIG. In this state, the worker activates the image processing apparatus 1 and instructs the image processing apparatus 1 to start processing.

  The acquisition unit 102 of the image processing apparatus 1 determines whether or not all the image data of the document form 5 and the recorded character string information corresponding to the image data have been read from the database 4 (step S901). In the case of NO, the acquisition unit 102 reads the image data of the document form 5 and information of the recorded character string corresponding to the image data from the database 4 (step S902). The acquisition unit 102 outputs the image data and the recorded character string to the feature amount extraction unit 103. The feature amount extraction unit 103 performs OCR processing on the image data to detect all character strings in the image data and coordinates in the image data (step S903). The character string is a group of characters composed of a plurality of characters. The feature amount extraction unit 103 analyzes the range of one group based on an interval with other characters, extracts one or a plurality of characters included in the range as a character string, and character strings in the image data The coordinates indicating the range of are detected. Characters included as a character string may include symbols such as ideograms and phonetic characters, marks, icon images, and the like.

  The feature amount extraction unit 103 compares the character string extracted from the image data by OCR processing with the recorded character string read from the database 4 together with the image data. Of the character strings extracted from the image data by OCR processing, the feature amount extraction unit 103 includes a character string in the image data that matches the character information of the recorded character string, a character attribute included in the character string, and a range of the character string. The coordinates are specified (step S904). Character attributes are information represented by numbers, alphabets, hiragana, kanji, number of characters, character height, font, and the like. The coordinates of the range of the character string are information indicating the coordinates of the first character and the end character included in the character string. The feature quantity extraction unit 103 generates a feature quantity including the specified information for one document form 5 (step S905).

  The feature amount is generated for each recorded character string in the document form 5 by the feature amount extraction unit 103. That is, the feature quantity extraction unit 103 generates a feature quantity for each document form and each recorded character string. This feature amount for each document form and each recorded character string is referred to as an individual first feature amount. The individual first feature amount may include one or both of a character attribute and a coordinate indicating a character string range. The feature amount extraction unit 103 records the individual first feature amount of each of the one or more recorded character strings in each document form 5 in the database 4 in association with the identifier of the document form 5 and the identifier of the recorded character string ( Step S906). As the identifier of the recorded character string, for example, a coordinate value indicating the position of the recorded character string can be used.

  For example, the feature amount extraction unit 103 includes the date 51, the ordering party 52, the product name 53, the quantity 54, and the amount 55 of the recorded character strings included in the document form 5, the character attributes, the coordinates indicating the range of the character strings, and the like. Each individual first feature amount shown is recorded in the database 4 in association with the identifier of the document form 5 and the identifier of the recording character string.

  The feature amount extraction unit 103 also identifies a non-recorded character string in the image data that does not match the character information included in the recorded character string, a character attribute included in the non-recorded character string, and a range coordinate. (Step S907). The feature amount extraction unit 103 generates a feature amount including the specified information for the document form 5 (step S908).

  The feature amount is generated for each non-recorded character string in the document form 5 by the feature amount extraction unit 103. That is, the feature amount extraction unit 103 generates a feature amount for each document form and for each non-recorded character string. This feature amount for each document form and for each non-recorded character string is referred to as an individual second feature amount. The individual second feature amount may include one or both of a character attribute and a coordinate indicating a character string range. The feature amount extraction unit 103 records each individual second feature amount of each of one or more non-recorded character strings in each document form 5 in the database 4 in association with the identifier of the document form 5 and the identifier of the non-recorded character string. (Step S909). As an identifier of the non-recorded character string, for example, a coordinate value indicating the position of the recorded character string can be used.

  For example, the feature amount extraction unit 103 includes individual second feature amounts indicating the orderer's name 501, the orderer's emblem image, the document form title 503, the greeting 504, etc., which are non-recorded character strings included in the document form 5. Are recorded in the database 4 in association with the identifier of the document form 5 and the identifier of the recording character string.

  The database 4 records image data of a plurality of different document forms 5 and information of recording character strings corresponding to the image data. The acquisition unit 102 of the image processing apparatus 1 repeats the processing from step S901 to step S911 until image data and recording character string information for all document forms 5 are read. Assume that it is determined in step S901 that all the data has been read (step S901: YES).

  In this case, the group classification unit 106 groups the document form 5 based on the individual second feature amount included in the image data of the document form 5 (step S912). For example, the group classification unit 106 groups each document form 5 based on the matching degree of the non-recorded character string indicated by the individual second feature amount, the matching degree of the emblem image, the matching degree of the coordinate range of the non-recording character string, and the like. Divide. The group classification unit 106 determines the group identifier of the document form 5 in this grouping process. The group classification unit 106 determines whether grouping has been completed for all the document forms 5 (step S913).

  The group classification unit 106 repeats the process of step S912 when the grouping of all the document forms 5 is not completed. When the grouping of all the document forms 5 is completed, the group classification unit 106 records the identifier of the document form 5 and the group identifier assigned to the document form 5 in the group table of the database 4 in association with each other. (Step S914).

  The reading target feature value generation unit 104 reads each individual first feature value and each individual second feature value of one or a plurality of document forms 5 belonging to a certain group from the database 4 and reads each individual document form 5 belonging to the group. Each group first feature value and each group second feature value corresponding to the first feature value and each individual second feature value are generated (step S915). Each group first feature value may be a value such as an average of the individual first feature values of the document forms 5 belonging to the group. Similarly, each group second feature value may be a value such as an average of the individual second feature values of the document forms 5 belonging to the group. Each group first feature amount, each group second feature amount is not the average of each individual first feature amount, the average of each individual second feature amount, using a method such as predetermined statistical processing or machine learning, As long as it is a feature amount calculated so as to be able to specify a recorded character string or a non-recorded character string of one or a plurality of document forms 5 belonging to the group, any technique can be used for each group first feature amount, A group second feature value may be generated. The read target feature value generation unit 104 calculates each group first feature value and each group second feature value for each group, and records them in the database 4 in association with the group identifier (step S916).

  With the above processing, the image processing apparatus 1 can generate information necessary for reducing the labor of recording the operator's recorded character string for each group of document forms and store it in the database 4. As a result, the image processing apparatus 1 can automatically record the recorded character string in the database 4 based on the image data of the new document form. Hereinafter, the processing will be described.

FIG. 10 is a second diagram showing a processing flow of the image processing apparatus according to the second embodiment.
The operator performs an operation for causing the image reading apparatus 2 to read a new document form. As a result, the image reading device 2 generates image data of the document form and outputs it to the image processing device 1. The acquisition unit 102 of the image processing apparatus 1 acquires image data (step S1001). The acquisition unit 102 outputs the image data to the feature amount extraction unit 103. The feature amount extraction unit 103 performs OCR processing on the image data, and detects a character string, a character feature included in the character string, and coordinates in the image data within the range of the character string (step S1002). The feature amount extraction unit 103 generates a third feature amount including the detected information for each character string in the image data (step S1003). The third feature amount is information indicating the character string feature included in the document form of the newly read image data.

  Next, the group specifying unit 107 reads a group second feature amount used for specifying a group of a new document form among the group second feature amounts from the database 4. The group second feature amount may be, for example, a feature amount corresponding to the orderer's emblem image 502 displayed in the image data of the document form. The group specifying unit 107 determines whether information indicated by a certain group second feature amount can be specified from the image data of the document form acquired in step S1001. The group specifying unit 107 performs the same processing using the group second feature amount for all groups. When the information that matches the group second feature value read from the database 4 can be specified from the image data of the newly read document form, the group specifying unit 107 newly reads the group having the group second feature value. The document form image data group is identified (step S1004). Thereafter, the group specifying unit 107 reads out one or more group first feature values for the group from the database 4 (step S1005). The group specifying unit 107 outputs the third feature amount and one or more group first feature amounts to the recording unit 105. The group first feature amount is a feature amount for specifying one or a plurality of recorded character strings in the document form belonging to the group.

  The recording unit 105 acquires a third feature amount and one or more group first feature amounts for one or more character strings in the image data. The recording unit 105 uses the coordinates indicating the range of the character string included in each group first feature amount to determine whether all the third feature amounts having coordinates corresponding to the coordinates indicated by each group first feature amount exist. Determination is made (step S1006). When all the third feature values having coordinates corresponding to the coordinates of the first feature values of each group exist, the description of characters exists in all the description items in the document form corresponding to the recorded character string. On the other hand, when all the third feature values having coordinates corresponding to the coordinates of the first feature values of each group do not exist, there is no description of characters in any description items in the document form.

  In the case of YES in step S1006, the recording unit 105 determines whether or not the character attribute included in the group first feature amount matches the character attribute included in the corresponding third feature amount specified based on the coordinates. (Step S1007).

  If the determination result in step S1007 is YES and the character attributes match, the recording unit 105 forms a rectangular frame in the range of the recorded character string based on the coordinates indicated by one or more third feature amounts in the currently processed image data. Generate a confirmation screen displaying. The recording unit 105 outputs the confirmation screen to the monitor (step S1008). The operator can confirm the rectangular area displayed on the confirmation screen and confirm the recorded character string that the image processing apparatus 1 intends to record. Thereby, the operator can confirm whether there is a shortage in the recorded character string. An icon image of either OK or NG button is displayed on the confirmation screen. By selecting an OK button among the icon images of this button, the operator can instruct that there is no shortage of selection as a recorded character string. On the other hand, by selecting an NG button among the icon images of the buttons, the operator can instruct that there is a lack in selection as a recorded character string.

  The recording unit 105 determines whether or not there is an insufficient selection of the recording character string in response to the pressing of the icon image of the operator's button (step S1009). If there is no shortage, the recording unit 105 records the character string included in the third feature amount in the recording table in association with the identification information of the document form (step S1010).

  For example, it is assumed that the third feature value a3, the third feature value b3, the third feature value c3, and the third feature value d3 can be acquired from the image data of the document form. And the third feature quantity g3 in which the third feature quantity a3 is recorded in the database in advance, the third feature quantity b3 is the group first feature quantity g12, the third feature quantity c3 is the group first feature quantity g13, Assume that the third feature value d3 and the group first feature value g14 have the same feature value. In this case, the recording unit 105 records the character strings included in the third feature value a3, the third feature value b3, the third feature value c3, and the third feature value d3 in the record table of the document form.

  In the case of NO in step S1006 described above, in the case of NO in step S1007, or in the case of NO in step S1009, the recording unit 105 has a third feature quantity having coordinates corresponding to the coordinates indicated by the group first feature quantity. Process when it does not exist. Specifically, the recording unit 105 generates image data for input of a form image in which an input field is provided in the coordinate range of the group first feature value in which the third feature value of the corresponding coordinate in the image data does not exist. And output to the monitor (step S1011). The input image data may be data described in a markup language such as HTML or XML. While viewing the input image data, the operator operates an input device such as a keyboard of the image processing apparatus 1 to input a recorded character string in the input field in the input image data displayed on the monitor. A save button is displayed on the input image data. When the save button is pressed, the recording unit 105 inputs a new feature to the input field of the input image data in addition to the third feature amount already acquired for the document form. A third feature amount including the character string thus generated is generated (step S1012). The recording unit 105 records the identifier of the form image data and the character string input in the input field in association with each other in the database 4. The image processing apparatus 1 updates the group first feature value and the group second feature value by executing the processing flow shown in FIG. 9 again, and can extend the range of character strings that can be automatically recorded. As a result, the next time the same document form is processed, the character string can be automatically recorded, thereby saving the operator from having to input the character string. The recording unit 105 records the character strings included in all the third feature amounts in the document form recording table (step S1013).

According to such processing, the image processing apparatus 1 uses the image data and recorded character strings of a plurality of different document forms recorded in advance by the operator, regardless of the type of the newly entered document form. The recorded character string in the image data of the document form can be automatically recorded. Therefore, the image processing apparatus 1 can reduce the labor of the operator who records the recorded character string in the document form.
Further, even when the recorded character string is not described in the document form, the image processing apparatus 1 outputs the input image data when the description items corresponding to the recorded character string that should originally be described are not described. . This makes it possible to find an error that has not been entered for a description item to be described in the document form, and to easily record a recorded character string indicated by the description item.

<Third embodiment>
As another example of the processing of the image processing apparatus 1, an operator may register a document form group in the image processing apparatus 1 in advance. For example, when registering image data of a document form in the past, an operator inputs a group identifier according to the type of document form and registers it in the database 4 in association with the image data of the document form. This prevents different forms from being mixed due to a processing error of the image processing apparatus 1 in the same group, and the group first feature value and group second feature value can be extracted with high accuracy. In this case, at the time of registration, the operator inputs a group of document forms, but for a new form, the group is specified using the group second feature amount as in step S1004.

<Fourth embodiment>
As another example of the processing of the image processing apparatus 1, in step S <b> 912, the image processing apparatus 1 not only uses the individual second feature value to group document forms but also uses the individual first feature value. Further, the document forms may be grouped using the individual first feature quantity together with the individual second feature quantity. The individual first feature value is the feature value of the recorded character string, but if it is the same type of document form, the coordinates of the recorded character string and its character attributes are considered to be the same. Forms can be grouped.
In this case, the acquisition unit 102 acquires a plurality of form image data and a recorded character string to be recorded among character strings included in the form image data. Then, the group classification unit 106 groups the form image data based on the individual first feature amount. Then, the reading target feature value generation unit 104 generates the group first feature value of the recorded character string for each group using the individual first feature value corresponding to the form image data included in the group.
Alternatively, the first grouping may be performed by an operator as shown in the third embodiment, and a new document form may be grouped using the individual first feature amount by the process of step S1004. As a result, the recorded character string can be read with high accuracy in the OCR process.

<Fifth embodiment>
In the second embodiment, a new form group is specified based on the second feature amount in step S1004. However, as another processing mode, the image processing apparatus 1 reads out the group first feature amount by sequentially specifying each group for all groups set by the worker without performing the process of specifying the group. The number that matches the third feature value is counted. In the case of a correct group, the group first feature value and the third feature value should match most, so the image processing apparatus 1 uses the character string included in each third feature value of the specific group when the number of matches is the largest. Is recorded in step S1008. As a result, the recorded character string can be recorded without specifying the group.
In this case, the image data of the document form stored in the database 4 for generating the group first feature amount may be grouped in advance by the operator. The acquisition unit 102 acquires a plurality of form image data and a recorded character string to be recorded among character strings included in the form image data. Then, the feature amount extraction unit 103 extracts the individual first feature amount indicating the feature of the recorded character string based on the result of character recognition processing of the form image data acquired by the acquisition unit 102. The reading target feature amount generation unit 104 generates a group first feature amount of a recorded character string for each group using the individual first feature amount corresponding to the form image data included in a predetermined group set in advance.

<Sixth embodiment>
In the process of FIG. 7 of the first embodiment, the image processing apparatus 1 may perform machine learning of the feature amount in both cases where the character recognition succeeds and fails. In the sixth embodiment, this point will be described.
6th Embodiment is the same as that of 1st Embodiment except the process of FIG. 7 of 1st embodiment, and FIGS.

FIG. 11 is a diagram illustrating a processing flow of the image processing apparatus according to the sixth embodiment. Steps S1101 to S1108 in FIG. 11 are the same as steps S701 to S708 in FIG. Steps S1110 to 1112 in FIG. 11 are the same as steps S709 to S711 in FIG. Step S1114 in FIG. 11 is the same as step S712 in FIG.
Therefore, in the process of FIG. 11, step S1109 is inserted between steps S708 and S709 of FIG. 7, and step S1113 is inserted between steps S711 and S712 of FIG. 7 is the same.
In either step S1109 or S1113, the read target feature value generation unit 104 of the image processing apparatus 1 performs the process of FIG. 6 to update the first feature value. The reading target feature amount generation unit 104 does not need to redo the process of FIG. 6 from an unlearned state, and may perform additional learning for one piece of image data.

Here, machine learning is not performed in step S1109 in FIG. 11 and machine learning is performed only in step S1113 so that learning is performed only when the OCR processing fails (when the recorded character string cannot be obtained properly by OCR processing). Can be considered. However, in this case, if the image processing apparatus 1 succeeds in the OCR process, the machine learning is not performed. As a result, the statistical information on the format of the document form may not be reflected in the machine learning.
For example, consider a case where up to 99 of 100 document forms have a recorded character string on the lower left and only one recorded character string on the upper right. In this case, if the image processing apparatus 1 succeeds in the OCR process in the document form having the recorded character string at the same position (step S1108: YES), if the recorded character string is in the lower left, the upper right In each case, learning is performed once. Although the actual ratio is 99: 1, learning by the image processing apparatus 1 is performed at a ratio of 1: 1, and there is a possibility of overlearning when the recorded character string is in the upper right. is there.
On the other hand, as shown in FIG. 11, when the image processing apparatus 1 succeeds in the OCR processing or fails, machine learning (updating the first feature amount) is performed, so that the format of the document form is statistically determined. Information can be reflected in machine learning.
Similarly, in the second to fifth embodiments, the image processing apparatus 1 performs the process of FIG. 9 between steps S1009 and S1010 of FIG. 10 and between steps S1012 and S1013 of FIG. Also good.

<Seventh embodiment>
The image processing apparatus 1 may acquire data from a system that scans or captures a document form, such as an existing OCR system, and performs machine learning. In the seventh embodiment, this point will be described. Hereinafter, a case where the image processing apparatus 1 acquires data from an existing OCR system will be described as an example. However, the data provider to the image processing apparatus 1 is not limited to the existing OCR system, and can be various systems or apparatuses. For example, the image processing apparatus 1 may acquire data from a system (for example, a sales information system or an accounting information system) that stores image data serving as a sales date, sales destination, amount, and trail in a database. Good. In particular, the information providing system or apparatus does not need to have the OCR function.

  FIG. 12 is a diagram illustrating a connection example between the image processing system and the existing OCR system according to the seventh embodiment. In the configuration shown in FIG. 12, an existing OCR system 9 is connected to the image processing apparatus 1 in addition to the configuration in FIG. 1. With this configuration, the existing OCR system 9 uses an image data generation function (scan function) and a user interface, and does not use a character recognition function. Character recognition is performed by the image processing apparatus 1. Other points are the same as in the first embodiment. The acquisition of image data and the user interface with the aid of FIGS. 1 to 7 are read as using the existing OCR system 9.

FIG. 13 is a diagram illustrating a processing flow of the image processing apparatus according to the seventh embodiment. FIG. 13 shows the relationship between the processing performed by the image processing apparatus 1 and the processing performed by the existing OCR system 9.
In the process of FIG. 13, the existing OCR system scans according to the operator's operation and generates image data (sequence S1301). The existing OCR system 9 transmits the obtained image data to the image processing apparatus 1 (sequence S1302).
When receiving the image data, the image processing apparatus 1 performs OCR processing (sequence S1303). For example, the image processing apparatus 1 performs the processing from step S702 onward in FIG. 7 to obtain a recorded character string. In particular, the recording unit 104 identifies a recorded character string from the character string of the OCR processing result.
The image processing apparatus 1 transmits the character string (recorded character string) of the character recognition result to the existing OCR system 9 (sequence S1304).
The existing OCR system 9 presents the character recognition result to the operator, and receives a confirmation operation after confirming and correcting the character recognition result (sequence S1305). The existing OCR system 9 transmits the confirmed character string (corrected result recording character string) to the image processing apparatus 1 (sequence S1306).
When receiving the correction result, the image processing apparatus 1 performs machine learning (sequence S1307).
The image processing apparatus 1 acquires image data in sequence S1302, and acquires a correction result character string determined by the operator in sequence S1306. Using this image data and character string, for example, machine learning can be performed by the processing procedure of FIG. The image processing apparatus 1 may collect machine learning after a certain amount of data has accumulated, or may perform additional learning each time data is acquired.
As described above, according to the seventh embodiment, the image processing apparatus 1 can perform machine learning by using the general use of the existing OCR system 9 by an operator. In particular, the reading target feature value generation unit 104 can update the first feature value.
Similarly, in the second to sixth embodiments, the image processing apparatus may perform machine learning using an existing OCR system.

<Eighth embodiment>
The learning function of the image processing apparatus may be made into a server to accelerate learning. In the eighth embodiment, this point will be described.
FIG. 14 is a diagram showing an overview of an image processing system 200 according to the eighth embodiment. In the configuration of FIG. 14, the image reading apparatus 2 is the same as that of FIG.
The function of the image processing apparatus 1 in FIG. 1 is divided into a learning function unit 1b that performs machine learning and an OCR function unit 1a that executes other functions, and has a server client configuration. In particular, the learning function unit 1b generates and updates the first feature amount. The learning function unit 1b also executes the function of the recording device 3 and manages the learning result database 4b. The OCR function unit 1a performs functions such as OCR processing, extraction of third feature values, identification of a recorded character string, and a user interface for an operator. The OCR function unit 1a detects a character string, detects a character feature included in the character string, and detects coordinates of the character string range in the OCR process.
The OCR function unit 1 a is stored in the terminal device 6. The learning function unit 1 b is stored in the server device 7.
The database 4 in FIG. 1 is divided into an OCR result database 4a and a learning result database 4b. The OCR result database 4a stores the result of OCR processing such as a recording table. The learning result database 4b stores data obtained during machine learning such as the first feature amount. The learning result database 4b is stored in the server device 7. On the other hand, the OCR result database is configured as one apparatus.

FIG. 15 is a diagram illustrating a processing flow of the terminal device according to the eighth embodiment.
In the processing of FIG. 15, the terminal device 6 reads the image data of the document form by the image processing device 2, and downloads the learning result corresponding to the image data from the server device 7 (step S1501). In particular, the terminal device 6 acquires the first feature amount as a learning result.
The terminal device 6 performs OCR processing on all character strings included in the image data, and displays a screen that applies to the learning result (step S1502). Specifically, the terminal device 6 specifies a recorded character string that applies to the first feature value, using the first feature value that is a learning result.

The terminal device 6 determines whether all the OCR target character strings in the image data are correctly read (step S1503). If it is determined that there is a character string that has not been read (step S1503: NO), the terminal device 6 presents the OCR result to the operator in a screen display and receives a correction operation (step S1504).
The terminal device 6 transmits and records the read character string (or the corrected character string when it is corrected by the operator) to the OCR result database 4a (step S1505). Specifically, the terminal device 6 stores the recorded character string in the OCR result database 4a. Also in the case of YES in step S1503, the process proceeds to step S1505.
The terminal device 6 transmits learning data such as the read character string and position information to the server device 7 (step S1506).
With the processing in FIG. 15, the terminal device 6 records a recording character string to be recorded as an OCR result in the OCR result database, and provides learning data to the server device 7.

FIG. 16 is a diagram illustrating a processing flow of the server device according to the eighth embodiment.
In the process of FIG. 16, the server device 7 receives and stores learning data from the terminal device 6 (step S1601).
The server device 7 determines whether N pieces of unlearned learning data (N is a positive integer) have been accumulated (step S1602). If it is determined that it has not been accumulated (step S1602: NO), the process returns to step S1601.
On the other hand, when it is determined that N pieces of unlearned learning data have been accumulated (step S1602: YES), the server device 7 reads the reception result data of the form received as the learning data from the learning result database (step S1603). .
Next, the server device 7 adds new learning data to the read learning result data, and re-learns (step S1604).
The server device 7 registers the relearning result in the learning result database 4b (step S1605).
The server device 7 stands by until learning data is received from the terminal device 6 (step S1606). After step S1606, the process returns to step S1601.
In the process of FIG. 16, the server device 7 updates the first feature amount as a learning result. The updated first feature amount is registered in the learning result database 4 b and is used for the terminal device 6.
According to the eighth embodiment, the learning function of the image processing apparatus can be made into a server to collect learning data and accelerate learning.

  Note that the timing at which the server device 7 performs machine learning such as relearning is not limited to the timing at which a certain number of learning data is accumulated, which is shown in step S1602 of FIG. For example, the server device 7 may learn at regular intervals. In particular, the server device 7 may start learning according to a temporal condition in addition to or instead of the data number condition.

  As an example in which the server device 7 learns at regular intervals, the server device 7 may learn at night or on closed days when the terminal device 6 is not operating. Here, when the server device 7 performs learning during operation of the terminal device 6, access to the learning result database by the learning function unit 1 b of the server device 7 occurs. As a result, there is a possibility that a delay occurs in access from the terminal device 6 to the learning result database 4b, or inconvenience that access cannot be made at a timing when access is required. Therefore, the server device 7 performs learning when the terminal device 6 is not operating, thereby avoiding the influence on the access from the terminal device 6 to the learning result database 4b.

The terminal device 6 may perform machine learning in addition to or instead of the server device 7.
Here, when the server device 7 performs machine learning and the terminal device 6 does not perform learning, the server device 7 performs machine learning to update the learning result (first feature amount), and the terminal device 6 and the learning result are updated. Until sharing, the processing accuracy of the terminal device 6 is not improved. For example, when the server device 7 performs machine learning only at night every day, the terminal device 6 can use the updated learning result after the next morning. For example, even when the terminal device 6 generates learning data in the morning and transmits it to the server device 7, the learning result is reflected in the processing of the terminal device 6 on the morning of the next day after about one day.

Therefore, each time the terminal device 6 processes one form, it performs machine learning. The terminal device 6 reflects the learning result in its own process, and transmits the learning result or learning data to the server device 7. The terminal device 6 generates a feature amount (third feature amount) by processing the form. And the terminal device 6 updates the 1st feature-value acquired from the learning result database using the produced | generated feature-value. For example, the terminal device 6 transmits the learning data to the server device 7 in step S1506 in FIG. 15, performs machine learning using the learning data, and stores the first data stored in the terminal device 6 itself. Update feature values. Alternatively, the terminal device 6 may transmit the learning result (the obtained feature amount) to the server device 7 in addition to the learning data in step S1506 of FIG.
The terminal device 6 can reflect its learning result in its own processing by performing its own learning. In this regard, it is expected that the terminal device 6 itself does not perform machine learning and the processing system of the terminal device 6 is improved earlier than when waiting for an update of a learning result by the server device 7.
Machine learning performed by each terminal device 6 is referred to as provisional learning.

When the terminal device 6 transmits the learning data to the server device 7, the processing performed by the server device 7 is the same as described with reference to FIG.
On the other hand, when the terminal device 6 transmits the learning result to the server device 7, the server device 7 stores the learning result from each terminal device 6. Then, the server device 7 reflects the learning result obtained from the terminal device 6 at night, for example, in the learning result stored in the learning result database 4b. Specifically, the learning function unit 1b of the server device 7 updates the first feature value stored in the learning database 4b using the feature value acquired by the terminal device 6 through learning.
The process of updating the first feature value stored in the learning result database is referred to as main learning.

FIG. 17 is a diagram illustrating a minimum configuration of the image processing apparatus.
As shown in this figure, the image processing apparatus 1 may include at least a feature amount extraction unit 103 and a read target feature amount generation unit 104.
The feature amount extraction unit 103 extracts, for each form image data, a feature amount indicating the feature of the character string included in the form image data based on the result of character recognition processing of a plurality of form image data registered in the past.
The reading target feature value generation unit 104 generates the first feature value of the recorded character string in the form image data using the feature value corresponding to the form image data.

  Each of the above devices has a computer system inside. A program for causing each device to perform the above-described processes is stored in a computer-readable recording medium of each device, and the above-described processing is performed by the computer of each device reading and executing the program. Done. Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Alternatively, the computer program may be distributed to the computer via a communication line, and the computer that has received the distribution may execute the program.

  Further, the program may be for realizing a part of the functions of each processing unit described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus 1a ... OCR function part 1b ... Learning function part 2 ... Image reading apparatus 3 ... Recording apparatus 4 ... Database 4a ... Learning result database 4b ... OCR result database 5 ... document form 6 ... terminal device 7 ... server device 101 ... control unit 102 ... acquisition unit 103 ... feature quantity extraction unit 104 ... read target feature quantity generation Unit 105 ... Recording unit 106 ... Group classification unit 107 ... Group identification unit 108 ... Work target determination unit 109 ... Work data generation unit

Claims (12)

Based on the result of character recognition processing of a plurality of form image data, a feature amount extraction unit that extracts a feature amount indicating a feature of a character string included in the form image data for each form image data;
A read target feature value generation unit that generates a first feature value that is a feature value of a recorded character string using the feature value corresponding to the form image data;
An image processing apparatus comprising: An acquisition unit that acquires the plurality of form image data and the recording character string that is a recording target among character strings included in the form image data;
The feature amount extraction unit extracts the feature amount corresponding to the form image data based on a result of character recognition processing of the form image data acquired by the acquisition unit,
The image processing apparatus according to claim 1, wherein the reading target feature amount generation unit generates a first feature amount of the recorded character string using the feature amount corresponding to the form image data. An acquisition unit that acquires the plurality of form image data and the recording character string that is a recording target among character strings included in the form image data;
The feature amount extraction unit includes:
Based on the result of image recognition processing of the form image data acquired by the acquisition unit, an individual first feature amount indicating the feature of the recorded character string and an individual second feature amount indicating recognition information other than the recorded character string Are extracted for each form image data and for each character string,
Further, a group classification unit that groups the form image data based on the individual second feature amount,
The reading target feature value generation unit generates a group first feature value, which is the first feature value for each group, using the individual first feature value corresponding to the form image data included in the group. Item 8. The image processing apparatus according to Item 1. An acquisition unit for acquiring the plurality of form image data and the recording character string to be recorded among character strings included in the form image data;
A group classification unit that groups the form image data based on the individual first feature values extracted by the feature amount extraction unit for each form image data and for each recording character string;
The reading target feature value generation unit generates a group first feature value, which is the first feature value for each group, using the individual first feature value corresponding to the form image data included in the group. Item 8. The image processing apparatus according to Item 1. An acquisition unit that acquires the plurality of form image data and the recording character string that is a recording target among character strings included in the form image data;
The feature amount extraction unit obtains an individual first feature amount indicating a feature of the recorded character string for each form image data and the recorded character based on a result of character recognition processing on the form image data acquired by the acquisition unit. Extract by column,
The reading target feature quantity generation unit uses the individual first feature quantity corresponding to the form image data included in a predetermined group set in advance, and the group first feature that is the first feature quantity for each group. The image processing apparatus according to claim 1, wherein the quantity is generated. The acquisition unit acquires new form image data,
A recording unit that records the recorded character string of character strings read from the new form image data using the first feature amount;
The image processing apparatus according to claim 2, further comprising: The acquisition unit acquires new form image data,
A group specifying unit for specifying the group to which the new form image data belongs based on a group second feature value set for each group as a feature value of a character string other than the recorded character string;
A recording unit that records the recorded character string of character strings read from the new form image data, using the first group feature amount for the identified group;
The image processing apparatus according to claim 3 or 5. The reading target feature amount generation unit is configured so that the recording unit succeeds in selecting any one of the character strings read from the new form image data as the recording character string, and fails. Updating the first feature amount;
The image processing apparatus according to claim 6 or 7. The reading target feature amount generation unit updates the first feature amount by using a recording target character string confirmed and confirmed by an operator with respect to the recording target character string by the recording unit.
The image processing apparatus according to claim 1. A server device having a function of a read target feature value generation unit according to any one of claims 1 to 9,
A terminal device for obtaining the recorded character string from the character string of the character recognition processing result using the first feature amount by the server device;
An image processing system comprising: Based on the result of recognition processing of a plurality of form image data, a step of extracting a feature amount indicating a character string feature included in the form image data for each form image data;
Generating a first feature amount that is a feature amount of a recorded character string using the feature amount corresponding to the form image data;
An image processing method including: On the computer,
Based on the result of recognition processing of a plurality of form image data, a step of extracting a feature amount indicating a character string feature included in the form image data for each form image data;
Generating a first feature amount that is a feature amount of a recorded character string using the feature amount corresponding to the form image data;
A program for running

Images