JP2021125040A - Document sorting system, document sorting method and program - Google Patents

Abstract

To provide a document sorting system, a document sorting method and a program capable of sorting subspecies so as to enable OCR character recognition, without correcting a definition body used in the OCR character recognition.SOLUTION: A document sorting system comprises: an acquisition unit that acquires a target document that is a document to be sorted; a type classification unit that classifies the type of the target document using a learnt model; a feature extraction unit that extracts the features of the target document; and a subspecies classification unit that classifies the target document by performing machine learning using the degree of similarity with the features of the document corresponding to the pre-registered definition body for character recognition, based on the classification result by the type classification unit and the extraction result by the feature extraction unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to a form sorting system, a form sorting method, and a program.

Conventionally, various forms have been used in various industries. For example, text information is obtained by reading a paper form with a reading device such as a scanner and performing character recognition from the image of the reading result by OCR (Optical Character Reader) technology. As a result, paperwork such as data entry can be made more efficient.

Generally, forms are classified into standard forms, semi-standard forms, and non-standard forms. A standard form is a form in which items, the position and size of an entry frame are defined, and the format is fixed to one. A semi-standard form is a form in which items are defined, but the position and size of the entry frame are not defined, and there are multiple different formats. An atypical form is a form in which there is no format in which items, the position and size of the entry frame are fixed.

In other words, there are a plurality of forms (hereinafter, also referred to as subspecies) having slightly different formats even if they are of the same type, such as semi-standard forms. When various variants are mixed in this way, a form in which one form can be recognized by OCR, but another form in a slightly different form cannot be recognized by OCR. It was a hindrance to the use of OCR.

As a countermeasure, in Patent Document 1, the form layout of the definition program defined for OCR character recognition is corrected according to the ruled line layout extracted from the target form image, so that the forms are similar in style. On the other hand, although a technique for performing character recognition with one definition program is disclosed, it is possible.

Japanese Patent No. 6342292

However, if the definition program of OCR character recognition is corrected every time the OCR is to read the variant form, it takes time and effort to correct it. In particular, when trying to recognize a large number of forms in which variants are mixed, there is a problem that it is inefficient and impractical to apply the technique of Patent Document 1.

The present invention has been made to solve the above problems, and an object of the present invention is a form sorting system capable of sorting variants so that OCR character recognition can be performed without correcting the definition program used for OCR character recognition. The purpose is to provide a form sorting method and a program.

In order to solve the above problem, one aspect of the present invention includes an acquisition unit that acquires a target form, which is a form to be sorted, and a type classification unit that classifies the types of the target form using a learned model. Similarity between the feature extraction unit that extracts the features of the target form and the features of the form corresponding to the pre-registered character recognition definition body based on the classification result by the type classification unit and the extraction result by the feature extraction unit. The trained model is provided with a subspecies classification unit that classifies the target form by performing machine learning using the degree, and the output obtained by inputting the learning form corresponds to the learning form. It is a form sorting system characterized in that it is a model learned to approach the type to be performed and is a model for predicting the type of the input form.

Further, one aspect of the present invention is defined in advance based on an acquisition unit that acquires a target form, which is a form to be sorted, a feature extraction unit that extracts features of the target form, and an extraction result by the feature extraction unit. It is a form sorting system including a subtype classification unit that classifies the target form by performing machine learning using the degree of similarity with the feature of the form corresponding to the definition structure of the character recognition.

Further, one aspect of the present invention is that in the form sorting system described above, the subspecies classification unit has the characteristics of the ruled lines extracted by the feature extraction unit and the characteristics of the ruled lines in the form corresponding to the definition program. The target forms may be classified by performing a cluster analysis using the degree of similarity of.

Further, in one aspect of the present invention, in the form sorting system described above, the form corresponding to the definition program is a form selected from the forms in the cluster obtained by performing the cluster analysis on the registration form. It may be.

Further, in one aspect of the present invention, in the form sorting system described above, the target form classified into the same group as the form corresponding to the definition program based on the classification result by the subspecies classification unit is defined as described above. A conformity determination unit for determining whether or not the character recognition using the body is suitable may be further provided.

Further, one aspect of the present invention is that in the form sorting system described above, the conformity determination unit has features of ruled lines in the target form classified into the same group as the form corresponding to the definition program, and the definition program. Based on the degree of similarity with the characteristics of the ruled lines in the form corresponding to the above, it may be determined whether or not the character recognition using the definition program is suitable.

Further, in one aspect of the present invention, the acquisition unit acquires the target form, which is the form to be sorted, the type classification unit classifies the type of the target form using the learned model, and the feature extraction unit , The feature of the ruled line in the target form is extracted, and the subtype classification unit corresponds to the definition structure of the character recognition defined in advance based on the classification result by the type classification unit and the extraction result by the feature extraction unit. It is a form sorting method that classifies the target form by performing machine learning using the degree of similarity with, and in the trained model, the output obtained by inputting the learning form is the learning form. It is a form sorting method characterized in that it is a model learned so as to approach the type corresponding to, and is a model for predicting the type of the input form.

Further, one aspect of the present invention is an acquisition means for acquiring a target form which is a form to be sorted, a type classification means for classifying the type of the target form using a learned model, and a ruled line in the target form. Based on the feature extraction means for extracting the features of, the classification result by the type classification means, and the extraction result by the feature extraction means, machine learning is performed using the degree of similarity with the form corresponding to the predefined character recognition definition program. It is a program for functioning as a subspecies classification means for classifying the target form by performing the above, and in the trained model, the output obtained by inputting the learning form corresponds to the learning form. It is a program that is a model that is learned so as to approach the type to be performed and that predicts the type of the input form.

According to the present invention, variants can be sorted so that OCR reading can be performed without correcting the definition program for OCR reading.

It is a block diagram which shows the structural example of the form recognition system 1 of embodiment. It is a block diagram which shows the structural example of the type classification apparatus 10 of an embodiment. It is a block diagram which shows the structural example of the subspecies classification apparatus 20 of an embodiment. It is a flow chart which shows the flow of the process performed by the type classification apparatus 10 of an embodiment. It is a sequence diagram which shows the flow of the process performed by the form recognition system 1 of embodiment. It is a flow chart which shows the flow of the process performed by the subspecies classification apparatus 20 of an embodiment. It is a figure explaining the process performed by the subspecies classification apparatus 20 of an embodiment. It is a sequence diagram which shows the flow of the process performed by the form recognition system 1 of embodiment. It is a flow chart which shows the flow of the process performed by the type classification apparatus 10 of an embodiment. It is a flow chart which shows the flow of the process performed by the subspecies classification apparatus 20 of an embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing a configuration example of the form recognition system 1 of the embodiment. The form recognition system 1 includes, for example, a form sorting system 100 and a character recognition device 30.

The form sorting system 100 is a system that sorts forms of various formats into groups that can be read by the same OCR definition program. Here, the OCR definition program is information about a form that is the target of OCR character recognition, and is information used for OCR character recognition. The OCR definition program includes, for example, ruled line layout information indicating the number, length, and arrangement of ruled lines, and form-specific information indicating the form title, item name, and the like.

In the present embodiment, the form to be sorted (hereinafter, also referred to as the target form) is information necessary for business, transaction, application, etc., such as withholding slip, salary statement, various books and slips, application form, etc. It is a document used for entry and printing, and is a document laid out so that an item column and an entry frame are formed by ruled lines and the like, and a defined description is made at a predetermined position.

The form sorting system 100 includes, for example, a type classification device 10 and a plurality of subspecies classification devices 20 (subspecies classification devices 20-1, 20-2, ..., 20-N). N is a natural number determined according to the number of types classified by the type classification device 10.

The type classification device 10 is a computer that classifies target forms for each type by using a machine learning method. The machine learning method used by the classification device 10 may be any existing learning method, but for example, it is supervised learning and uses a deep learning model by CNN (Convolutional Neural Network) or the like. This is the method that was used. When classification is performed using the trained model, how the classification device 10 classifies the forms is determined by what kind of data is machine-learned by the trained model. The trained model will be described in detail later.

The type classification device 10 classifies the target forms according to those having different appearances at first glance. For example, the type classification device 10 classifies the target form according to the title of the form. Alternatively, the type classification device 10 classifies the target form according to the form of the form. In other words, the type classification device 10 classifies target forms whose appearance does not change at first glance into the same type. That is, the type classification device 10 does not distinguish between subspecies and classifies them as the same type. Specifically, when the type classification device 10 classifies the target form for each form title, there are a plurality of target forms (subspecies) having the same form title but slightly different positions and sizes of the entry frames. If so, classify these variants as the same type of form. The classification device 10 outputs the classification result to the subspecies classification device 20.

The subspecies classification device 20 is a computer that classifies a group of forms classified into the same type by the type classification device 10 for each subspecies. The subspecies classification device 20 acquires the classification result from the type classification device 10. The subspecies classification device 20 extracts each feature amount of the form group classified into the same type based on the acquired information. The feature amount here is a degree indicating the characteristics of the form necessary for classifying the subspecies, and is, for example, the mode of the ruled line used in the form (for example, the interval of the ruled line).

The subspecies classification device 20 classifies subspecies by performing machine learning using the extracted features. The machine learning method used by the subspecies classification device 20 may be any existing method, but for example, it is unsupervised learning and is a method using cluster analysis.

The subspecies classification device 20 classifies subspecies within a range in which character recognition using the same OCR definition program is possible. As a result, it is possible to make it difficult for one variant to recognize characters by OCR, but another variant to be unable to recognize characters by OCR. Therefore, it is possible to promote the use of OCR for forms.

The subspecies classification device 20 classifies subspecies based on the degree of similarity between the feature amount of the form (hereinafter, also referred to as the representative form) corresponding to the pre-registered OCR definition program and the feature amount of the target form. That is, the subspecies classification device 20 classifies subspecies having characteristics similar to the representative form into the same group. The representative form is a form that has been registered in advance and an OCR definition structure has already been generated so that OCR characters can be recognized. As a result, it is highly possible that the subspecies classified as having characteristics similar to the representative form can perform OCR character recognition using the OCR definition program corresponding to the representative form. The method by which the subspecies classification device 20 classifies subspecies will be described in detail later. The subspecies classification device 20 outputs the classification result of classifying the subspecies to the character recognition device 30.

The character recognition device 30 is a computer that performs OCR character recognition. An OCR definition program corresponding to each of the plurality of representative forms is registered in the character recognition device 30. An OCR definition program is generated by an operator or the like based on the representative form designated by the subspecies classification device 20, and the generated OCR definition program is registered (stored) in the character recognition device 30. When the character recognition device 30 has a function of generating an OCR definition structure based on the form, the character recognition device 30 is configured to generate an OCR definition structure corresponding to the representative form designated by the subspecies classification device 20. May be good. The method by which the subspecies classification device 20 designates the representative form will be described in detail later.

The character recognition device 30 acquires the classification result from the subspecies classification device 20. The character recognition device 30 performs OCR character recognition on a subspecies classified as having characteristics similar to the representative form using the OCR definition program corresponding to the representative form.

In the example of FIG. 1, a configuration is shown in which the target form is classified into any of N types (type 1, type 2, ... Type N) by the type classification device 10. Further, a configuration is shown in which the form group classified into each type is classified into a plurality of subspecies by each of the subspecies classification devices 20. For example, a configuration is shown in which the subspecies classification device 20-1 classifies a plurality of subspecies (type 1 subspecies A, type 1 subspecies B, ...). The subspecies classification apparatus 20-2 shows a configuration in which the subspecies are classified into a plurality of subspecies (type 2 subspecies A, type 2 subspecies B, ...). A configuration is shown in which the subspecies classification device 20-N classifies the subspecies into a plurality of subspecies (type N subspecies A, type N subspecies B, ...).

FIG. 2 is a block diagram showing a configuration example of the type classification device 10 of the embodiment. The type classification device 10 includes, for example, a target image acquisition unit 11, a learning image acquisition unit 12, a preprocessing unit 13, a learning unit 14, a prediction unit 15, a type classification unit 16, and an output unit 17. A trained model parameter storage unit 18 is provided.

The process performed by the classification device 10 has two stages, "preparation" and "classification execution". In the "preparation" stage, the type classification device 10 prepares a trained model to be used for actual classification before classifying the target forms by type. At the stage of "classification execution", the actual classification for classifying the target forms by type is performed by 10. Hereinafter, the two stages of "preparation" and "classification execution" will be described in order.

(Advance preparation)
The classification device 10 generates a trained model as a preliminary preparation. The trained model is a model trained so that the type of the input unlearned form can be predicted by learning the correspondence between the learning form (hereinafter, also referred to as the learning form) and the type. Is. That is, the trained model is a model that is trained so that the output obtained by inputting the learning form approaches the type corresponding to the learning form, and is a model that predicts the type of the input form. Is.

The learning image acquisition unit 12 acquires the image data of the learning form. The learning form is, for example, a form that has already been published as a standard format, or an actual form that has been used in the past. The image data is, for example, electronic information of an image obtained by reading a paper form with a reading device such as a scanner. The learning image acquisition unit 12 acquires, for example, image information read by a scanner connected to the classification device 10. Alternatively, the learning image acquisition unit 12 may acquire the image data of the learning form stored in the external DB (database) server device via a network or the like. The learning image acquisition unit 12 outputs the acquired image data of the learning form to the preprocessing unit 13.

The preprocessing unit 13 generates a data set to be trained by the learning model. The learning model is a model before the trained model is trained, and is, for example, a deep (multilayer) model by CNN or the like. The preprocessing unit 13 generates a data set in which the learning data and the teacher data are associated with each other as the data to be trained by the learning model. The learning data is data to be input to the learning model, and is image data of the learning form acquired by the learning image acquisition unit 12. The teacher data is data for calculating the error of the predicted value output from the learning model, and is information indicating the type of the learning form. The preprocessing unit 13 generates a data set to be trained by the learning model by associating the learning form as the learning data with the type of the learning form as the teacher data. The preprocessing unit 13 outputs the generated data set to the learning unit 14.

The learning unit 14 trains the learning model using the learning data set generated by the preprocessing unit 13. The learning unit 14 causes the learning model to input the learning data of the data set. The learning unit 14 uses a method such as an error back propagation method to set the parameters of the learning model so that the data (predicted value) output from the learning model approaches the teacher data (type) corresponding to the learning data. adjust. The learning unit 14 ends the learning of the learning model when it is determined that the predetermined end condition is satisfied, such as the error of the predicted value output from the output layer of the learning model is equal to or less than a predetermined threshold value. The learning unit 14 determines the learning model at the end of learning as a learned model. The learning unit 14 stores the parameters set in the learning model when the learning is completed in the learned model parameter storage unit 18. The parameters here are variables used to generate a trained model, and for example, if a trained model is generated using a CNN training model, the CNN input layer, intermediate layer, and so on. Information indicating the number of units in each layer of the output layer, the number of layers in the hidden layer, the activation function, etc., and information indicating the coupling coefficient and weight that connect the nodes of each layer.

The trained model parameter storage unit 18 stores the parameters of the trained model generated by the learning unit 14.

(Classification execution)
The type classification device 10 classifies the type of the target form at the stage of classifying execution.

The target image acquisition unit 11 acquires the image data of the target form. The method in which the target image acquisition unit 11 acquires the image data of the target form is the same as the method in which the learning image acquisition unit 12 acquires the image data of the registration form, and thus the description thereof will be omitted. The target image acquisition unit 11 outputs the acquired image data of the target image to the prediction unit 15.

The prediction unit 15 predicts the type of the target image. The prediction unit 15 acquires image data of the target image from the target image acquisition unit 11. The prediction unit 15 acquires (reconstructs) the trained model generated by the learning unit 14 by referring to the trained model parameter storage unit 18. The prediction unit 15 uses the output obtained by inputting the target image into the trained model as the prediction result. The prediction unit 15 outputs the prediction result to the classification unit 16.

Here, the trained model outputs the prediction result together with its certainty. The degree of certainty here is the certainty of the predicted type, and is information indicating, for example, the probability that the trained model is the predicted type. For example, by using the Softmax function as the activation function of the model, it is possible to output the establishment (certainty degree) of the prediction result from the trained model. For example, the trained model outputs a prediction result that the probability that the target form is type 1 (for example, a final tax return) is 90%. For example, in the trained model, the probability that the target form is type 1 (for example, final tax return) is 55%, and the probability that the target form is type 2 (for example, examination request) is 40%. Is output. It should be noted that the degree of certainty may be at least a degree indicating the certainty of the predicted type, and is not limited to the probability. For example, the certainty may be binary information indicating whether it is "high" or "low" (certainty), "high", "slightly high", "slightly low", "low", etc. , Information indicating a plurality of levels may be used.

The type classification unit 16 determines the type of the target form based on the prediction result from the prediction unit 15. The type classification unit 16 determines, for example, the type having the highest degree of certainty among those having a certainty degree equal to or higher than a predetermined threshold value as the type of the target form. When the certainty level is less than a predetermined threshold value, the type classification unit 16 determines that the type of the target form is unknown. The type classification unit 16 outputs the determination result of determining the type of the target form via the output unit 17.

Here, the type classification unit 16 may change the output destination according to the determined type. For example, when the determined type is the type to be converted into text, the type classification unit 16 outputs the determination result of determining the type of the target form to the subspecies classification device 20. On the other hand, when the determined type is a type that is not the target of text conversion, the type classification unit 16 outputs the determination result to another device. Another device is, for example, a database in which image data of forms that are not targeted for text conversion are aggregated.

Further, when the certainty level is less than a predetermined threshold value, the classification unit 16 may display a warning to that effect on, for example, a display unit (not shown) so that the operator can perceive it. As a result, the type classification unit 16 can notify the worker that there is a target form whose type is unknown. In response to the warning, the worker can visually check the target form of unknown type and take individual measures, or take measures such as re-learning the trained model. When retraining the trained model, the training form includes a target form of unknown type and a data set in which the type is associated with the target form. As a result, it is possible to accurately predict the type of the form predicted to be unknown in the model before relearning by the trained model after relearning.

FIG. 3 is a block diagram showing a configuration example of the subspecies classification device 20 of the embodiment. The subspecies classification device 20 includes, for example, a target image acquisition unit 21, a definition program registration image acquisition unit 22, a ruled line extraction unit 23, a similarity calculation unit 24, a subspecies clustering unit 25, and a conformity determination unit 26. , A subspecies classification unit 27, and an output unit 28.

The process performed by the subspecies classification device 20 has two stages, "preparation" and "classification execution". At the stage of "preparation", before classifying the target forms classified into the same type by subspecies, the representative form that is the standard of classification and the OCR definition structure corresponding to the representative form are prepared. NS. At the stage of "classification execution", the subspecies classification device 20 performs the actual classification in which the target form groups classified into the same type are classified by subspecies. Hereinafter, the two stages of "preparation" and "classification execution" will be described in order.

(Advance preparation)
The subspecies classification device 20 selects a representative form as a preliminary preparation. The representative form is a form in which an OCR definition program used when performing OCR character recognition is generated. By classifying variants having characteristics similar to the representative form into the same group based on the representative form, OCR character recognition can be performed using the OCR definition program corresponding to the representative form.

The definition program registration image acquisition unit 22 acquires image data of a form for registering the OCR definition program (hereinafter, also referred to as a registration form). The registration form is, for example, a form that has already been published as a standard format, or an actual form that has been used in the past. The definition program registration image acquisition unit 22 acquires, for example, image information read by a scanner connected to the subspecies classification device 20. Alternatively, the definition program registration image acquisition unit 22 may acquire the image data of the registration form stored in the external DB (database) server device via a network or the like. The definition program registration image acquisition unit 22 outputs the acquired image data of the registration form to the ruled line extraction unit 23.

The ruled line extraction unit 23 extracts a ruled line from the registration form. The ruled line extraction unit 23 extracts a ruled line by using an existing technique. For example, the ruled line extraction unit 23 extracts the ruled line by Hough transforming the image data of the registration form. Alternatively, the ruled line extraction unit 23 may extract the ruled lines in the registration form by applying a Laplacian filter or a Sobel filter to the registration form. The ruled line extraction unit 23 outputs the information indicating the ruled lines extracted from the registration form to the similarity calculation unit 24 in association with the registration form.

The similarity calculation unit 24 calculates the similarity between the forms based on the characteristics of the ruled lines in the registration form. The characteristic of the ruled line is a characteristic ruled line mode that is used for recognition, especially when OCR character recognition is performed. For example, information indicating the length, number of lines, position and size of rectangles, number of rectangles, and the like. be. The similarity calculation unit 24, for example, digitizes (vector representation) the features of these ruled lines and arranges them on a high-dimensional vector space. The similarity calculation unit 24 calculates the amount of correlation between forms in the vector space to which the feature amount of the ruled line is mapped by cosine, inner product, distance, and the like. The similarity calculation unit 24 uses the calculated correlation amount as the similarity between the forms. The similarity calculation unit 24 outputs the calculated similarity to the subspecies clustering unit 25.

The subspecies clustering unit 25 performs cluster analysis based on the similarity between the forms calculated by the similarity calculation unit 24. Cluster analysis is a method of classifying a group of different properties into several sets with similar properties.

The subspecies clustering unit 25 performs, for example, a hierarchical cluster analysis. That is, the subspecies clustering unit 25 does not set the number of groups to be classified (the number of clusters) in advance when performing cluster analysis. In a situation where variants occur almost every year, it is unknown how many OCR definition programs should be defined for the form group to be converted into text and how many variants should be classified to enable OCR recognition. Because there is. The subspecies clustering unit 25 outputs the result of the cluster analysis to the conformity determination unit 26 and the subspecies classification unit 27.

The conformity determination unit 26 detects anomalies in individual populations (form groups classified as the same subspecies) that have been cluster-analyzed. The abnormality detection here is to detect whether or not there is an extremely low degree of similarity among the classified form groups. Among the forms classified in the same group, those with high similarity and similar characteristics are likely to be OCR-recognizable using the same OCR definition program, but those with extremely low similarity are likely to be OCR-recognizable. This is because it is considered to have low sex. The conformity determination unit 26 determines whether the forms classified as the same subspecies can recognize OCR using the same OCR definition program, that is, whether or not they conform to the same OCR definition program.

For example, when an abnormality, that is, a form having an extremely low degree of similarity is detected, the conformity determination unit 26 may display a warning to that effect on the display unit. As a result, the type classification unit 16 can inform the worker that there is a target form having an extremely low similarity among the forms classified as the same subspecies. In response to the warning, the worker can take measures such as visually checking the target form and taking individual measures. The conformity determination unit 26 outputs the determination result to the subspecies classification unit 27.

The subspecies classification unit 27 classifies the registration form for each subspecies using the result of the cluster analysis by the subspecies clustering unit 25 and the result of the abnormality detection by the conformity determination unit 26. The subspecies classification unit 27 sets a group in which abnormally detected forms are removed from each of the subspecies groups classified by cluster analysis as a form group classified into the same subspecies. The subspecies classification unit 27 may arbitrarily determine which layer of the classification result to use from the result of the hierarchical cluster analysis by the subspecies clustering unit 25. The subspecies classification unit 27 determines, for example, a range that can be read by the OCR definition program according to the distribution of the forms classified as the same subspecies, the accuracy of OCR recognition, and the like. The subspecies classification unit 27 outputs the classification result classified for each subspecies to the character recognition device 30 via the output unit 28.

(Classification execution)
The subspecies classification device 20 classifies the target form for each subspecies at the stage of executing the classification.

The target image acquisition unit 21 acquires the image data of the target form. The method of acquiring the image data of the target form by the target image acquisition unit 21 is the same as the method of acquiring the image data of the registration form by the definition program registration image acquisition unit 22, and thus the description thereof will be omitted. The target image acquisition unit 21 outputs the acquired image data of the target image to the ruled line extraction unit 23.

The ruled line extraction unit 23 extracts a ruled line from the target form. Since the method of extracting the ruled lines from the target form by the ruled line extraction unit 23 is the same as the method of extracting the ruled lines from the registration form, the description thereof will be omitted. The ruled line extraction unit 23 outputs the information indicating the ruled lines extracted from the target form to the similarity calculation unit 24 in association with the registration form.

The similarity calculation unit 24 calculates the similarity with the representative form based on the characteristics of the ruled lines in the target form. Since the method for calculating the similarity by the similarity calculation unit 24 has already been described, the description thereof will be omitted. The similarity calculation unit 24 outputs the calculated similarity to the subspecies clustering unit 25.

The subspecies clustering unit 25 performs cluster analysis based on the similarity between the target forms calculated by the similarity calculation unit 24. Since the method by which the subspecies clustering unit 25 performs the cluster analysis has already been described, the description thereof will be omitted. The subspecies clustering unit 25 outputs the result of the cluster analysis to the conformity determination unit 26 and the subspecies classification unit 27.

The conformity determination unit 26 detects anomalies in individual populations (form groups classified as the same subspecies) that have been cluster-analyzed. Since the method for detecting the abnormality by the conformity determination unit 26 has already been described, the description thereof will be omitted. The conformity determination unit 26 outputs the result of abnormality detection to the subspecies classification unit 27.

The subspecies classification unit 27 classifies the target form for each subspecies using the result of the cluster analysis by the subspecies clustering unit 25 and the result of the abnormality detection by the conformity determination unit 26. Since the method of classifying each subspecies by the subspecies classification unit 27 has already been described, the description thereof will be omitted. The subspecies classification unit 27 outputs the classification result of classifying the target form for each subspecies to the character recognition device 30 via the output unit 28.

FIG. 4 is a flow chart showing a flow of processing performed by the type classification device 10 of the embodiment. FIG. 4 shows a flow of processing in which the classification device 10 generates a trained model at the stage of preparation.

The type classification device 10 acquires a learning form (learning form group) at the stage of advance preparation (step S11). The type classification device 10 uses the learning form to generate a learning data set in which the learning data (learning form) and the teacher data (type) are associated with each other (step S12). The type classification device 10 inputs learning data (learning form) into the learning model (step S13). The type classification device 10 updates the parameters of the learning model so that the error becomes smaller according to the error between the output obtained from the learning model (predicted value of the type) and the teacher data (correct answer of the type) (step). S14). The type classification device 10 determines whether or not the predetermined end condition is satisfied (step S15). The termination condition here is, for example, that the error is below a predetermined threshold value, or that the upper limit of the number of learnings has been reached. The type classification device 10 ends learning when the end condition is satisfied. If the end condition is not satisfied, the type classification device 10 returns to step S13 and repeats learning.

FIG. 5 is a sequence diagram showing a flow of processing performed by the form recognition system 1 of the embodiment. FIG. 5 shows the flow of processing in which the OCR definition program is registered in the preparatory stage.

The subspecies classification device 20 performs cluster analysis (described as subspecies clustering) of the registration form (form for defining definition program registration) (step S21). The subspecies classification device 20 selects a representative form from the forms in the group (cluster) obtained as a result of the cluster analysis (step S22). The subspecies classification device 20 generates, for example, a group obtained by removing the forms detected as abnormalities from the group obtained as a result of cluster analysis. The subspecies classification device 20 selects the form having the most common characteristics as the representative form from the group excluding the forms in which the abnormality is detected. The form having the most common features is, for example, a form arranged at the position closest to the representative value (for example, the median value) of the form group mapped in the vector space of the feature amount. The subspecies classification device 20 outputs the representative form in the group (cluster) to the character recognition device 30. The character recognition device 30 generates an OCR definition program corresponding to the representative form acquired from the subspecies classification device 20, stores the generated definition program, and registers the generated definition program (step S23).

FIG. 6 is a flow chart showing a flow of processing performed by the subspecies classification device 20 of the embodiment. FIG. 6 shows the details of the processing flow corresponding to steps S21 to S22 of FIG.

The subspecies classification device 20 acquires a plurality of registration forms (form registration form group) (step S211), and extracts ruled lines from the image data of each of the acquired registration forms (step S212). The subspecies classification device 20 calculates the similarity of the feature amount of the ruled line between each form of the registration form (step S213). The subspecies classification device 20 performs cluster analysis (subspecies clustering) based on the similarity (step S214).

The subspecies classification device 20 performs abnormality detection (conformity determination) for each of the forms classified in the same cluster (step S221). The subspecies classification device 20 selects a representative form in the cluster using the result of the conformity determination (step S222). For example, the subspecies classification device 20 removes a form having an extremely low similarity as a result of abnormality detection (conformity determination) from the cluster. The subspecies classification device 20 selects as the representative form the form having the most common characteristics in the group from the form group after excluding the forms having extremely low similarity.

FIG. 7 is a diagram illustrating a process performed by the subspecies classification device 20 of the embodiment. FIG. 7 schematically shows the result of the cluster analysis. The horizontal axis and the vertical axis of FIG. 7 indicate the feature amounts, respectively. FIG. 7 is a vector space of two-dimensional features. As shown in FIG. 7, if the form is mapped in the vector space according to its feature amount, it can be classified into several groups according to the distance. FIG. 7 shows an example of being classified into five groups of clusters K1 to K5 as a result of cluster analysis. For example, cluster K2 contains five forms corresponding to points P1 to P5. Of these, the three forms corresponding to points P2 to P4 are close to each other and have a high degree of similarity to each other. On the other hand, the point P1 is mapped to a distance slightly away from the point cloud of the points P2 to P4, and is not similar to the three forms corresponding to the points P2 to P4, that is, the similarity with the three forms is low. Conceivable. Similarly, it is considered that the point P5 has a low degree of similarity to the three forms corresponding to the points P2 to P4. In this case, the conformity determination unit 26 abnormally detects, for example, the forms corresponding to the points P1 and P5 as the forms having extremely low similarity from the forms classified into the cluster K2.

FIG. 8 is a sequence diagram showing a flow of processing performed by the form recognition system 1 of the embodiment. FIG. 8 shows the processing flow of the form recognition system 1 at the stage of execution of classification.

The type classification device 10 classifies the types of target forms using the trained model (step S30). The classification device 10 outputs the classification result to the subspecies classification device 20.

For example, the type classification device 10 outputs information indicating each of the target forms classified into the type 1 (form group of the type 1) to the subspecies classification device 20-1. The subspecies classification device 20-1 performs cluster analysis of the type 1 form group and classifies each subspecies (step S31). The subspecies classification device 20-1 outputs the classification result to the character recognition device 30. For example, the subspecies classification device 20-1 outputs information indicating each of the target forms classified into the subspecies A of the type 1 (form group of the subspecies A of the type 1) to the character recognition device 30. The subspecies classification device 20-1 outputs information indicating each of the target forms classified into the subspecies B of the type 1 (form group of the subspecies B of the type 1) to the character recognition device 30.

For example, the type classification device 10 outputs information indicating each of the target forms classified into the type 2 (form group of the type 1) to the subspecies classification device 20-2. The subspecies classification device 20-2 performs cluster analysis of the type 2 form group and classifies each subspecies (step S32). The subspecies classification device 20-2 outputs the classification result to the character recognition device 30. For example, the subspecies classification device 20-2 outputs information indicating each of the target forms classified into the subspecies A of the type 2 (form group of the subspecies A of the type 2) to the character recognition device 30. The subspecies classification device 20-2 outputs information indicating each of the target forms classified into the subspecies B of the type 2 (form group of the subspecies B of the type 2) to the character recognition device 30.

For example, the type classification device 10 outputs information indicating each of the target forms classified into the type N (form group of the type N) to the subspecies classification device 20-N. The subspecies classification device 20-N performs cluster analysis of the form group of type N and classifies each subspecies (step S33). The subspecies classification device 20-N outputs the classification result to the character recognition device 30. For example, the subspecies classification device 20-N outputs information indicating each of the target forms classified into the subspecies A of the type N (form group of the subspecies A of the type N) to the character recognition device 30. The subspecies classification device 20-N outputs information indicating each of the target forms classified into the subspecies B of the type N (form group of the subspecies B of the type N) to the character recognition device 30.

The character recognition device 30 uses the OCR definition program corresponding to the representative form of the subspecies for each subspecies acquired from the subspecies classification device 20 to recognize the form group classified into the subspecies as OCR characters ( Step S34).

FIG. 9 is a flow chart showing a flow of processing performed by the type classification device 10 of the embodiment. FIG. 9 shows the details of the processing flow corresponding to step S30 of FIG.

The type classification device 10 acquires the target form (step S301) and predicts (estimates) the type by inputting the image data of each of the acquired target forms into the trained model (step S302). The type classification device 10 determines whether or not the certainty of the type predicted by the trained model is equal to or higher than a predetermined threshold value (step S303). When the certainty level is equal to or higher than a predetermined threshold value, the type classification device 10 determines the predicted type as the type of the target form (step S304), while the type classification device 10 determines the certainty level to be less than the predetermined threshold value. If, the type of the target form is set to another type (type unknown) (step S305).

FIG. 10 is a flow chart showing a flow of processing performed by the type classification device 10 of the embodiment. FIG. 10 shows the details of the processing flow corresponding to step S31 (S32, S33) of FIG. Here, the details of the processing flow will be described by taking the processing of step S31 as an example. The same processing flow is also applied to steps S32 and S33.

The subspecies classification device 20-1 acquires a target form for each type (step S311), and extracts a ruled line from each image data of the acquired target form (step S312). The subspecies classification device 20-1 calculates the degree of similarity between the target form and the representative form (step S313). The subspecies classification device 20-1 performs cluster analysis (subspecies clustering) based on the similarity (step S314). The subspecies classification device 20-1 performs conformity determination (abnormality detection) (step S315), and determines a form that is not the target of abnormality detection as a form classified into the subspecies (step S316). On the other hand, the subspecies classification device 20 classifies the form in which the abnormality is detected as another subspecies (subspecies unknown) (step S317).

As described above, the form recognition system 1 of the embodiment includes the target image acquisition unit 11, the type classification unit 16, the ruled line extraction unit 23 (an example of the “feature extraction unit”), and the subspecies classification unit 27. Be prepared. The target image acquisition unit 11 acquires the target form. The type classification unit 16 classifies the type of the target form by using the trained model. The trained model is a model that is trained so that the output obtained by inputting the learning form approaches the type corresponding to the learning form, and is a model that predicts the type of the input form. .. The ruled line extraction unit 23 extracts the features of the target form. The subspecies classification unit 27 has the characteristics of the form corresponding to the OCR definition structure (pre-registered character recognition definition structure) of the representative form based on the classification result by the type classification unit 16 and the extraction result by the ruled line extraction unit 23. The target forms are classified by performing cluster analysis (machine learning) using the degree of similarity of.

Thereby, in the form recognition system 1 of the embodiment, the form group similar to the representative form can be classified as one subspecies based on the similarity of the representative form with the OCR definition program. Therefore, the form group classified as a subspecies can be made to recognize OCR characters by using the OCR definition program corresponding to the representative form, and it is possible to reduce the cases where the character recognition is erroneous. Therefore, the variants can be sorted so that OCR character recognition can be performed without correcting the definition program used for OCR character recognition.

Further, in the form recognition system 1 of the embodiment, the subspecies classification unit 27 has a degree of similarity between the characteristics of the ruled lines extracted by the ruled line extraction unit 23 and the characteristics of the ruled lines in the representative form (form corresponding to the OCR definition program). The target forms are classified by performing cluster analysis using. As a result, in the form recognition system 1 of the embodiment, it is possible to perform classification according to the degree of similarity without preparing teacher data, and it is possible to perform classification with less time and effort.

Further, in the form recognition system 1 of the embodiment, the representative form is a form selected from the forms in the cluster obtained by performing the cluster analysis on the registration form. As a result, in the form recognition system 1 of the embodiment, it is possible to select a representative form from a group of forms classified into the same group as a subspecies, which has the characteristics common to the group of forms. Therefore, it is possible to recognize the form group classified into the same group with the same OCR definition program.

Further, the form recognition system 1 of the embodiment further includes a conformity determination unit 26. Based on the classification result by the subspecies classification unit 27, the conformity determination unit 26 determines whether or not the subspecies classified into the same group as the representative form conforms to the character recognition using the OCR definition program corresponding to the representative form. judge. As a result, in the form recognition system 1 of the embodiment, it is possible to remove from the group the forms that are difficult to recognize characters with the same OCR definition program, and it is possible to reduce the phenomenon that the OCR character recognition is erroneous. ..

Further, in the form recognition system 1 of the embodiment, whether the conformity determination unit 26 conforms based on the degree of similarity between the characteristics of the ruled lines in the target form classified into the same group as the representative form and the characteristics of the ruled lines in the representative form. Judge whether or not. As a result, in the form recognition system 1 of the embodiment, it is possible to detect an abnormality as a form whose ruled line characteristics are not similar to those of the representative form, and it is possible to perform OCR character recognition more accurately.

Further, in the above-described embodiment, a case where classification is performed for each subspecies within the same type after classification for each type has been described as an example. However, it is not limited to this. Instead of classifying by type, it is possible to directly classify by subspecies from a mixture of various types of forms. Even with this configuration, it can be read by the OCR definition program corresponding to the representative form selected in advance, except for special cases where the ruled line configuration is the same and there are multiple forms with different form titles. It is possible to classify by subspecies.

In the above-described embodiment, various data (image data of learning form, image data of registration form, etc.) are stored in an external DB (database) server device, and the form recognition system 1 communicates via a network or the like. The configuration for acquiring the various data has been described. The external DB server device in this case may be any computer device, but may be, for example, a storage device connected to a network, so-called NAS (Network Attached Storage) or the like. NAS is equipped with a file system and network communication function. Therefore, it is easy to introduce it into the form recognition system 1, and it is also easy to add it according to the amount of data to be stored. In addition, data from each of a plurality of devices of different types (type classification device 10, subspecies classification device 20, character recognition device 30, etc.) of the form recognition system 1 can be shared by each of the plurality of devices. It will be easy.

All or part of the form recognition system 1 in the above-described embodiment may be realized by a computer. In that case, the program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer system and executed. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage device such as a hard disk built in a computer system. Further, a "computer-readable recording medium" is a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line, and dynamically holds the program for a short period of time. It may also include a program that holds a program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or a client in that case. Further, the above program may be for realizing a part of the above-mentioned functions, and may be further realized for realizing the above-mentioned functions in combination with a program already recorded in the computer system. It may be realized by using a programmable logic device such as FPGA (Field Programmable Gate Array).

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs and the like within a range that does not deviate from the gist of the present invention.

1 Form recognition system 10 Type classification device 11 Target image acquisition unit 16 Type classification unit 20 Subspecies classification device 21 Target image acquisition unit 23 Ruled line extraction unit 27 Subspecies classification unit 30 Character recognition device

Claims (8)

The acquisition department that acquires the target form, which is the form to be sorted,
A type classification unit that classifies the types of the target form using the trained model,
A feature extraction unit that extracts the features of the target form,
Based on the classification result by the type classification unit and the extraction result by the feature extraction unit, the target form is performed by performing machine learning using the degree of similarity with the characteristics of the form corresponding to the pre-registered definition program of character recognition. Subspecies classification department that classifies
With
The trained model is a model that is trained so that the output obtained by inputting the learning form approaches the type corresponding to the learning form, and is a model that predicts the type of the input form. be,
A form sorting system characterized by this. The acquisition department that acquires the target form, which is the form to be sorted,
A feature extraction unit that extracts the features of the target form,
Based on the extraction result by the feature extraction unit, the subspecies classification unit that classifies the target form by performing machine learning using the degree of similarity with the feature of the form corresponding to the definition structure of the character recognition defined in advance.
A form sorting system characterized by being equipped with. The subspecies classification unit classifies the target form by performing cluster analysis using the degree of similarity between the characteristics of the ruled lines extracted by the feature extraction unit and the characteristics of the ruled lines in the form corresponding to the definition program. ,
The form sorting system according to claim 1 or 2. The form corresponding to the definition program is a form selected from the forms in the cluster obtained by performing the cluster analysis on the registration form.
The form sorting system according to claim 3. Based on the classification result by the subspecies classification unit, the conformity determination unit determines whether or not the target form classified into the same group as the form corresponding to the definition program conforms to the character recognition using the definition program. Further prepare,
The form sorting system according to any one of claims 1 to 4. The conformity determination unit is based on the degree of similarity between the characteristics of the ruled lines in the target form classified into the same group as the form corresponding to the definition program and the characteristics of the ruled lines in the form corresponding to the definition program. Judging whether or not it conforms to character recognition using
The form sorting system according to claim 5. The acquisition department acquires the target form, which is the form to be sorted,
The type classification unit classifies the type of the target form using the trained model, and then
The feature extraction unit extracts the features of the ruled lines in the target form,
Based on the classification result by the type classification unit and the extraction result by the feature extraction unit, the subspecies classification unit performs machine learning using the degree of similarity with the form corresponding to the predefined character recognition definition program. Classify the target forms according to
It is a form sorting method,
The trained model is a model that is trained so that the output obtained by inputting the learning form approaches the type corresponding to the learning form, and is a model that predicts the type of the input form. be,
A form sorting method characterized by that. Computer,
Acquisition method to acquire the target form, which is the form to be sorted,
A type classification means for classifying the types of the target form using the trained model,
A feature extraction means for extracting the characteristics of ruled lines in the target form,
Based on the classification result by the type classification means and the extraction result by the feature extraction means, the target form is classified by performing machine learning using the degree of similarity with the form corresponding to the definition structure of the character recognition defined in advance. Subspecies classification means,
It is a program to function as
The trained model is a model that is trained so that the output obtained by inputting the learning form approaches the type corresponding to the learning form, and is a model that predicts the type of the input form. be,
program.

Images