JP2021047797A - Machine learning device, machine learning method, and program - Google Patents

Abstract

To provide a machine learning device, a machine learning method, and a program that can improve the accuracy in distinguishing between pixels belonging to a character area and pixels belonging to a figure area in an image by using a learned model.SOLUTION: A machine learning device comprises: an estimation unit that estimates an element type for classifying pixels in an image including characters and geometric figures into character elements indicating elements constituting the characters, geometric elements indicating elements constituting the geometric figures, and background elements indicating elements constituting the background that is not the characters and geometric figures; an evaluation unit that evaluates a result of estimation made by the estimation unit based on training data corresponding to the image; and a learning control unit that, based on a result of evaluation made by the evaluation unit, makes the estimation unit learn to determine the element type of pixels present in a predetermined area of the image including the characters as the character elements.SELECTED DRAWING: Figure 1

Description

The present invention relates to machine learning devices, machine learning methods, and programs.

In recent years, various techniques for recognizing characters and geometric figures in a document image by character recognition have been proposed.

For example, Patent Document 1 below discloses a technique for determining whether or not a pixel in a predetermined region of a document image is a character pixel indicating a character by using machine learning. In this technique, a machine learning model in which a document image is input determines whether or not a predetermined area is a character-representing area based on a character probability indicating the probability that a pixel in a predetermined area is a character-representing pixel. judge.

JP-A-2019-57803

However, when the purpose of the determination is to roughly determine whether the character is a character or another area (geometric figure / background), the technique of Patent Document 1 may make an erroneous determination. In the technique of Patent Document 1, the type of a character is determined with a fine particle size of one pixel unit. For example, it is mistaken that a part of a confusing geometric figure having a shape similar to that of a character is a character. It is judged. On the contrary, the character may be erroneously determined to be part of a geometric figure.

In view of the above problems, an object of the present invention is a machine learning device and machine learning capable of improving the accuracy of distinguishing a pixel belonging to a character area and a pixel belonging to a graphic area in an image by using a trained model. To provide methods and programs.

In order to solve the above-mentioned problems, in the machine learning device according to one aspect of the present invention, whether the pixels in the image including the character and the geometric figure are character elements indicating the elements constituting the character, or geometry. An estimation unit that estimates the element type that distinguishes between a geometric element that indicates an element that constitutes a target figure and a background element that indicates an element that is not a character and a background that is not a geometric figure, and the image. An evaluation unit that evaluates the estimation result by the estimation unit based on the corresponding teacher data, and the element type of the pixel existing in a predetermined region including the character in the image based on the evaluation result by the evaluation unit. It includes a learning control unit that causes the estimation unit to learn to determine that it is a character element.

In the machine learning method according to one aspect of the present invention, the estimation unit determines that the pixels in the image including the character and the geometric figure are character elements indicating the elements constituting the character, or the geometric figure is formed. Estimating the element type that distinguishes whether it is a geometric element indicating an element or a background element indicating an element constituting a background that is not a character or a geometric figure, and an evaluation unit corresponds to the image. Based on the teacher data, the estimation result by the estimation unit is evaluated, and the learning control unit evaluates the element of the pixel existing in a predetermined region including the character in the image based on the evaluation result by the evaluation unit. It includes having the estimation unit learn to determine the type as the character element.

In a program according to an aspect of the present invention, a computer indicates that a pixel in an image including a character and a geometric figure is a character element indicating an element constituting the character, or an element constituting the geometric figure. Based on the estimation unit that estimates the element type that distinguishes whether it is a geometric element or a background element that indicates an element that constitutes a background that is not a character or a geometric figure, and the teacher data corresponding to the image. Based on the evaluation unit that evaluates the estimation result by the estimation unit and the evaluation result by the evaluation unit, it is determined that the element type of the pixel existing in the predetermined region including the character in the image is determined as the character element. It functions as a learning control unit to be trained by the estimation unit.

According to the present invention, it is possible to improve the accuracy of distinguishing between the pixels belonging to the character area and the pixels belonging to the graphic area in the image by using the trained model.

It is a block diagram which shows the structural example of the machine learning apparatus which concerns on embodiment of this invention. It is a figure which shows the input / output relation of various data which concerns on this embodiment. It is a flowchart which shows the flow of processing in the machine learning apparatus which concerns on this embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Configuration example of machine learning device>
First, the machine learning device according to the present embodiment will be described with reference to FIGS. 1 and 2. FIG. 1 is a block diagram showing a configuration example of the machine learning device 10 according to the embodiment of the present invention. FIG. 2 is a diagram showing an input / output relationship of various data according to the embodiment of the present invention.

The machine learning device 10 is a device having a function of generating a trained model used for machine learning. Hereinafter, an example in which the machine learning device 10 generates a trained model used when determining the element type for each pixel in an image by machine learning will be described.

The element type is information indicating what kind of element each pixel in the image constitutes, and is information indicating any one of a character element, a line segment element, and a background element. The character element indicates that the pixel is an element that constitutes a character area in the image. The line segment element indicates that the pixel is an element that constitutes a line segment in the image. The background element indicates that the pixel is an element that constitutes the background (not a line segment and not a character area) in the image. Here, the line segment element is an example of a "geometric element".

The character area is an area (predetermined area) including characters in the image. The character area may include a background in addition to the characters. It should be noted that one character area may include one character in the image, or may include a plurality of characters in the image. The shape of the character area is, for example, a rectangle, but is not limited to such an example. The size of the character area is set, for example, according to the size of the character. The area in the image that does not include characters is also referred to as a graphic area below. The graphic area is, for example, an area other than the character area.

The trained model can be used as a device (hereinafter, also referred to as a “determination device”) for determining what kind of content each pixel in the image constitutes in the image. For example, the determination device uses the trained model to determine whether a pixel in an image is a character or another element. Here, the other elements are, for example, geometric figures. A geometric figure is a geometric figure, for example, a line, a line segment, a group of symbols arranged in a state satisfying a certain condition, and the like.

An image is an image that includes lines, characters, and a background. Ruled lines, borders, and the like may be formed by combining lines or by bending (or bending) a part of the lines. The image is an image to be determined by the determination device. That is, the image is an example of a "target image".

When the target image (input data) is input, the trained model determines the element type of the pixel in the target image. The trained model is generated by performing machine learning in advance using the training data in order to determine the element type of the pixel of the input target image.

In the generation of the trained model, for example, supervised learning is performed. In supervised learning, a learning model is trained using a learning data set. A data set is a set of input data and teacher data corresponding to the input data.

The input data is data that is input during learning. The input data according to the present embodiment is an image including characters and geometric figures (hereinafter, also referred to as “input image”), and has image information. The image information is information in which information about an image is associated with each pixel, and is, for example, information showing a grayscale value for each pixel, information showing an RGB value for each pixel, and the like.

The teacher data is data indicating the correct answer of the output data output based on the input data. The teacher data according to the present embodiment is information in which information indicating an element type is associated with each pixel of the input image. Further, the teacher data may be associated with information indicating that it is a character area.

(Basics of DCNN)
A trained model in supervised learning is generated by training a model such as DCNN (Deep Convolutional Neural Network) using a data set for learning. DCNN is a deep neural network that uses the Convolution layer as the main part. In image recognition, by using a two-dimensional Convolution layer as an input layer in DCNN, it is possible to efficiently recognize image feature information in which information of both a pixel of interest and a pixel in the vicinity thereof is added. Further, it is known that by superimposing two-dimensional Convolutions and applying them in multiple layers, it is possible to recognize not only the information of pixels in the vicinity of the pixel of interest but also the global image feature information including the information of pixels farther away.

(Learning of DCNN)
The calculation of the Convolution layer can be expressed by a mathematical linear transformation formula (y = <W, x> + b). That is, this is a differentiable formula. The differentiable computational layer can perform learning using the principle of supervised learning of neural networks known as backpropagation.

In DCNN, when data is output from a unit in a certain layer to a unit in a deeper layer, data with a weight W corresponding to the coupling coefficient of the node connecting the units and a bias component b is output. The node. The learning model performs operations between each unit on the input data (input data), and outputs the output data from the output layer.

In the process of learning, the learning model is made to input the input data of the data set for learning. In the learning model, the parameters (weight W and bias component b) of the learning model so that the data (output data) output from the output layer with respect to the input data approaches the output (teacher data) of the data set for training. The learning model is trained by adjusting.

For example, the backpropagation method is used to adjust the parameters (weight W and bias component b) of the DCNN model. In the back-propagation method, the degree of deviation between the data output from the output layer of the learning model and the output of the training data set is expressed as a loss function. Any index may be used for the degree of dissociation here, and for example, the square of the error (square error), cross entropy, or the like is used. In the back-propagation method, the values of the weight W and the bias component b are determined (updated) so that the loss function is minimized in the direction from the output layer to the input layer side. As a result, the learning model is trained and the accuracy of the judgment is improved.

The learning model is not limited to DCNN. As the learning model, for example, a method such as a decision tree, hierarchical Bayes, or SVM (Support Vector Machine) may be used.

As shown in FIG. 1, the machine learning device 10 includes an estimation unit 110, an evaluation unit 120, and a learning control unit 130 in order to realize a function of generating a trained model.

(Estimating unit 110)
The estimation unit 110 has a function of estimating the element type of a pixel in an image. The function is realized by machine learning such as DCNN. That is, the calculation process inside the estimation unit 110 is governed by the model parameter 112, which is a parameter group that changes by learning. Immediately after the start of learning, a random value or the like is set as the initial value of the model parameter 112 of the estimation unit 110, and the parameter is not adjusted so as to estimate the element type as a character element. Is output. As the learning progresses, the learning control unit 130 corrects the parameters and reduces estimation errors. When the error is sufficiently small, the model parameter 112 is determined and the learning is completed. The parameter group obtained in this way is called a "trained model".

Specifically, the estimation unit 110 estimates the element type based on the model parameter 112. In the model parameter 112, for example, a parameter determined by learning is set so as to estimate the element type of the pixel in the character area as a character element. As a result, the estimation unit 110 can estimate the pixels whose element type is a character element in the input image in units of regions.

More specifically, the estimation unit 110 acquires the output (element type) from the model parameter 112 by inputting the image information contained in the input image, which is the input data of the data set, into the model parameter 112, for example. In the output from the model parameter 112, for example, each pixel has an element type such as "12% probability of being a character element, 80% probability of being a line segment element, and 8% probability of being a background element". It is information indicating each possibility by probability (hereinafter, also referred to as "element type probability"). Based on the output from the learning model, the estimation unit 110 estimates, for example, the element type indicated by the highest probability for each pixel as the element type in the image.

After the estimation, the estimation unit 110 outputs information indicating the element type of each pixel as an estimation result. For example, the estimation unit 110 generates an image in which an integer value indicating an estimated element type is shown for each pixel. An image in which an integer value indicating an element type is shown for each pixel is hereinafter referred to as a "label image". The label image generated by the estimation unit 110 is hereinafter referred to as an "estimated label image". Further, the label image which is the teacher data is hereinafter referred to as a "teacher label image".

Further, the integer value indicating the element type is hereinafter referred to as a "label value". An arbitrary value may be set for the label value, and an arbitrary definition may be set for each value. In the present embodiment, for example, a value of 0 to 2 is set for the label value, 0 is defined as a label value indicating a background, 1 is a label value indicating a figure, and 2 is a label value indicating a character area.

After generating the estimated label image, the estimation unit 110 inputs the generated estimated label image to the evaluation unit 120.

Here, an example of the data input / output relationship in the estimation unit 110 will be described with reference to FIG. As shown in FIG. 2, the data set 20 of the present embodiment is composed of an input image 30 which is input data and a teacher label image 40 which is teacher data corresponding to the input image 30.

The input image 30 is an image including the characters "Ai" and two geometric figures that are rectangular frames. The teacher label image 40 is an image in which the element type in each pixel of the input image 30 is replaced with an integer value. 2 is stored as a label value in the pixel to which the character element of the teacher label image 40 is associated. Further, 1 is stored as a label value in the pixel to which the line segment element of the teacher label image 40 is associated. Further, 0 is stored as a label value in the pixel to which the background element of the teacher label image 40 is associated.

As shown in FIG. 2, when the input image 30 is input to the estimation unit 110, the estimation unit 110 outputs the estimation label image 50, which is a label image showing the estimation result, to the evaluation unit 120. The definition of the estimated label image 50 here is the same as the definition of the teacher label image 40 described above. In the example shown in FIG. 2, since the learning of the model parameter 112 is insufficient, the estimation unit 110 outputs an estimated label image 50 including an erroneous label value.

(Evaluation unit 120)
The evaluation unit 120 has a function of evaluating the estimation result. For example, the evaluation unit 120 evaluates the estimation result input from the estimation unit 110 based on the teacher data. Specifically, the evaluation unit 120 compares the input teacher label image with the estimated label image, and calculates the information expressing the degree of deviation by the loss function as the evaluation value. The degree of divergence referred to here is, for example, the degree to which the element types of pixels at the corresponding positions of the teacher label image and the estimated label image are different.

Here, an example of the data input / output relationship in the evaluation unit 120 will be described with reference to FIG. As shown in FIG. 2, the teacher label image 40 of the data set 20 and the estimated label image 50 generated by the estimation unit 110 are input to the evaluation unit 120. The evaluation unit 120 calculates an evaluation value based on the input teacher label image 40 and the estimated label image 50. Then, the evaluation unit 120 inputs the calculated evaluation value as an evaluation result to the learning control unit 130.

(Learning control unit 130)
The learning control unit 130 has a function of controlling machine learning in the machine learning device 10. For example, the learning control unit 130 causes the model parameter 112 of the estimation unit 110 to learn based on the evaluation result input from the evaluation unit 120. At this time, the learning control unit 130 modifies the model parameter 112 based on the evaluation value indicating the evaluation result so that the model parameter 112 learns. Specifically, the learning control unit 130 modifies the model parameter 112 with the value of the parameter determined so that the evaluation value expressed by the loss function is minimized by the error back propagation method. In this way, the learning control unit 130 can improve the accuracy of estimation based on the model parameter 112 by the estimation unit 110 by training the model parameter 112.

Specifically, the learning control unit 130 of the present embodiment causes the estimation unit 110 to learn so that the pixels existing in the character area are determined to be character elements. For example, when the estimation unit 110 estimates the element type probability of the character element of the pixel existing in the character area to be lower than the other element type probabilities, it can be said that the estimation result by the estimation unit 110 is incorrect. Therefore, the learning control unit 130 repeatedly modifies the model parameter 112 until the element type probability of the character element in the pixel becomes higher than the other element type probabilities, and causes the estimation unit 110 to repeatedly learn.

Further, the learning control unit 130 of the present embodiment causes the estimation unit 110 to learn so that the pixels other than the character region are determined to be line segment elements or background elements. For example, the learning control unit 130 causes the estimation unit 110 to learn so that the pixels existing in the predetermined region including the dotted line are determined to be line segment elements. Further, the learning control unit 130 causes the estimation unit 110 to learn the pixels existing in the region where the pixel density changes depending on the boundary line along the vertical direction or the horizontal direction so as to be determined as a line segment element. Further, the learning control unit 130 causes the estimation unit 110 to learn pixels that are not character elements and line segment elements so that they are determined to be background elements. Since the method of learning by the learning control unit 130 is the same as the above-described method of learning so that the pixels existing in the character area are determined to be character elements, the description thereof will be omitted.

The learning control unit 130 modifies the model parameter 112 so as to determine the element type of the pixel existing in the character region in the input image 30 as a character element, and causes the estimation unit 110 to repeatedly estimate the learning. As a result, when the input image 30 is input, the model parameter 112 can determine that the pixels existing in the character area including the characters in the input image are character elements.

Here, an example of the data input / output relationship in the learning control unit 130 will be described with reference to FIG. As shown in FIG. 2, an evaluation value is input from the evaluation unit 120 to the learning control unit 130. The learning control unit 130 corrects the parameter of the model parameter 112 based on the input evaluation value.

The process from the estimation of the element type by the estimation unit 110 shown in FIG. 2 to the modification of the model parameter 112 by the learning control unit 130 can be repeated. That is, the parameter correction and learning in the model parameter 112 are repeated. Immediately after the start of learning, the estimation unit 110 outputs an estimation result containing many errors. However, as the learning progresses, the model parameter 112 is modified, and the errors contained in the estimation result are reduced. That is, the accuracy of estimation in the model parameter 112 is improved by repeating the learning. Therefore, by repeating the learning, the estimation unit 110 can improve the accuracy of estimating the element type based on the model parameter 112.

Further, by repeating the learning, the estimation unit 110 can acquire the model parameter 112 that estimates the type of the pixel based on the arrangement pattern characteristics of the pixel and the pixels existing around the pixel.

When the learning is repeated and the number of errors included in the estimation result is sufficiently reduced, the estimation unit 110 completes the learning. Then, the model parameter 112 obtained by repeating this learning is used as the "learned model".

From the above, the learning control unit 130 determines that the element type of the pixel existing in the character area in the image is a character element from the model parameter 112 in which the initial value is randomly set by causing the estimation unit 110 to repeat the learning. A trained model can be generated.

The model parameter 112 may be preset with an initial value for estimating the element type of the pixel existing in the character area in the image as a character element. In this case, the learning control unit 130 can generate the model parameter 112, which has improved the accuracy of estimating the character element as compared with immediately after the start of learning, as a trained model by causing the estimation unit 110 to repeat the learning.

<Processing flow>
The configuration example of the machine learning device 10 has been described above. Subsequently, the flow of processing in the machine learning device 10 according to the present embodiment will be described. FIG. 3 is a flowchart showing a processing flow in the machine learning device 10 according to the embodiment of the present invention. The modification process of the model parameter based on one data set will be described below.

First, the estimation unit 110 of the machine learning device 10 estimates the element type of the pixel in the input image which is the input data, and generates an estimated label image showing the estimation result (S102). After generation, the estimation unit 110 inputs the generated estimated label image to the evaluation unit 120.

Next, the evaluation unit 120 of the machine learning device 10 evaluates the estimated label image input from the estimation unit 110 based on the teacher label image which is the teacher data, and calculates the evaluation value (S104). After the calculation, the evaluation unit 120 inputs the calculated evaluation value to the learning control unit 130.

Then, the learning control unit 130 of the machine learning device 10 corrects the model parameter 112 of the estimation unit 110 based on the evaluation value input to the evaluation unit 120 (S106).

After modifying the model parameter 112, the learning control unit 130 determines whether or not the error included in the estimation result is sufficiently reduced (S108).

When the number of errors is sufficiently small (S108 / YES), the learning control unit 130 completes the learning and ends the process. When the error is not sufficiently reduced (S108 / NO), the learning control unit 130 repeats the processing of S102 to S106, and repeats the learning to the estimation unit 110 until the error included in the estimation result is sufficiently reduced. Let me.

As described above, the machine learning device 10 according to the present embodiment first estimates the element type of the pixel in the image including the character and the geometric figure. Next, the machine learning device 10 evaluates the estimation result based on the teacher data corresponding to the image. Then, the machine learning device 10 learns to determine the element type of the pixel existing in the character region in the image as the character element based on the evaluation result. With such a configuration, the machine learning device 10 determines pixels whose element type is a geometric element or a background element on a pixel-by-pixel basis, and determines pixels whose element type is a character element on a region-by-area basis.

As a result, the machine learning device 10 can generate a trained model that clearly distinguishes the pixels that are the elements that make up the characters in the image from the pixels that are the elements that make up the figure. Therefore, the machine learning device 10 can improve the accuracy of distinguishing the pixels belonging to the character area and the pixels belonging to the graphic area in the image by using the trained model.

The embodiment of the present invention has been described above. The machine learning device 10 in the above-described embodiment may be realized by a computer. In that case, the program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer system and executed. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage device such as a hard disk built in a computer system. Further, a "computer-readable recording medium" is a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line, and dynamically holds the program for a short period of time. In that case, a program may be held for a certain period of time, such as a volatile memory inside a computer system serving as a server or a client. Further, the above program may be for realizing a part of the above-mentioned functions, and may be further realized for realizing the above-mentioned functions in combination with a program already recorded in the computer system. It may be realized by using a programmable logic device such as FPGA (Field Programmable Gate Array).

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the gist of the present invention. It is possible to do.

10 Machine learning device 110 Estimating unit 112 Model parameter 120 Evaluation unit 130 Learning control unit

Claims (5)

A pixel in an image containing a character and a geometric figure is a character element indicating an element that constitutes a character, a geometric element that indicates an element that constitutes a geometric figure, or a character and a geometric figure. An estimation unit that estimates the element type that distinguishes whether it is a background element that indicates an element that constitutes a background that is not
An evaluation unit that evaluates the estimation result by the estimation unit based on the teacher data corresponding to the image, and an evaluation unit.
Based on the evaluation result by the evaluation unit, a learning control unit that causes the estimation unit to learn to determine the element type of the pixel existing in a predetermined region including the character in the image as the character element.
A machine learning device equipped with. The estimation unit estimates the element type of the pixel in the image based on the model parameters learned based on the evaluation result so as to estimate the element type of the pixel in the predetermined region as the character element. , The machine learning device according to claim 1. The machine learning device according to claim 1 or 2, wherein the predetermined area is an area including the characters and the background. Whether the pixel in the image containing the character and the geometric figure is a character element indicating an element constituting the character, or a geometric element indicating an element constituting the geometric figure, the character and the estimation unit. Estimating the element type that distinguishes whether it is a background element that indicates an element that constitutes a background that is not a geometric figure,
The evaluation unit evaluates the estimation result by the estimation unit based on the teacher data corresponding to the image.
To make the estimation unit learn that the learning control unit determines that the element type of the pixel existing in the predetermined region including the character in the image is the character element based on the evaluation result by the evaluation unit. When,
Machine learning methods, including. Computer,
A pixel in an image containing a character and a geometric figure is a character element indicating an element that constitutes a character, a geometric element that indicates an element that constitutes a geometric figure, or a character and a geometric figure. An estimation unit that estimates the element type that distinguishes whether it is a background element that indicates an element that constitutes a background that is not
An evaluation unit that evaluates the estimation result by the estimation unit based on the teacher data corresponding to the image, and an evaluation unit.
Based on the evaluation result by the evaluation unit, a learning control unit that causes the estimation unit to learn to determine the element type of the pixel existing in a predetermined region including the character in the image as the character element.
A program that functions as.

Images