JP7377661B2 - Image semantic region segmentation device, region detection sensitivity improvement method, and program - Google Patents

Description

The present invention relates to an image semantic region segmentation device, a method for improving region detection sensitivity, and a program.

In recent years, in semantic region segmentation (also referred to as semantic segmentation) of digital images, a method of automatically extracting image features by machine learning using DCNN (Deep Convolutional Neural Network) has been frequently used.

For example, there is a problem of performing semantic region segmentation on a document image obtained by converting the document into a digital image using a scanner, camera, etc., and determining the class (=element type) of each pixel. Here, the class is defined according to the classification type that the user wants to distinguish semantically in the target image, such as a character class, figure class, photograph class, or background class.

The discrete two-dimensional convolution (Convolution) operation in the DCNN described above is for extracting spatial shape features of an image pattern. For example, the Convolution operation in DCNN can be a means of extracting basic shape features constituting an image, such as horizontal edges, vertical edges, diagonal edges, rectangular corners, circles, etc. By using the convolution operation, it is ideally possible to have feature extraction power that is compatible with all feature patterns and the directions in which the features occur (compatible with all directions). However, in real convolution based on machine learning (hereinafter also referred to as learning-type convolution), the characteristics of feature extraction are determined by the weight parameters acquired through a finite number of learnings, so it is not completely ideal. It is normal to include a certain amount of bias. The constant bias is, for example, a bias in which the output for an edge at 45 degrees to the left is slightly larger than the output for an edge at 45 degrees to the right.

Therefore, in the semantic region segmentation using the learning type convolution described above, the output may not be completely line-symmetric or rotationally symmetric with respect to the input image. For example, ideally, the region segmentation result for a certain image pattern A and the semantic region segmentation result for an image obtained by rotating pattern A by 90 degrees should match, but in reality, they often differ slightly. .

Similarly, convolution calculations on a digital computer are discrete, so even if the input image pattern is the same, the output results may differ depending on the position on the image. For example, the output label of a certain pattern B may not match the output label of pattern B shifted by one pixel.

Due to the practical characteristics of convolution using machine learning, pixels that should be detected may not be detected due to rotation, vertical flipping, or positional shift of the image during semantic region segmentation. As a result, detection sensitivity is reduced.

In Patent Document 1, segmentation (area division) of concrete images in the architectural field is performed using CNN (Convolutional Neural Network), and in order to improve the accuracy, a first image feature detector using machine learning is used. This problem is dealt with by using a region divider based on second machine learning.

JP 2019-133433 Publication

The problem to be solved by the present invention is to prevent a decrease in region detection sensitivity due to discrete convolution in semantic region segmentation of an image using machine learning. The aim is to be particularly effective in images such as document images in which characters, figures, etc. are arranged on a relatively uniform background (=non-natural images).

Patent Document 1 is an example of a segmentation application using CNN, but here it uses two machine learning devices to improve accuracy. Since two machine learning machines are used, there is a problem that the calculation time and memory capacity required for learning become large. Furthermore, this document also has the problem of not making good use of the characteristics of discrete convolution obtained by machine learning, as explained in the background art section of this case.

The present invention aims to improve the sensitivity (also called accuracy) of semantic region segmentation using only one machine learning machine by taking into account and effectively utilizing the realistic characteristics of discrete convolution as described above. .

The present invention has been made in view of this situation, and provides an image semantic region segmentation device and region detection sensitivity improvement that can improve the sensitivity of semantic region segmentation using only one machine learning device. A method and program are provided.

In order to solve the above-mentioned problems of the present invention, the present invention includes an acquisition unit that acquires image information including pixel values for each pixel in a target image including characters and geometric figures; Based on the image information obtained, the pixel value of each pixel in the target image is changed to a first pixel value if the pixel is an edge, and the first pixel value is changed if the pixel is not an edge. a generation unit that generates a modulated image changed to a different second pixel value , and a pixel in the image is a character element indicating an element constituting a character or a geometric element indicating an element constituting a geometric figure; An estimation unit (meaning based on the result of estimating the element type for each pixel in each of the target image and the modulated image, which is estimated by the estimation unit. The determination device includes a determination unit that determines the element type.
In order to solve the above-mentioned problems of the present invention, the present invention includes an acquisition unit that acquires image information including pixel values for each pixel in a target image including characters and geometric figures; a generation unit that generates a modulated image by moving the pixel coordinates of each pixel in the target image based on the image information that has been The element type that distinguishes whether it is a geometric element that indicates an element that constitutes a geometric figure or a background element that indicates an element that constitutes a background that is not a character or a geometric figure has been learned with a convolution integral layer. An estimating unit that estimates using a model (a part that uses semantic region segmentation) estimates the element type for each pixel in each of the target image and the modulated image, which is estimated by the estimating unit. The determination device includes a determination unit that determines the element type for each pixel in the target image based on the result.

Further, in the above-mentioned determination device, the present invention provides that the generation unit changes the pixel value of each pixel in the target image to a predetermined pixel value depending on whether the pixel is an edge. do.

Further, in the above-mentioned determination device, the present invention provides that the estimating unit estimates the element type of a pixel in an image using a trained model, and the trained model is a learning image that is a learning image. This is a learning result in which a data set is information that associates image information with the element type of a pixel in the learning image, and a learning model is subjected to machine learning on the data set.

Further, in the above-mentioned determination device, the present invention provides at least one of the element type of a predetermined pixel in the target image and the element type of a corresponding pixel corresponding to the predetermined pixel in the modulated image. is the character element, the element type of the predetermined pixel is determined to be the character element, and at least one of the element type of the predetermined pixel and the element type of the corresponding pixel is the geometric element. If it is determined that the element type of the predetermined pixel is the geometric element, and the element type of the predetermined pixel and the element type of the corresponding pixel are both the background element, the predetermined The element type of the pixel is determined to be the background element.

Further, the present invention provides an acquisition unit that acquires image information including a pixel value for each pixel in a target image including characters and geometric figures, and a generation unit that generates information about the image information acquired by the acquisition unit. , the pixel value of each pixel in the target image is changed to a first pixel value when the pixel is an edge, and a second pixel value different from the first pixel value when the pixel is not an edge. The estimator generates a modulated image changed to The determining unit estimates the element type to distinguish whether it is a background element indicating an element constituting a background that is not a geometric figure, using the same region divider regardless of the image to be estimated , and the determining unit This determination method determines the element type of a pixel in the target image based on the result of estimating the element type for each pixel in each of the target image and the modulated image.
Further, the present invention provides an acquisition unit that acquires image information including a pixel value for each pixel in a target image including characters and geometric figures, and a generation unit that generates information about the image information acquired by the acquisition unit. The estimator generates a modulated image in which the pixel coordinates of each pixel in the target image are moved based on A trained model with a convolutional integral layer is used to distinguish element types to distinguish between geometric elements indicating constituent elements and background elements indicating elements constituting backgrounds other than characters and geometric figures. and the determining unit determines the element type of the pixel in the target image based on the result of estimating the element type for each pixel in each of the target image and the modulated image, estimated by the estimating unit. This is a determination method.

Further, the present invention is a program for causing a computer to operate as the determination device described above, and a program for causing the computer to function as each unit included in the determination device.

According to the present invention, characters and geometric figures in an image can be distinguished.

FIG. 1 is a block diagram illustrating an example of the configuration of a region dividing device 10 according to an embodiment. FIG. 2 is a diagram illustrating processing performed by the region dividing device 10 according to the embodiment. It is a flowchart showing the flow of processing performed by the region dividing apparatus 10 according to the embodiment.

Embodiments of the invention will be described below with reference to the drawings.

The region dividing device 10 is a device that divides the region of an image for each semantic type of content shown in the image (semantic region division). In the following description, an example will be explained in which regions are divided by the types of lines, characters, and background shown in an image as semantic types. However, the semantic type of content shown in the image is not limited to this. In addition to lines, characters, and backgrounds, the region dividing device 10 may divide an image into regions according to types such as images, figures, symbols, colors, shapes, etc., and divides the image into regions according to such semantic types. In this case, a method similar to the method described below can be applied.

The region dividing device 10 determines, for each pixel of the image, what kind of content shown in the image the pixel constitutes, and based on the determined result, divides the region into regions for each content shown in the image. Divide. That is, the area dividing device 10 is an example of a "determination device" that determines whether a pixel in an image is an element of what content shown in the image. For example, the area dividing device 10 determines whether the element is a character or another element. Here, the other elements are, for example, geometric figures. A geometric figure is a geometric figure, such as a line, a line segment, or a group of symbols arranged to satisfy certain conditions.

FIG. 1 is a block diagram showing an example of the configuration of a region dividing apparatus 10 according to an embodiment. The region dividing device 10 includes, for example, an image information acquisition section 11, a modulated image generation section 12, an element type estimation section 13, an element type determination section 14, a region map generation section 15, and a map information output section 16. Be prepared. Here, the image information acquisition section 11 is an example of an "acquisition section." The modulated image generation unit 12 is an example of a “generation unit”. The element type estimation unit 13 is an example of an “estimation unit”. The element type determining unit 14 is an example of a “determining unit”.

The image information acquisition unit 11 acquires image information of the scan image G11 (see FIG. 2). The scan image G11 is an image including lines and characters. A ruled line, a frame line, etc. may be formed by combining lines or bending (or curving) a part of the line. The scan image G11 is an image targeted for region division by the region division device 10. That is, the scan image G11 is an example of a "target image."

The scanned image G11 is an image created by reading, for example, a printed image G10 (see FIG. 2), which is a printed image displayed on a display or the Web, with a scanner. The image information is information in which image-related information is associated with each pixel, such as information indicating a gray scale value for each pixel or information indicating RGB values for each pixel. The image information acquisition unit 11 outputs the acquired image information to the modulated image generation unit 12 and the element type estimation unit 13.

The modulated image generation unit 12 generates an enhanced image G12 (see FIG. 2) based on the image information acquired from the image information acquisition unit 11. The emphasized image G12 is an image in which the pixel value (gray scale value or RGB value) of each pixel in the scan image G11 is changed based on predetermined modulation conditions, and is an example of a "modulated image."

The modulated image generation unit 12 generates, for example, an image that has undergone enhancement processing to emphasize the edges of the scan image G11 as an enhanced image G12. In this case, the modulated image generation unit 12 detects edges in the scan image G11. The modulated image generation unit 12 detects edges using any conventional method. For example, the modulated image generation unit 12 detects edges by detecting the difference between the scan image G11 subjected to median filter processing and the scan image G11 subjected to smoothing processing using a Gaussian filter or the like. Alternatively, the modulated image generation unit 12 may detect edges in the scan image G11 by applying a Laplacian filter or a Sobel filter.

The modulated image generation unit 12 sets the detected edge to a certain pixel value (for example, a gray scale value indicating "black" or an RGB value), and sets the pixel value of the pixel that is not detected as another edge to another pixel value. An enhanced image G12 is generated in which the pixel values are changed to specific pixel values (for example, grayscale values indicating "white" or RGB values).

Note that the emphasized image G12 is not limited to an image with emphasized edges as described above. The emphasized image G12 may be any image generated according to predetermined modulation conditions, and for example, pixels within the image may be moved horizontally and/or vertically at a predetermined distance (for example, by a predetermined number of pixels). The image may also be an image that has been moved by a distance corresponding to minutes. The modulated image generation unit 12 outputs image information of the generated emphasized image G12 to the element type estimation unit 13.

The element type estimation unit 13 estimates the element type for each pixel in each of the scan image G11 and the emphasized image G12. The element type is information indicating what kind of element a pixel constitutes in an image, and is information indicating one of a character element, a line segment element, and a background element. The character element indicates that the pixel is an element constituting a character in the image. A line segment element indicates that a pixel is an element constituting a line segment in an image. The background element indicates that the pixel is an element constituting the background (not a line segment and not a character) in an image. Here, the line segment element is an example of a "geometric element."

The element type estimation unit 13 estimates the element type in an image using, for example, a machine learning method. For example, the element type estimating unit 13 estimates the element type in the image using the learned model. The learned model is a model that has learned the relationship between image information and element type for each pixel.

(Basics of DCNN)
The learning method for the trained model is, for example, supervised learning. The trained model is generated by training a model such as a DCNN (Deep Convolutional Neural Network) using a training data set. DCNN is a deep neural network that mainly uses a convolution layer. In image recognition, a two-dimensional convolution layer is used as an input layer in DCNN. Thereby, image feature information that takes into account information about both the pixel of interest and pixels in its vicinity can be efficiently recognized. In image recognition, two-dimensional convolutions are further layered and applied. This makes it possible to recognize global image feature information that takes into account not only information in the vicinity of the pixel of interest but also information on pixels further away.
(Learning DCNN)
Convolution layer calculations can be expressed by a mathematical linear transformation formula (y=<W,x>+b). That is, this is a differentiable calculation formula. The differentiable computational layer can perform learning using the principle of supervised learning of neural networks known as error backpropagation.

In DCNN, when data is output from a unit in a certain layer to a unit in a deeper layer, the data is given a weight W according to the coupling coefficient of the node connecting the units and a bias component b. Ru. The learning model performs calculations between each unit on input data (input data), and outputs output data from the output layer.

The learning data set in this embodiment is information that associates pixel information as input with an element type for each pixel.

During the learning process, input data of a training dataset is input to the learning model. The learning model adjusts the parameters (weight W and bias component b) of the learning model so that the data output from the output layer (output data) with respect to the input data approaches the output of the training dataset. The learning model is trained.

For example, the error backpropagation method is used to adjust the parameters (weight W and bias component b) of the DCNN model. In the error backpropagation method, the degree of deviation between the data output from the output layer of the learning model and the output of the training data and set is expressed as a loss function. Any index may be used for the degree of deviation here, and for example, the square of the error (squared error), cross entropy, etc. may be used. In the error backpropagation method, the values of the weight W and the bias component b are determined (updated) so that the loss function is minimized in the direction from the output layer to the input layer side. This allows the learning model to learn and improves estimation accuracy.

Note that the learning model is not limited to DCNN. As the learning model, for example, CNN, decision tree, hierarchical Bayes, SVM (Support Vector Machine), or other methods may be used.

The element type estimation unit 13 acquires an output (element type) from the learned model by inputting image information to the learned model. The output from the trained model is for each element type, for example, ``12% chance of being a text element, 80% chance of being a line element, 8% chance of being a background element.'' This is information that indicates possibility by probability. The modulated image generation unit 12 estimates, for example, the element type indicated by the highest probability for each pixel as the element type in the image, based on the output from the trained model.

The element type estimation unit 13 estimates the element type for each pixel in the scan image G11 based on the output obtained by inputting the image information of the scan image G11 into the learned model, and sends the estimation result to the element type determination unit. Output to 14. The element type estimating unit 13 estimates the element type for each pixel in the emphasized image G12 based on the output obtained by inputting the image information of the emphasized image G12 into the learned model, and sends the estimation result to the element type determining unit. Output to 14.

Note that the above description has been made using an example in which the element type estimating unit 13 estimates the element type for each pixel using a machine learning method. However, it is not limited to this. The element type estimation unit 13 may estimate the element type for each pixel using a method that does not use a machine learning method, for example, using a rule base. In this case, estimation is made based on rules registered in advance. The rules here specify conditions according to the element type of the pixel. For example, if a predetermined number of pixels with a predetermined gray scale value are consecutive in the horizontal direction, these pixels Rules such as making the line element a line element.

The element type determination unit 14 determines the element type in the scan image G11 based on the estimation result of estimating the element type for each pixel in both the scan image G11 and the emphasized image G12, which is obtained from the element type estimation unit 13. .

The element type determining unit 14 obtains, for example, a pixel (corresponding pixel) of the emphasized image G12 that corresponds to a predetermined pixel (predetermined pixel) in the scan image G11. The relationship between a predetermined pixel and a corresponding pixel may be arbitrarily determined according to modulation processing (processing corresponding to predetermined modulation conditions). For example, when the modulation process is a process that emphasizes edges, the predetermined pixel and the corresponding pixel are pixels located at the same position coordinates in each image (scan image G11 and emphasized image G12). For example, if the modulation process is a process of moving a pixel in an image a predetermined distance in the horizontal and/or vertical direction, the relationship between the predetermined pixel and the corresponding pixel is determined by the position coordinate of the predetermined pixel. The position after moving the distance is the position coordinates of the corresponding pixel.

The element type determination unit 14 estimates the element type of the predetermined pixel based on the estimation result of the predetermined pixel and the estimation result of the corresponding pixel. If at least one of the element type of the predetermined pixel and the element type of the corresponding pixel is estimated to be a text element, the element type determination unit 14 determines that the predetermined pixel is a text element. That is, when the predetermined pixel is estimated to be a character element, the element type determination unit 14 determines the predetermined pixel to be a character element, regardless of the element type estimated for the corresponding pixel. Furthermore, when the corresponding pixel is estimated to be a text element, the element type determination unit 14 determines the predetermined pixel to be a text element, regardless of the element type estimated for the predetermined pixel.

If at least one of the element type of the predetermined pixel and the element type of the corresponding pixel is estimated to be a line segment element, the element type determination unit 14 determines that the predetermined pixel is a line segment element. That is, when the predetermined pixel is estimated to be a line segment element, the element type determination unit 14 determines the predetermined pixel to be a line segment element, regardless of the element type estimated for the corresponding pixel. Furthermore, when the corresponding pixel is estimated to be a line segment element, the element type determination unit 14 determines the predetermined pixel to be a line segment element, regardless of the element type estimated for the predetermined pixel.

The element type determining unit 14 determines that the predetermined pixel is a background element when both the element type of the predetermined pixel and the element type of the corresponding pixel are estimated to be background elements. That is, when the predetermined pixel is estimated to be a background element and the corresponding pixel is estimated to be a background element, the element type determining unit 14 determines the predetermined pixel to be a background element. The element type determination unit 14 outputs information indicating the element type determined for each pixel in the scan image G11 to the area map generation unit 15.

The area map generation unit 15 generates an area map based on information indicating the element type determined for each pixel in the scan image G11 from the element type determination unit 14. The area map is a map (image) in which pixels are associated with element types. The area map generation unit 15 generates an area map for each element type, for example.
The area map generation unit 15 sets pixels whose element type is a text element to a certain specific color (for example, black), and sets pixels whose element type is not a text element to a different color (for example, white). Generate a region map.
The area map generation unit 15 sets pixels whose element type is a line segment element to a certain specific color (for example, black), and sets pixels whose element type is not a line segment element to a different color (for example, white). Generate a region map of minute elements.
The area map generation unit 15 sets pixels whose element type is a background element to a certain color (for example, black), and sets pixels that are not background elements to a different color (for example, white), thereby changing the color of the background element. Generate a region map.
The area map generation unit 15 causes a storage unit (not shown) to store information indicating the generated area map.

The map information output unit 16 refers to the storage unit according to a user's operation and outputs information indicating a predetermined area map. The map information output unit 16 may display the area map by outputting information indicating the area map to a display connected to the area dividing device 10. Furthermore, the map information output unit 16 may print the area map by outputting information indicating the area map to a printer connected to the area dividing device 10.

FIG. 2 is a diagram illustrating processing performed by the region dividing apparatus 10 according to the embodiment.
As shown in FIG. 2, the print image G10 is an image in which, for example, characters such as "AIUEO", "X", and "ABC" are shown, as well as a frame line that is a combination of a plurality of line segments. As shown in this example, the print image G10 may include a mixture of bold and thin characters, and may also include characters with different character colors and background colors. Furthermore, characters may be written within the frame, or a frame line may be further drawn within the frame.
In the scanned image G11 (an example of a "target image"), for example, noise occurred throughout the image when it was read by a scanner, and the parts shown in white in the printed image G10 were shown in light gray and black. This is an image in which the text and background appear to have changed to a dark gray color.

The modulated image generation unit 12 generates an emphasized image G12 by performing a predetermined process (herein referred to as "modulation process") based on the image information of the scan image G11. The emphasized image G12 is, for example, an image in which both the edges of characters and the edges of frames are emphasized. In this example, characters written in bold in the scanned image G11 are converted into so-called white characters in the emphasized image G12, regardless of the color of the characters. Further, the characters shown in fine print in the scan image G11 are shown as they are in the emphasized image G12 along the shape of the characters. Furthermore, line segments drawn as thick lines in the scan image G11 have been converted to look like double frames in the emphasized image G12.

The element type estimating unit 13 (described as region divider (processing) in FIG. 2) estimates the element type for each pixel in the scan image G11. The element type estimating unit 13 outputs an estimated area map M10 of the scan image G11 as an estimation result. The estimated area map M10 is a map (image) in which estimated element types are associated with each pixel of the scan image G11. In this way, the element type estimation unit 13 may output the estimation result in the form of a map (image).

Furthermore, the element type estimating unit 13 estimates the element type for each pixel with respect to the emphasized image G12. The element type estimation unit 13 outputs an estimation area map M11 of the emphasized image G12 as an estimation result. The estimated region map M11 is a map (image) in which estimated element types are associated with each pixel of the emphasized image G12. In this way, the element type estimation unit 13 may output the estimation result in the form of a map (image).

The element type determination unit 14 determines the element type for each pixel of the scan image G11 by combining the estimated area maps M10 and M11. The combination here indicates that the element type of a predetermined pixel is determined according to the estimation result of a predetermined pixel in the scan image G11 and the estimation result of a corresponding pixel in the emphasized image G12, as described above. ing.
The area map generation unit 15 generates the area map M12 of the scan image G11 based on the element type for each pixel of the scan image G11 determined by the element type determination unit 14.

FIG. 3 is a flowchart showing the flow of processing performed by the region dividing apparatus 10 according to the embodiment. The image information acquisition unit 11 of the region dividing device 10 acquires image information of the scan image G11 (step S10). The modulated image generation unit 12 generates an enhanced image G12 based on the image information of the scan image G11 (step S11). The element type estimation unit 13 estimates the element type of the pixel in the scan image G11 based on the image information of the scan image G11 (step S12). The element type estimating unit 13 estimates the element type of the pixel in the emphasized image G12 based on the image information of the emphasized image G12 (step S13). The element type determination unit 14 determines the element type for each pixel in the scan image G11 based on the estimation result of the element type of each pixel in the scan image G11 and the emphasized image G12 (step S14). The area map generation unit 15 generates an area map M12 for each element type based on the element type for each pixel in the scan image G11 (step S15).

Note that in the above-described flow, an example is shown in which the element type of the pixel in the scanned image G11 is estimated in step S12, and then the element type of the pixel in the emphasized image G12 is estimated in step S13. After the process shown in S13 is performed, the process shown in step S13 may be performed.

As described above, the region dividing apparatus 10 of the embodiment includes the image information acquisition section 11, the modulated image generation section 12, the element type estimation section 13, and the element type determination section 14. The image information acquisition unit 11 acquires image information of the scan image G11. The modulated image generation unit 12 generates an emphasized image G12 in which the pixel value of each pixel in the scan image G11 is changed according to predetermined modulation conditions. The element type estimation unit 13 estimates the element type for each pixel in the image. The element type determining unit 14 determines the element type of the scan image G11 based on the estimation result of estimating the element type for each pixel in the image. Thereby, the region dividing apparatus 10 of the embodiment can determine character elements and line segment elements as the element types of the scan image G11.

Here, as a comparative example, consider a configuration in which the element type is determined using only the estimation result of the scan image G11. Generally, when performing estimation using a trained model, it is possible to perform estimation with high accuracy for inputs that are the same as or similar to the training dataset. On the other hand, it is difficult to accurately estimate inputs that are not in the learning dataset. Therefore, if the image information of scanned image G11 includes a pixel arrangement pattern that was not in the training dataset, errors may occur in the estimation results for pixels included in that pattern or pixels around that pattern. There is a high possibility that there is. Since the estimation results will not change unless the trained model is retrained, the estimation accuracy cannot be improved in such a case. That is, it becomes difficult to accurately estimate a portion where the image information of the scan image G11 and the content of the learning data set deviate. The same holds true even when a rule-based estimation (determination) result is used instead of a learned model.

In contrast, in the region dividing apparatus 10 of this embodiment, the element type of the scan image G11 is determined using the estimation result of the scan image G11 and the estimation result of the emphasized image G12. The emphasized image G12 is an image generated by performing predetermined modulation processing on the scan image G11. In this way, even if there is an error in the estimation result for a specific pixel in the scanned image G11, there is a possibility that the corresponding pixel in the emphasized image G12 can be estimated with high accuracy. That is, by using the estimation result of the scan image G11 and the estimation result of the enhanced image G12, it is possible to accurately estimate even the portion where the image information of the scan image G11 and the content of the learning data set diverge. It becomes possible. That is, it is possible to improve the sensitivity of semantic region segmentation using only one machine learning device.

Furthermore, in the region dividing device 10 of the present embodiment, the modulated image generation unit 12 sets the pixel value of each pixel in the scan image G11 to a predetermined pixel value depending on whether the pixel is an edge. Change to Thereby, the region dividing apparatus 10 of the embodiment can generate an emphasized image G12 in which the edges of character elements and line segment elements are emphasized. Therefore, even if there are characters or line segments that are difficult to estimate accurately using only the image information of the scan image G11, the estimation results when character elements and line segment elements are emphasized are Using this method, it becomes possible to estimate with high accuracy.

Furthermore, in the region dividing apparatus 10 of this embodiment, the element type estimating unit 13 estimates the element type of a pixel in an image using the learned model. The learned model is a learning result obtained by machine learning the data set using the data set, which is information that associates the image information of the learning image, which is the learning image, with the element type of the pixel in the learning image. It is. Thereby, in the region dividing apparatus 10 of this embodiment, element types can be estimated by a simple method of inputting image information to a trained model.

Furthermore, in the region dividing apparatus 10 of the present embodiment, the element type determination unit 14 determines whether at least one of the element type of a predetermined pixel in the scan image G11 and the element type of the corresponding pixel in the emphasized image G12 is a text element. , the predetermined pixel is determined to be a character element. If at least one of the element type of the predetermined pixel in the scan image G11 and the element type of the corresponding pixel in the emphasized image G12 is a line segment element, the element type determining unit 14 determines that the predetermined pixel is a line segment element. do. If the element type of the predetermined pixel in the scan image G11 and the element type of the corresponding pixel in the emphasized image G12 are both background elements, the element type determination unit 14 determines that the predetermined pixel is a background element. As a result, in the region dividing apparatus 10 of the present embodiment, even if there are line segments or characters that are difficult to estimate accurately using only the image information of the scan image G11, line segment elements, It becomes possible to estimate with high accuracy using the estimation results when character elements are emphasized.

All or part of the region dividing apparatus 10 in the embodiment described above may be realized by a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed. Note that the "computer system" herein includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. Furthermore, a "computer-readable recording medium" refers to a storage medium that dynamically stores a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It may also include a device that retains a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. Further, the above-mentioned program may be one for realizing a part of the above-mentioned functions, or may be one that can realize the above-mentioned functions in combination with a program already recorded in the computer system. It may also be realized using a programmable logic device such as an FPGA.

Although the embodiments of the present invention have been described above in detail with reference to the drawings, the specific configuration is not limited to these embodiments, and includes designs within the scope of the gist of the present invention.

10...Region dividing device 11...Image information acquisition unit (acquisition unit)
12...Modulated image generation unit (generation unit)
13... Element type estimation section (estimation section)
14...Element type determination section (determination section)
15... Area map generation unit 16... Map information output unit

Claims (7)

an acquisition unit that acquires image information including pixel values for each pixel in a target image including characters and geometric figures;
Based on the image information acquired by the acquisition unit, the pixel value of each pixel in the target image is changed to the first pixel value when the pixel is an edge, and the pixel value is changed to the first pixel value when the pixel is not an edge. a generation unit that generates a modulated image changed to a second pixel value different from the first pixel value ;
Pixels in the image are character elements that represent elements that constitute characters, geometric elements that represent elements that constitute geometric figures, or elements that constitute background that are not characters or geometric figures. an estimation unit that estimates an element type to distinguish whether it is a background element using the same region divider regardless of the image to be estimated ;
a determining unit that determines the element type for each pixel in the target image based on the result of estimating the element type for each pixel in each of the target image and the modulated image, which is estimated by the estimating unit;
A determination device comprising: an acquisition unit that acquires image information including pixel values for each pixel in a target image including characters and geometric figures;
a generation unit that generates a modulated image in which pixel coordinates in the target image are moved based on the image information acquired by the acquisition unit;
Pixels in the image are character elements that represent elements that constitute characters, geometric elements that represent elements that constitute geometric figures, or elements that constitute background that are not characters or geometric figures. an estimation unit that estimates an element type to distinguish whether it is a background element using a trained model having a convolutional integral layer ;
a determining unit that determines the element type for each pixel in the target image based on the result of estimating the element type for each pixel in each of the target image and the modulated image, which is estimated by the estimating unit;
A determination device comprising: The estimation unit estimates the element type of the pixel in the image using the learned model,
The trained model has a data set that includes image information of a learning image, which is a learning image, and information that associates the element type of a pixel in the learning image, and has the learning model perform machine learning on the data set. The learning result is
The determination device according to claim 1 . The determining unit is
When at least one of the element type of the predetermined pixel in the target image and the element type of the corresponding pixel corresponding to the predetermined pixel in the modulated image is the character element, the element type of the predetermined pixel is It is determined that it is a character element,
If at least one of the element type of the predetermined pixel and the element type of the corresponding pixel is the geometric element, determining that the element type of the predetermined pixel is the geometric element,
When the element type of the predetermined pixel and the element type of the corresponding pixel are both the background element, determining that the element type of the predetermined pixel is the background element;
The determination device according to any one of claims 1 to 3. the acquisition unit acquires image information including a pixel value for each pixel in a target image including characters and geometric figures;
A generation unit changes the pixel value of each pixel in the target image to a first pixel value when the pixel is an edge, and when the pixel is not an edge, based on the image information acquired by the acquisition unit. generate a modulated image with a second pixel value different from the first pixel value ,
The estimation unit determines whether the pixels in the image are character elements representing elements that constitute a character, geometric elements representing elements that constitute a geometric figure, or constitute a background that is not a character or a geometric figure. The element type that distinguishes whether an element is a background element or not is estimated using the same region divider regardless of the image to be estimated ,
a determining unit determining the element type of the pixel in the target image based on the result of estimating the element type for each pixel in each of the target image and the modulated image, estimated by the estimating unit;
Judgment method. the acquisition unit acquires image information including a pixel value for each pixel in a target image including characters and geometric figures;
a generation unit generates a modulated image in which pixel coordinates in the target image are moved based on the image information acquired by the acquisition unit;
The estimation unit determines whether the pixels in the image are character elements representing elements that constitute a character, geometric elements representing elements that constitute a geometric figure, or constitute a background that is not a character or a geometric figure. Estimating the element type that distinguishes whether the element is a background element or not using a trained model having a convolutional integral layer ,
a determining unit determining the element type of the pixel in the target image based on the result of estimating the element type for each pixel in each of the target image and the modulated image, estimated by the estimating unit;
Judgment method. A program for causing a computer to operate as the determination device according to any one of claims 1 to 4, the program for causing the computer to function as each unit included in the determination device.

Images