JP2017058743A - Recognition program, recognition method and recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a recognition device capable of sorting input images to document images and scene images.SOLUTION: A recognition device 10 executes a series of processing: to acquire input images; to generate a plurality of blurred images in which the degree of blur is different from each other from the input images; to calculate the number of connection components which are obtained by applying a series of labeling processing to each of the blurred images; and to determine whether, among the plurality of blurred images, there are any changes in the degree of blur, in which the number of connection components changes exceeding a predetermined threshold level.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a recognition program, a recognition method, and a recognition apparatus.

  As an example of a technique for recognizing characters included in an image, OCR (Optical Character Recognition) is known.

  For example, an image to which the OCR software is applied includes an image in which the subject of the content included in the image is a character, such as an image in which a document is captured, or an image in which a landscape or the like is captured. There are also images whose main content is other than text, such as landscapes, figures and tables. In the following, the former image is sometimes referred to as a “document image” and the latter image is sometimes referred to as a “scene image”.

  Among these, in a document image, there is a tendency that characters are arranged according to a certain rule, for example, a format of characters, lines, chapters and the like. For this reason, from the character image, the character area where the character exists on the image can be estimated by performing the layout analysis for detecting the line direction, that is, horizontal writing, vertical writing, and line spacing. For example, the histogram of black pixels is generated for each direction by binarizing the document image and scanning the binarized image in the horizontal and vertical directions to count black pixels. Then, the line direction and the line spacing are detected from the histogram of black pixels generated for each direction, and the character area is estimated according to the line direction and the line spacing.

  On the other hand, a character area cannot be easily estimated from a scene image as in a document image. This is because even if characters are included in the scene image, the characters are not always aligned according to the format, such as a signboard character that fits in the landscape, and the characters are locally and irregularly displayed on the scene image. Because it exists. Therefore, when a layout analysis of a scene image is performed, it is difficult to estimate a character area by using a format such as a line direction or a line space. Therefore, an algorithm with an emphasis on discriminating whether or not a portion confused with a character on a scene image is a character is used for layout analysis.

  Thus, the algorithm applied to the layout analysis differs between the document image and the scene image. For this reason, OCR software for document images is applied to document images, while OCR software for scene images is applied to scene images.

Japanese Patent Laid-Open No. 04-330587 Japanese Patent Laid-Open No. 08-305793 Japanese Patent Laid-Open No. 06-020092 JP 2013-157756 A

Teruo Akiyama, Isao Masuda “A method for extracting paper component elements without using format specification information” Vol., J66-D No. 1

  However, with the above technique, the input image cannot be classified into a document image and a scene image.

  In other words, the classification of the document image and the scene image depends on the operation by the user, that is, the operation of switching between the OCR software applied to the input image for the document image or the scene image. . Therefore, if there is no operation by the user, OCR software for a scene image may be applied to the document image, or OCR software for the document image may be applied to the scene image. In this case, there arises a problem that the accuracy of character recognition is reduced, an unknown character is recognized from a part other than the character, or the processing time is increased.

  In addition, there is a case where an erroneous classification is performed even if an attempt is made to classify an input image into a document image or a scene image depending on whether a line direction, a line spacing, or the like is detected by using layout analysis for document images. For example, when a binary image of a scene image includes stripe-shaped black pixels, the row direction and the line spacing may be detected from the histogram of black pixels as in the case of a binary image of a document image. In this case, even if the input image is a scene image, it is erroneously classified into a document image.

  In one aspect, an object of the present invention is to provide a recognition program, a recognition method, and a recognition device that can classify an input image into a document image and a scene image.

  In one aspect, a process for acquiring an input image, a process for generating a plurality of blur images having different degrees of blur from the input image, and a labeling process for the blur image are applied to each blur image. A process for calculating the number of connected components obtained by this, and whether there is a change in the degree of blurring between the plurality of blurred images, wherein the amount of change in the number of connected components is greater than or equal to a predetermined threshold. The determination process is executed.

  The input image can be classified into a document image and a scene image.

FIG. 1 is a diagram illustrating a functional configuration of the recognition apparatus according to the first embodiment. FIG. 2A is a diagram illustrating an example of an image. FIG. 2B is a diagram illustrating an example of an image. FIG. 2C is a diagram illustrating an example of an image. FIG. 2D is a diagram illustrating an example of an image. FIG. 3 is a diagram illustrating an example of an image pyramid. FIG. 4 is a diagram illustrating an example of a blob extraction result in a blurred image. FIG. 5 is a diagram illustrating an example of the blurring function. FIG. 6 is a diagram illustrating an example of a blob extraction result in a blurred image. FIG. 7 is a diagram illustrating an example of the blurring function. FIG. 8 is a flowchart illustrating the procedure of the classification process according to the first embodiment. FIG. 9 is a flowchart illustrating the procedure of the blur image generation process according to the first embodiment. FIG. 10 is a flowchart illustrating a procedure of blob number calculation processing according to the first embodiment. FIG. 11 is a diagram illustrating a hardware configuration example of a computer that executes a recognition program according to the first and second embodiments.

  A recognition program, a recognition method, and a recognition apparatus according to the present application will be described below with reference to the accompanying drawings. Note that this embodiment does not limit the disclosed technology. Each embodiment can be appropriately combined within a range in which processing contents are not contradictory.

[Configuration of recognition device]
FIG. 1 is a diagram illustrating a functional configuration of the recognition apparatus according to the first embodiment. The recognition device 10 shown in FIG. 1 performs character recognition processing for recognizing characters included in an input image. As part of such character recognition processing, it is determined whether or not there is a change in the number of blobs in which a black pixel blob moves from a character unit to a line unit between a series of blurred images generated from an input image with different degrees of blurring. Thus, a classification process for classifying an input image into a document image and a scene image is realized.

  As one embodiment, the recognition apparatus 10 can be implemented by installing a character recognition program in which the above character recognition processing is provided as package software or online software, so-called OCR software, in a desired computer. Such OCR software may be provided as an application program that includes a classification program for realizing the above classification processing as a component, or may be provided as a library in which the classification program is added to the OCR software. For example, the character recognition program is installed not only on mobile communication terminals such as smartphones, mobile phones and PHS (Personal Handyphone System) but also on mobile terminal devices including tablet terminals and slate terminals. Thereby, the mobile terminal device can function as the recognition device 10. Here, as an example, the following description is given by taking the case where the recognition device 10 is implemented as a mobile terminal device, but the recognition program is installed in a stationary terminal device such as a personal computer. You can also.

  Hereinafter, as an example, a case where the above character recognition program is executed on a computer used by a user is exemplified, but the above character recognition program may not necessarily be executed on the user's computer. That is, an image transmitted from a user's computer can be used as an input image, and a function for outputting a classification result of the input image or a character recognition result of the input image can be realized. It may be implemented as a cloud formed by a machine or a group of physical machines.

  FIG. 1 illustrates a processing unit that virtually implements the character recognition process and a storage unit that is referred to or registered by the processing unit. These are the minimum for performing the character recognition process. Only the processing unit and the storage unit are shown. That is, the recognition apparatus 10 may have various functional units that are standardly installed in known computers in addition to the functional units illustrated in FIG. For example, when the recognition apparatus 10 is mounted as a tablet terminal, it may have a motion sensor such as an acceleration sensor or an angular velocity sensor. When the recognition device 10 is implemented as a mobile communication terminal, the recognition device 10 may further include a functional unit such as an antenna or a GPS (Global Positioning System) receiver. When the recognition device 10 is mounted as a stationary terminal device, it may have an input / output device such as a keyboard, a mouse, and a display.

  As shown in FIG. 1, the recognition device 10 includes an image storage unit 11, an acquisition unit 12, a conversion unit 13, a generation unit 14, a classification unit 15, a first recognition unit 16a, and a second recognition unit 16b. And have.

  The image storage unit 11 is a storage unit that stores image data.

  As an embodiment, when image capturing is performed by a camera (not shown) or the like, image data captured by the camera is registered in the image storage unit 11. In addition, when image data is downloaded from an arbitrary computer on the network via a communication interface (not shown) or the like, the downloaded image is registered in the image storage unit 11. The image storage unit 11 can store images in any file format, so-called format.

  2A to 2D are diagrams illustrating examples of images. 2A to 2D show images partially including characters. 2A and 2B show images in which a page of a document is captured. The image shown in FIG. 2A shows one page of a horizontally written paper, and the image shown in FIG. 2B shows one page of a vertically written book. 2C and 2D show images in which a landscape is photographed. In the image shown in FIG. 2C, a character string such as “Senjin-dake” is shown on a signboard that fits in the landscape. In the image shown in FIG. 2D, a poster on the upper left, a bulletin board on the left in the center, and various places A character string is reflected on a part of the cardboard placed in the box.

  Thus, a situation occurs in which a document image and a scene image are stored in a mixed manner in a mobile terminal device represented by a smartphone. For example, as shown in FIGS. 2A and 2B, a document image obtained by taking a document instead of a memo is saved, or as shown in FIGS. 2C and 2D, a scene image obtained by taking a landscape as a record is saved. Or

  Examples of the technical significance of recognizing characters included in these images include the following points. For example, by recognizing characters included in a document image, it can be used for editing in an application such as a text editor, word processor software or spreadsheet software. Further, by recognizing characters included in the scene image, it can be given to the metadata of the scene image as a keyword used for searching the scene image.

  Here, if the operation for switching between the layout analysis applied to the image stored in the image storage unit 11 for the document image or the scene image is entrusted to the user, the convenience for the user is impaired. There is a case. In view of this, the recognition apparatus 10 according to the present embodiment aims to provide an environment in which classification into a document image or a scene image is automated, so that the operation by the user can be omitted and the character recognition process can be executed.

  The acquisition unit 12 is a processing unit that acquires an image.

  As one embodiment, the acquisition unit 12 starts the above classification process under the following conditions, and acquires an image stored in the image storage unit 11. For example, when the acquisition unit 12 receives a designation of an image from among images stored in the image storage unit 11 via an operation unit (not shown) or the like, the acquisition unit 12 reads the image that has received the designation from the image storage unit 11. . In addition, the acquisition unit 12 reads the newly registered image from the image storage unit 11 each time a new image is registered in the image storage unit 11. In this case, the above classification process is started by the acquisition unit 12, and the classification process is executed in the background. The image acquired by the acquisition unit 12 in this way is used as an input for the above classification process. Hereinafter, an image acquired by the acquisition unit 12, that is, an image that is input to the above classification process may be referred to as an “input image”.

  Here, as an example, the case where an image is acquired from the image storage unit 11 included in the recognition device 10 is illustrated as an example, but the recognition device 10 may not necessarily store the image. For example, the acquisition unit 12 can also acquire an image from a computer on a network, such as a file server, a Web server, or a cloud, via a communication interface (not shown). In addition, the acquisition unit 12 can also acquire an image from a removable medium such as a memory card or a USB (Universal Serial Bus) memory.

  The conversion unit 13 is a processing unit that converts the image format of the input image into a predetermined image format.

  As an embodiment, the conversion unit 13 determines whether the input image acquired by the acquisition unit 12 is in a color format. At this time, when the input image is in the color format, the conversion unit 13 converts the input image from the color format to the gray scale format. Such conversion to gray scale can be realized by any method. For example, when the input image is RGB color system image data, the conversion unit 13 converts the RGB pixel values of the pixels included in the input image into pixel values of the L * a * b color system or the YUV color system. By converting the color format to the gray scale format, the brightness L and the luminance Y are set to the pixel values of the respective pixels.

  In addition, although the case where it converts to the pixel value of another color system was illustrated here, it is an example to the last and the conversion method to a gray scale is not limited to this. For example, the conversion unit 13 may realize the conversion by extracting the pixel value of at least one of the RGB pixel values as a representative, or perform statistical processing on the pixel values of a plurality of components Thus, the conversion may be realized by calculating one representative value.

  The generation unit 14 is a processing unit that generates a plurality of blurred images having different degrees of blur from the input image.

As one embodiment, the generation unit 14 generates a blurred image by applying a Gaussian filter to the input image converted from color to grayscale by the conversion unit 13 and performing a filter convolution operation. Here, the generation unit 14 performs a convolution operation that applies a Gaussian filter that applies an image to the input image in a horizontal direction and a convolution operation that applies a Gaussian filter for a vertical direction that applies an image to the input image. Run independently. Thus, the generation unit 14 generates a blurred image in which the input image is blurred in the horizontal direction and a blurred image in which the input image is blurred in the vertical direction. Hereinafter, the former may be referred to as “horizontal blurred image” and the latter may be referred to as “vertically blurred image”. Further, generating unit 14, variance sigma ² of the Gaussian function used for generating the Gaussian ^filter, calculating a Gaussian filter while changing a predetermined update width Δt from the initial value t _s to the target value t _e scale t in other words To generate a Gaussian filter with a different degree of blurring, and perform a convolution operation for each Gaussian filter. As a result, an image pyramid having the structure shown in FIG. 3 is generated from one input image. FIG. 3 is a diagram illustrating an example of an image pyramid. As shown in FIG. 3, for each horizontal and vertical blur direction, a series of horizontal blur images having different horizontal scales t _H and a series of vertical blur images having different vertical scales t _V are obtained. Generated.

[Gaussian filter calculation formula]
Here, the 1st Gaussian filter used for generation of the above-mentioned horizontal blur image can be calculated according to the following formula (1) as an example. In addition, the second Gaussian filter used for generating the above-described vertical blur image can be calculated according to the following equation (2) as an example.

In the above equation (1), (x, y) indicates the coordinates on the image. Among these, “x” indicates the X-axis coordinate corresponding to the horizontal direction of the image, while “y” indicates the Y-axis coordinate corresponding to the vertical direction of the image. Further, “t _H ” in the above equation (1) indicates a horizontal scale, and “t _V ” in the above equation (2) indicates a vertical scale. Further, “c” in the above formula (1) and the above formula (2) indicates a constant that is sufficiently smaller than 1 and has a positive sign, and is set based on the result of an experiment or the like. For example, a value of 0.1 to 0.2 is adopted as the constant c.

Using these formulas (1) and (2), the generation unit 14 calculates a Gaussian filter with t = 0 by setting the scale t to an initial value t _s , for example, “0”, and then the value of the scale t. There is calculated while incrementing the value of the scale t by Δt a Gaussian filter of each scale t for each horizontal and vertical bulk lend direction until updated to the target value t _e. As a result, not only Gaussian filters with equal isotropic numbers of rows and columns are calculated, but filters with different numbers of rows and columns, that is, Gaussian filters with anisotropy are calculated. There is also. For example, as the scale t increases, a Gaussian filter having a number of rows larger than the number of columns is generated in the case of the first Gaussian filter, whereas a Gaussian having a number of columns larger than the number of rows in the case of the second Gaussian filter. The possibility that a filter will be generated increases.

Here, when the scale t is “0”, the variance σ ^{2 of the} Gaussian function is set to “0”. Therefore, from the input image to which the Gaussian filter of t = 0 is applied, an image without a blur, that is, the same image as the input image is generated as a blur image. Therefore, the Gaussian filter with t = 0 does not necessarily have to be calculated by the calculation using the above formula (1) and the above formula (2), and the input image is directly used as the gain image with t = 0. it can. Here, a case has been exemplified where the initial value t _s of scale t is "0", the initial value t _s is acceptable not necessarily "0".

[Filter size]
As described above, when the Gaussian filter is calculated while updating the scale t stepwise, the filter size of the Gaussian filter can be changed in conjunction with the scale t. This is because if a smaller filter size than the scale t is set despite the fact that the input image is blurred in a larger range as the scale t is set larger, the filter size becomes a constraint and the horizontal or vertical direction is limited. This is because the trick is not working properly.

  Therefore, the generation unit 14 sets a larger filter size as the scale t of the Gaussian filter increases. For example, the filter size L of the first Gaussian filter and the second Gaussian filter can be calculated by the following equation (3). For example, “σ” in the following equation (3) is a standard deviation of a Gaussian function. “N” in the following formula (3) is an integer, and the value can be changed depending on whether calculation speed or accuracy is prioritized. For example, when priority is given to calculation speed over accuracy, “1” can be adopted as the integer n. Further, when the accuracy is given priority over the calculation speed, “2” can be adopted as the integer n. In addition, “integer” in the following formula (3) indicates an integer type, and here, as an example, a multiplication value of n * σ is converted into an integer by rounding off or the like. However, when σ is a small value other than 0, the minimum value of L is 3.

  L = 2 * integer (n * σ) +1 (3)

  The classification unit 15 is a processing unit that classifies an input image into a document image and a scene image.

  As one embodiment, the classification unit 15 determines whether or not there is a change in the number of blobs in which a black pixel blob moves from a character unit to a line unit between a series of blurred images generated from input images with different degrees of blurring. By determining, the input image is classified into a document image and a scene image. That is, the classification unit 15 has an input image having a format-specific hierarchical structure, for example, a character, line, chapter, page, or other hierarchical structure, depending on the amount of change in the number of black pixels in the blurred image with respect to the change in the degree of blurring. It is determined whether or not.

  Here, even if the document image includes a figure, a table, and a photograph, it is highly probable that vertical or horizontal character portions exist at a certain ratio or more. Therefore, the document image includes image elements having a unique scale in each layer such as characters, lines, chapters (a group of lines), and pages. On the other hand, a scene image does not have a hierarchical structure with a unique scale, and is dotted with image elements of various sizes.

  For this reason, if there is a part where the decrease in the number of blobs of black pixels is large, a character-by-character blob can go to a line-by-line blob, or a line-by-line blob can go to a chapter or page-by-blob. Probability that there was a change in the degree of profit. If there is such a change in the degree of blurring, it can be estimated that the input image has a format-specific hierarchical structure. As a result, the likelihood that the input image is a document image is increased. On the other hand, if there is no change in the degree of blurring, it can be estimated that the input image does not have a format-specific hierarchical structure. As a result, the likelihood that the input image is a scene image increases. Thereby, the classification unit 15 can automatically classify the input image into a document image and a scene image.

  As shown in FIG. 1, the classification unit 15 includes a blob number calculation unit 15a, an extreme value calculation unit 15b, and a determination unit 15c.

  The blob number calculation unit 15a is a processing unit that calculates the number of blobs included in the lost image. The “blob” here refers to a connected component of black pixels to which the same label is given by the labeling process.

  As an embodiment, the blob number calculation unit 15a performs the following process on each of the series of horizontal blur images and the series of vertical blur images generated by the generation unit 14. That is, the blob number calculation unit 15a determines whether or not the pixel value of each pixel included in the blurred image is equal to or greater than a threshold value. Then, the blob number calculation unit 15a sets a pixel value “0” corresponding to black to a pixel having a pixel value equal to or greater than a threshold value, and a pixel value “1” corresponding to white to a pixel having a pixel value less than the threshold value. "Is set. Accordingly, a binary image in which each pixel included in the blurred image is binarized to a white pixel value “1” or a black pixel value “0” can be obtained.

  Then, the blob number calculation unit 15a performs a labeling process on the binarized image. The “labeling process” here refers to a process of assigning the same label to pixels in which white or black continues on a binarized image, and any known method can be applied. Since characters in the document are expressed in black or a color similar to black, here, as an example, a case is assumed in which the same identification information is assigned to pixels in which black pixel values “0” are continuous. As described above, the labeling process is performed on the binarized image of the blurred image, so that connected components of black pixels to which the same label is assigned are extracted as “blob”. After that, the blob number calculation unit 15a counts the blob number of black pixels extracted from the binarized image of the blurred image by the labeling process, and converts the blob number of the black pixels into a logarithm.

Thus, for each horizontal and vertical blur direction, a correspondence relationship between the scale t of a series of blurred images and the logarithm of the number of blobs of black pixels included in the blurred image of each scale t is obtained. In the following, each processing unit virtually realized on the processor will treat the change in the logarithm of the number of blobs as a function having the scale t as an independent variable, and the function will be referred to as a “crimping function”. There is a case. Furthermore, when referring to the direction of the blurring function as a whole, it is described as “cracking function log f (t)”. In addition to describing the “cracking function log f (t _H )”, it refers to the whisker function whose vertical direction is the vertical direction, and “cracking function logf (t _V )”. There is a case.

  The extreme value calculation unit 15b is a processing unit that calculates the extreme value of the first-order differentiation of the blurring function.

As one embodiment, the extreme value calculation unit 15b calculates the first derivative dlog f (t) / dt by differentiating the blurring function log f (t) with a scale t. Further, the extreme value calculation unit 15b calculates the second derivative d ² log f (t) / dt ² of the blurring function. Then, the extreme value calculation unit 15b detects the zero cross point at which the value of the second derivative d ² log f (t) / dt ² of the recursive function is “0”, thereby obtaining 1 of the recursive function. The extreme value of the second derivative dlog f (t) / dt, here, the minimum value is calculated because the number of blobs monotonously decreases as the scale increases due to the Gaussian filter. In addition, every time the extreme value of the first derivative dlog f (t) / dt of the blurring function is obtained, the extreme value calculation unit 15b increments the number of extreme values obtained so far by one. To count the number of extreme values for each horizontal and vertical blur direction. In this way, the number of extreme values of the first derivative dlog f (t _H ) / dt and the extreme value of the first derivative dlog f (t _V ) / dt in each of the horizontal and vertical blur directions. A number is required.

  The determination unit 15c is a processing unit that determines whether or not the number of extreme values of the first derivative of the blurring function satisfies a predetermined condition.

As one embodiment, the determination unit 15c has the number of extreme values of the first derivative dlog f (t _H ) / dt of the blurring function whose horizontal direction is calculated by the extreme value calculation unit 15b. Determine if greater than zero. At this time, when the number of extreme values of the first derivative dlog f (t _H ) / dt is larger than zero, the determination unit 15c has the vertical direction calculated by the extreme value calculation unit 15b. It is further determined whether or not the number of extreme values of the first derivative dlog f (t _V ) / dt of the scare function is greater than zero. Here, if the number of extreme values of the first derivative dlog f (t _V ) / dt is larger than zero, it is estimated that the input image has a format-specific hierarchical structure in both horizontal and vertical blur directions. This increases the likelihood that the input image is a document image. In this case, the determination unit 15c classifies the input image into “document image”, and outputs the input image to the first recognition unit 16a that executes the character recognition process by the layout analysis for the document image. On the other hand, if the number of extreme values of the first derivative dlog f (t _H ) / dt is zero, or the number of extreme values of the first derivative dlog f (t _V ) / dt is zero, the horizontal or vertical In at least one of the above blur directions, it can be estimated that the input image does not have a hierarchical structure peculiar to the format, so that the input image is more likely to be a scene image. In this case, the determination unit 15c classifies the input image into a “scene image”, and outputs the input image to the second recognition unit 16b that executes character recognition processing by layout analysis for the scene image.

Here, an example of an input image classification method will be described with reference to FIGS. 4 and 6 are diagrams showing an example of blob extraction results in a blurred image. 5 and 7 are diagrams illustrating an example of the blurring function. 4 and 6 show blob extraction results extracted from a blurred image of an input image in which one page of a horizontally written two-column document is captured. FIG. 4 shows a blob extracted from a series of horizontal blur images with different horizontal scales t _H , and FIG. 6 shows an extract from a series of vertical blur images with different vertical scales t _V. The blobs are shown. Further, the upper part of FIG. 5 shows a blurring function log f (t _H ) whose horizontal direction is the horizontal direction, and the lower part of FIG. 5 shows the first derivative dlog f (t _H ). / dt is shown. Further, the upper part of FIG. 7 shows a blurring function log f (t _V ) whose vertical direction is the vertical direction, and the lower part of FIG. 7 shows the first derivative dlog f (t _V ). / dt is shown.

The upper part of FIG. 4 shows a blob extraction result from a horizontally blurred image (= input image) at t _H = 0. As can be seen, when the scale t _H is the initial value, the blob in character units is extracted from the lost image. When the scale t _H is updated in increments Delta] t, as shown in the middle of FIG. 4, blurred by blob horizontal halo However in characters into blobs row. Further, when the scale t _H is updated in increments of Δt, as shown in the lower part of FIG. 4, the row-by-row blobs are spread to the page-by-page blobs by horizontal fringing.

From the logarithm of the blob number calculated from the series of horizontal blur images, the blurring function log f (t _H ) shown in the upper part of FIG. 5 is obtained. By detecting a zero-cross point where the value of the second derivative d ² log f (t _H ) / dt ² of the above-described crack function is “0”, the first derivative d log f (t The extreme value of _H ) / dt is calculated. That is, as shown in the lower part of FIG. 5, the point of scale t = t _{HM1 and} the point of scale t = t _HM2 are calculated as extreme values. These two extremes are: a scale in which a character-wise blob has been turned into a line-by-line blob by a horizontal break, and a scale in which a line-by-line blob has been turned into a page-by-page blob by a horizontal break. Corresponding to

As described above, since the number of extreme values of the first derivative dlog f (t _H ) / dt can be determined to be “2” in the horizontal direction, the input image has characters and lines (columns) in the horizontal direction. It can be estimated that it has three hierarchical structures such as one line of a set) and a page.

On the other hand, the upper part of FIG. 6 shows the blob extraction result from the vertical blurred image (= input image) of t _V = 0. This street, at the stage of the scale t _V is the initial value, it can be seen that the blobs of character units are extracted from the blurred image. When the scale t _H is updated in increments Delta] t, as shown in the lower part of FIG. 6, blob character units blurs the blobs in page units by the vertical umbrella however.

The blurring function log f (t _V ) shown in the upper part of FIG. 7 is obtained from the logarithm of the blob number calculated from the series of vertical blur images. By detecting a zero-cross point where the value of the second derivative d ² log f (t _V ) / dt ² of the above-described crack function is “0”, the first derivative d log f (t The extreme value of _V ) / dt is calculated. That is, as shown in the lower part of FIG. 7, a point of scale t = t _VM1 is calculated as an extreme value. Such an extreme value corresponds to a scale in which a blob in a character unit has opened into a blob in a page unit by a vertical fringe.

As described above, since it is possible to determine that the number of extreme values of the first derivative dlog f (t _V ) / dt is “1” in the vertical direction, the input image is 2 characters such as characters and pages in the vertical direction. It can be estimated that it has two hierarchical structures.

As described above, whether or not the number of extreme values of the first derivative dlog f (t _H ) / dt is greater than zero, and further, the number of extreme values of the first derivative dlog f (t _V ) / dt is By performing the determination of whether it is greater than zero, the processor can estimate whether the input image has a format-specific hierarchical structure in both horizontal and vertical blur directions. In this example, the number of extreme values of the first derivative dlog f (t _H ) / dt is larger than zero, and the number of extreme values of the first derivative dlog f (t _V ) / dt is larger than zero. The input image can be accurately classified into “document image”.

  Here, the reason why the logarithm of the blob number is used in place of the blob number in the blurring function is as follows. For example, the amount of blob reduction is greatly different between a case where a character-by-character blob goes to a line-by-line blob and a case where a line-by-line blob goes to a page-by-page blob. That is, the reduction width when a blob in a line unit is opened to a blob in a page unit may be significantly smaller than the reduction width when a blob in a character unit is moved to a blob in a line unit. Thus, the larger the scale, the harder it is to detect the decrease in the number of blobs than when the scale is small. For this reason, the logarithm of the number of blobs is used instead of the number of blobs in order to cause the processor to handle the same level value regardless of whether the scale is small or large.

  Here, the case where the input image is classified into the document image and the scene image by determining the presence or absence of the extremum number is exemplified, but the change point at which the blob's constituent unit is switched by the gain is the change in the logarithm of the blob number. Since the amount, that is, whether the reduction width is equal to or larger than a predetermined threshold value, it is possible to estimate the input image depending on whether there is a change point of the scale where the logarithmic reduction width of the blob number is equal to or larger than the threshold value. It can also be classified into a document image and a scene image.

  The first recognition unit 16a and the second recognition unit 16b are both processing units that recognize characters included in the input image. Between the first recognizing unit 16a and the second recognizing unit 16b, the first recognizing unit 16a uses layout analysis for document images for character recognition processing, while the second recognizing unit 16b performs layout analysis for scene images as characters. It differs in the point used for recognition processing.

  The processing units such as the acquisition unit 12, the conversion unit 13, the generation unit 14, the classification unit 15, the first recognition unit 16a, and the second recognition unit 16b can be implemented as follows. For example, it can be realized by causing a central processing unit, a so-called CPU (Central Processing Unit), or the like to develop and execute a process that exhibits the same function as each of the above-described processing units on a memory. These processing units do not necessarily have to be executed by the central processing unit, but may be executed by an MPU (Micro Processing Unit). Each functional unit described above can also be realized by a hard wired logic such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

  Further, for example, various semiconductor memory elements such as a RAM (Random Access Memory) and a flash memory can be employed in the image storage unit 11. The image storage unit 11 is not necessarily a main storage device, and may be an auxiliary storage device. In this case, an HDD (Hard Disk Drive), an optical disk, an SSD (Solid State Drive), or the like can be employed.

[Process flow]
Next, the flow of processing of the recognition apparatus 10 according to the present embodiment will be described. Here, (1) the classification process executed by the recognition apparatus 10 will be described, and then (2) a generated image generation process and (3) a blob number calculation process executed as a classification process subroutine will be described. I will do it.

(1) Classification Processing FIG. 8 is a flowchart illustrating a classification processing procedure according to the first embodiment. As an example, this process is started when an image designation is received from images stored in the image storage unit 11 or when a new image is registered in the image storage unit 11.

  As shown in FIG. 8, when an image is acquired by the acquisition unit 12 (step S101), the conversion unit 13 converts the input image from a color format to a grayscale format (step S102). If the input image is grayscale, the process of step S102 can be skipped.

  Then, the generation unit 14 performs “bright image generation processing” that generates a plurality of blur images having different degrees of blur from the input image converted to grayscale in step S102 (step S103).

  Subsequently, the blob count calculation unit 15a calculates the number of blobs included in the blur image for each of the series of horizontal blur images and the series of vertical blur images generated in step S103. Number calculation process "is executed (step S104).

After that, the extremum calculation unit 15b calculates 2 of the blur function determined from the logarithm of the number of blobs of black pixels included in the blur image of each scale t calculated for each horizontal and vertical blur direction in step S104. By detecting the zero crossing point where the value of the ^second derivative d ² log f (t) / dt ² is “0”, the extreme value of the first derivative dlog f (t) / dt of the exponential function is horizontal And it calculates for every vertical direction (step S105). Then, the extreme value calculation unit 15b counts the number of times the extreme value is calculated in step S105, that is, the number of extreme values for each horizontal and vertical direction (step S106).

Subsequently, the determination unit 15c determines that the number of extreme values of the first derivative dlog f (t _H ) / dt of the stroke function in which the stroke direction is the horizontal direction among the number of extreme values counted in step S106. It is determined whether or not it is greater than zero (step S107).

At this time, when the number of extreme values of the first derivative dlog f (t _H ) / dt is greater than zero (Yes in step S107), the determination unit 15c determines the direction of the outliers counted in step S106. It is further determined whether or not the number of extreme values of the first derivative dlog f (t _V ) / dt of the wrinkle function in the vertical direction is greater than zero (step S108).

Here, when the number of extreme values of the first derivative dlog f (t _V ) / dt is larger than zero (Yes in step S108), the input image has a format-specific hierarchical structure in both horizontal and vertical direction. Therefore, it is highly probable that the input image is a document image. In this case, the determination unit 15c classifies the input image into “document image” (step S109), and outputs the input image to the first recognition unit 16a that executes the character recognition process by the layout analysis for the document image (step S109). S111), the process is terminated.

On the other hand, when the number of extreme values of the first derivative dlog f (t _H ) / dt is zero or the number of extreme values of the first derivative dlog f (t _V ) / dt is zero (step S107 No or In step S108 No), the following determination can be made. That is, since it can be estimated that the input image does not have a format-specific hierarchical structure in at least one of the horizontal and vertical blur directions, the likelihood that the input image is a scene image increases. In this case, the determination unit 15c classifies the input image into a “scene image” (step S110), and outputs the input image to the second recognition unit 16b that executes character recognition processing by layout analysis for the scene image (step S110). S111), the process is terminated.

(2) Blurred Image Generation Processing FIG. 9 is a flowchart illustrating the procedure of the blurred image generation processing according to the first embodiment. This process corresponds to the process of step S103 shown in FIG. 8, and is started when the input image is converted to gray scale in step S102, or when the gray scale input image is acquired in step S101. It should be noted that, here, whether the blur direction is horizontal or vertical, except for using the horizontal algorithm of equation (1) or the vertical algorithm of equation (2) for the calculation of the Gaussian filter. Since the processing contents are the same, a case where one blurred image is generated, that is, a horizontal blurred image is generated will be exemplified.

As illustrated in FIG. 9, the generation unit 14 sets the scale t _H to an initial value t _s , for example, “0” (step S301). Then, generation unit 14, the algorithm bulk lend direction to calculate the Gaussian filter is horizontal, that is, by applying the scale t _H in the above equation (1) to calculate a Gaussian filter of the scale t _H (Step S302).

Then, generation unit 14 applies the Gaussian filter scale t _H calculated in step S302 to the input image converted to grayscale format in step S102, it performs the convolution operation (step S303). As a result, a horizontally blurred image having a scale t _H is generated.

At this time, if the scale _{t H} is not updated to the target value _{t e} (step S304No), generating unit 14 increments the update value Δt value for the predetermined scale _{t H} (step S305), the above steps S302 And the process of step S303 is repeatedly executed. Thereafter, when the scale _{t H} are updated to the target value _{t e} (step S304Yes), the process ends.

  Here, the case where a series of horizontally blurred images is generated is illustrated with reference to FIG. 9, but the vertical of Equation (2) is also used in the calculation of the Gaussian filter when generating a vertically blurred image. By using a direction algorithm, a series of vertically blurred images can be generated in the same manner. As for the horizontal blur image and the vertical blur image, either blur image may be generated first, or both may be generated in parallel.

(3) Blob Number Calculation Processing FIG. 10 is a flowchart illustrating a procedure of blob number calculation processing according to the first embodiment. This process corresponds to the process of step S104 shown in FIG. 8, and is executed after the process of step S103 is executed.

  As shown in FIG. 10, the blob number calculation unit 15a selects one scale t used for generating a blurred image in step S103 (step S501). Subsequently, the blob count calculation unit 15a generates a binarized image of the horizontal blur image corresponding to the scale t selected in step S501 (step S502).

  Then, the blob count calculation unit 15a counts the blobs obtained by applying the labeling process to the binarized image of the horizontally blurred image obtained in step S502 (step S503). Subsequently, the blob number calculation unit 15a converts the blob number obtained in step S503 into a logarithm (step S504).

  Further, the blob number calculation unit 15a generates a binarized image of the vertical blur image corresponding to the scale t selected in step S501 (step S505).

  Then, the blob number calculation unit 15a counts the blobs obtained by applying the labeling process to the binarized image of the vertical blur image obtained in step S505 (step S506). Subsequently, the blob number calculation unit 15a converts the blob number obtained in step S506 into a logarithm (step S507).

  Thereafter, until there is no unselected scale (step S508 Yes), an unselected scale is selected (step S501), and the processes of steps S502 to S507 are repeatedly executed. When there is no unselected scale (No in step S508), the process is terminated.

[Effect of Example 1]
As described above, in the recognition apparatus 10 according to the first embodiment, the blob of the black pixel is generated from the character unit to the line unit between the series of blurred images generated from the input image with different degrees of blurring. By determining whether or not the number of blobs has changed, the input image is classified into a document image and a scene image.

  In other words, whether the input image has a format-specific hierarchical structure, for example, a character, line, chapter, page, or other hierarchical structure, depending on the amount of change in the number of black pixels in the blurred image with respect to the change in the degree of blurring. Determine. For example, if there is a part where the decrease in the number of blobs of black pixels is large, a character-by-character blob may be turned into a line-by-line blob, or a line-by-line blob may be turned into a chapter or page-by-blob. The likelihood that the degree of change will increase.

  If there is such a change in the degree of blurring, it can be estimated that the input image has a format-specific hierarchical structure. As a result, the likelihood that the input image is a document image is increased. On the other hand, if there is no change in the degree of blurring, it can be estimated that the input image does not have a format-specific hierarchical structure. As a result, the likelihood that the input image is a scene image increases.

  Therefore, according to the recognition apparatus 10 according to the present embodiment, the input image can be automatically classified into the document image and the scene image.

  Although the embodiments related to the disclosed apparatus have been described above, the present invention may be implemented in various different forms other than the above-described embodiments. Therefore, another embodiment included in the present invention will be described below.

[Scale update width Δt]
The scale update width Δt described in the first embodiment is too small to be observed in the graph of the first derivative of the logarithm of the blob number if the value is too large. And the processing time tends to increase.

From this, the following processing can be executed in order to appropriately set the scale update width Δt. For example, after extracting blobs by applying a labeling process to the binarized image obtained by binarizing the input image, the average height Sh and the average width Sw of the circumscribed rectangle of each blob are calculated. . It can be estimated that the average size Sh and the average width Sw of the height of the circumscribed rectangle of each blob are approximately equal to the average character height and the average character width. On top of that, when generating a horizontal blur images, scale _t sufficiently small value the initial value ts of _H, for example, starting from _t S = 0.0001, than Sw an update width Δt of the scale _{t H} small value, for example, and sets the Delta] t = Sw / 5, it is possible to set the number of updates Delta] t to the target value t _e width / Delta] t of the input image, i.e., the target value t _e and width / 2 of the input image . In the case of generating a vertical blur image, “Sw” is replaced with “Sh” in the description of the previous horizontal blur image, and “input image width” is replaced with “input image height”. , similarly, can be appropriately set update width Δt and the target value t _e of the scale t _V.

[How to generate a profitable image]
In the first embodiment, the case where a blurred image of an input image is generated using a Gaussian filter is illustrated, but the method for generating a blurred image is not limited to this. For example, a series of blurred images can be generated by reducing the input image in stages. In this case, the process of thinning out pixels applies the input image. However, in the thinning-out process, the pixel having the lowest pixel value among a plurality of pixels, that is, the pixel close to black is left while the other pixels are removed. By thinning out, it is possible to obtain a blur image equivalent to a blur image generated by a convolution operation using a Gaussian filter.

[Distribution and integration]
In addition, each component of each illustrated apparatus does not necessarily have to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, the acquisition unit 12, the conversion unit 13, the generation unit 14, the classification unit 15, the first recognition unit 16a, or the second recognition unit 16b may be connected as an external device of the recognition device 10 via a network. As an example, a server device having an acquisition unit 12, a conversion unit 13, a generation unit 14, and a classification unit 15 can be constructed, and the classification result can be transmitted to the client terminal. In addition, the acquisition unit 12, the conversion unit 13, the generation unit 14, the classification unit 15, the first recognition unit 16a or the second recognition unit 16b each have another device, and are connected to a network to cooperate with each other. You may make it implement | achieve the function of the recognition apparatus 10. FIG.

[Recognition program]
The various processes described in the above embodiments can be realized by executing a prepared program on a computer such as a personal computer or a workstation. In the following, an example of a computer that executes a recognition program having the same function as that of the above embodiment will be described with reference to FIG.

  FIG. 11 is a diagram illustrating a hardware configuration example of a computer that executes a recognition program according to the first and second embodiments. As illustrated in FIG. 11, the computer 100 includes an operation unit 110 a, a speaker 110 b, a camera 110 c, a display 120, and a communication unit 130. Further, the computer 100 includes a CPU 150, a ROM 160, an HDD 170, and a RAM 180. These units 110 to 180 are connected via a bus 140.

  As shown in FIG. 11, the HDD 170 stores a recognition program 170 a that exhibits the same functions as the acquisition unit 12, the conversion unit 13, the generation unit 14, and the classification unit 15 described in the first embodiment. This recognition program 170a may be integrated or separated as in the constituent elements of the acquisition unit 12, the conversion unit 13, the generation unit 14, and the classification unit 15 illustrated in FIG. That is, the HDD 170 does not necessarily have to store all the data shown in the first embodiment, and data used for processing may be stored in the HDD 170.

  Under such an environment, the CPU 150 reads the recognition program 170 a from the HDD 170 and expands it in the RAM 180. As a result, the recognition program 170a functions as a recognition process 180a as shown in FIG. The recognition process 180a expands various data read from the HDD 170 in an area allocated to the recognition process 180a in the storage area of the RAM 180, and executes various processes using the expanded various data. For example, the processes shown in FIGS. 8 to 10 are included as an example of the process executed by the recognition process 180a. Note that the CPU 150 does not necessarily operate all the processing units described in the first embodiment, and the processing unit corresponding to the process to be executed may be virtually realized.

  The above recognition program 170a does not necessarily have to be stored in the HDD 170 or the ROM 160 from the beginning. For example, the recognition program 170a is stored in a “portable physical medium” such as a flexible disk inserted into the computer 100, so-called FD, CD-ROM, DVD disk, magneto-optical disk, IC card or the like. Then, the computer 100 may acquire and execute the recognition program 170a from these portable physical media. In addition, the recognition program 170a is stored in another computer or server device connected to the computer 100 via a public line, the Internet, a LAN, a WAN, etc., and the computer 100 acquires the recognition program 170a from these and executes it. You may make it do.

DESCRIPTION OF SYMBOLS 10 Recognition apparatus 11 Image memory | storage part 12 Acquisition part 13 Conversion part 14 Generation part 15 Classification | category part 15a Blob number calculation part 15b Extreme value calculation part 15c Determination part 16a 1st recognition part 16b 2nd recognition part

Claims (6)

On the computer,
Processing to acquire the input image;
A process of generating a plurality of blur images having different blur degrees from the input image;
A process for calculating the number of connected components obtained by applying a labeling process to the blurred image for each blurred image;
A process for determining whether or not there is a change in the degree of blur in which the amount of change in the number of connected components is greater than or equal to a predetermined threshold value among the plurality of blur images. . The generating process includes a first blurred image group in which the input image is blurred in a first direction, and a second blur in which the input image is blurred in a direction perpendicular to the first direction. A group of images and
In the determination process, there is a change in the degree of blur in which the amount of change in the number of connected components is greater than or equal to a predetermined threshold between the first blur image groups, and the second blur image group 2. The recognition program according to claim 1, further comprising: In the computer,
Further executing a process of calculating a statistical value of the height or width of a connected component obtained by applying a labeling process to the input image;
The generating process generates the plurality of blur images while updating the blur degree according to an update width determined from a statistical value of a height or a statistical value of the connected component. The recognition program according to 1 or 2.   The generation process updates the blur degree from a first value to a second value determined by a statistical value of the height or width of the connected component and a height or width of the input image. The recognition program according to claim 3, wherein the plurality of blurred images are generated. Computer
Processing to acquire the input image;
A process of generating a plurality of blur images having different blur degrees from the input image;
A process for calculating the number of connected components obtained by applying a labeling process to the blurred image for each blurred image;
And a process of determining whether or not there is a change in the degree of blur in which the amount of change in the number of connected components is greater than or equal to a predetermined threshold value among the plurality of blur images. . An acquisition unit for acquiring an input image;
A generating unit that generates a plurality of blur images having different degrees of blur from the input image;
A calculation unit that calculates the number of connected components obtained by applying a labeling process to the blurred image for each blurred image;
And a determination unit that determines whether there is a change in the degree of blur in which the amount of change in the number of connected components is greater than or equal to a predetermined threshold value among the plurality of blur images. .