JP2015179552A - Image processing apparatus and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide technology for determining with a high degree of accuracy whether or not a target area is a character area.SOLUTION: An image processing apparatus comprises: a classification part for using image data showing an image to classify multiple pixels constituting a target area in the image into object pixels constituting an object and background pixels constituting a background excluding the object; a first determination part for determining whether or not to satisfy a first condition for a relationship between the object pixels and the background pixels; a second determination part for determining whether or not to satisfy a second condition for pixel values of the pixels; and an attribute identification part for using a result of the determination by the first determination part and a result of the determination by the second determination part to identify whether or not the target area is a character area.

Description

  The present invention relates to a technique for determining characteristics of a target area.

  A technique for determining the characteristics of a target area in an image based on image data representing the image is known. If the feature of the target area can be correctly determined, there is an advantage that image processing corresponding to the feature can be performed. For example, the image is divided into a plurality of blocks, and the contrast is calculated for each block. And the technique which classifies each block into a photograph, a character, etc. based on the contrast of each block is known (for example, patent document 1).

JP-A-5-225378

  However, a technique for determining whether or not the target region is a character with higher accuracy has been demanded.

  The main advantage of the present invention is to provide a technique for determining whether or not the target area is a character with higher accuracy.

  SUMMARY An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following application examples.

Application Example 1 Using image data representing an image, a plurality of pixels constituting a target area in the image are changed into an object pixel constituting an object and a background pixel constituting a background excluding the object. A classification unit that classifies, a first determination unit that determines whether the target region satisfies a first condition regarding a relationship between the object pixel and the background pixel, and the target region includes the plurality of pixels The target region using a second determination unit that determines whether or not a second condition regarding the pixel value of the first pixel value is satisfied, a determination result by the first determination unit, and a determination result by the second determination unit And an attribute specifying unit that specifies whether or not the character area represents a character.

  According to the above configuration, the plurality of pixels constituting the target area are classified into object pixels and background pixels. In addition to whether or not the second condition regarding the pixel values of the plurality of pixels is satisfied, it is determined whether or not the first condition regarding the relationship between the object pixel and the background pixel is satisfied. Then, using these determination results, it can be accurately determined whether or not the target area is a character area.

Application Example 2 Using image data representing an image, an area specifying unit that specifies a target area in the image and a peripheral area located around the target area, and pixels of the peripheral area Using a pixel value, a classification unit that classifies a plurality of pixels constituting the target region into an object pixel representing an object and a background pixel representing a background excluding the object, the number of the object pixels, and the A feature value calculation unit that calculates a feature value according to a ratio to the number of background pixels, and the target region is a character region that represents a character based on whether or not the target region satisfies a condition related to the feature value. And an attribute specifying unit that specifies whether or not.

  According to the above configuration, the plurality of pixels constituting the target area are classified into object pixels and background pixels using the pixel values of the pixels constituting the surrounding area, and the number of object pixels and the number of background pixels are calculated. It is determined whether or not a condition relating to the feature value according to the ratio is satisfied. Then, using the determination result, it can be accurately determined whether or not the target region is a character region.

  The present invention can be realized in various forms, for example, an image processing server, an image processing system, an image processing method, a computer program for realizing these functions, and a recording in which the computer program is recorded. It can be realized in the form of a medium.

1 is a block diagram illustrating a configuration of a multifunction machine 200 as an image processing apparatus in an embodiment. The flowchart which shows the process step of an image process. The figure which shows an example of the target image G and the edge image E. The figure which shows an example of the determination image BG. The flowchart which shows the process step of an area | region determination process. The figure which shows an example of the histogram showing luminance distribution. The figure which shows an example of the determination table 292. The figure explaining the typical example of the image area | region whose attribute is "character". The figure explaining the typical example of the image area | region whose attribute is "drawing". The figure explaining the typical example of the image area | region whose attribute is "photograph."

A. Example:
A-1. Multi-function machine configuration:
Next, embodiments of the present invention will be described based on examples. FIG. 1 is a block diagram illustrating a configuration of a multifunction machine 200 as an image processing apparatus according to an embodiment.

  The multi-function device 200 includes a CPU 210, an internal storage device 240 including a ROM and a RAM, a printer unit 250 that prints an image by a predetermined method (for example, inkjet or laser), and a photoelectric conversion element (for example, CCD or CMOS). A scanner unit 260 for reading a document using a computer, an operation unit 270 such as a touch panel, a communication unit 280 for data communication with an external device such as the digital camera 300, personal computer 400, USB memory (not shown), and hard disk. And an external storage device 290 such as a drive or an EEPROM.

The internal storage device 240 is provided with a buffer area 241 for temporarily storing various intermediate data generated when the CPU 210 performs processing. The external storage device 290 stores various computer programs 291 for controlling the multifunction device 200, a determination table 292 that is referred to in image processing described later, and image data 293 that is a target of image processing described later. . The computer program 291 can be provided in a form recorded on a computer-readable recording medium. The recording medium here includes a removable recording medium such as a CD-ROM and a USB storage, an internal storage device of a computer such as a ROM and a RAM, and an external storage device connected to the computer such as a hard disk drive. The image data 293 may include image data generated by reading a document using the scanner unit 260, or may include image data acquired from an external device via the communication unit 280. For example, the image data 293 may include image data generated by the digital camera 300 or image data generated by an application program for document creation or image creation installed in the personal computer 400.

  The CPU 210 functions as an image processing unit M100 that executes image processing to be described later by executing the computer program 291. The image processing unit M100 includes a region specifying unit M110, an edge strength calculating unit M120, and a region determining unit M130. The area determination unit M130 includes a binarization processing unit M131, a pixel density calculation unit M132, a color distribution calculation unit M133, a pixel density determination unit M134, a color distribution determination unit M135, and an attribute specifying unit M136. Yes. Processing executed by each of these functional units will be described later.

A-2. Image processing:
The image processing unit M100 executes a series of processes for determining the content of the image represented by the image data using the image data. FIG. 2 is a flowchart showing processing steps of image processing.

  In step S100, the image processing unit M100 selects image data to be processed (target image data). For example, the image processing unit M100 receives a designation instruction for designating target image data from the user via the operation unit 270, and a plurality of image data stored in the external storage device 290 for the target image data designated in the designation instruction. Select from 293.

  FIG. 3 is a diagram illustrating an example of the target image G (FIG. 3A) represented by the target image data and the edge image E (FIG. 3B) represented by the edge image data. The target image G includes a background image Bg1, a first character object Ob2 and a second character object Ob4 that are objects representing characters, a photo object Ob3 that is an object representing a photo, and a drawing that is an object representing drawing. Object Ob5. Here, the drawing includes illustrations, tables, diagrams, patterns, and the like. In the following description, it is assumed that the target image data is bitmap data composed of RGB pixel data. The RGB pixel data is pixel data including three component values of red (R), green (G), and blue (B) (each component value is, for example, a gradation value of 256 gradations). In addition, the component value of pixel data is also called a pixel value. When the target image data is in another data format, the image processing unit M100 performs the following processing after converting the target image data into the bitmap data described above.

In step S200, the area specifying unit M110 of the image processing unit M100 creates edge image data representing the edge image using the target image data. The edge image data is obtained by applying a sobel filter to each component value of a plurality of RGB pixel data constituting the target image data. The edge image data is constituted by a plurality of RGB pixel data respectively corresponding to a plurality of pixels constituting the target image. The RGB pixel data is pixel data that represents the edge intensity of the corresponding pixel of the target image for each of the R, G, and B component values. Specifically, as the predetermined component value S (x, y) of the pixel at the pixel position (x, y) in the edge image, each component value P corresponding to the component value S of nine pixels in the target image is used. And calculated by the following equation (1).
... Formula (1)
Here, as shown in the above formula (1), the nine pixels are the top, bottom, left, and right nines centered on the pixel position (x, y) in the target image corresponding to the pixel position (x, y) in the edge image. It is the pixel at the position. The first term and the second term of the above formula (1) are absolute values of the sum of values obtained by multiplying the pixel values of the pixels at nine positions by the corresponding coefficients. As understood from the above equation (1), the pixel data of the edge image (pixel edge strength) includes the component value (pixel value) of the first pixel and the component value of the second pixel that are close to each other in the target image. It is calculated using the difference between. The creation of the edge image is not limited to the Sobel filter, and various edge detection filters such as a Prewitt filter and a Roberts filter can be used.

  FIG. 3B shows an edge image E based on the target image G shown in FIG. Note that in FIG. 3B, in order to make the drawing easier to see, pixels with low edge strength are shown in white and pixels with high edge strength are shown in black. It can be seen that the edge image E includes the edges Eg2 to Eg5 of the objects Ob2 to Ob5 of the target image G.

  In step S300, the area specifying unit M110 sets a plurality of unit blocks B (FIG. 3B) for the edge image E. The unit block B is a block arranged in a grid pattern with respect to the edge image E. One unit block B has a size in which, for example, the pixels of the edge image E are set as one unit, and the one unit of pixels is arranged in N rows × N columns (N is a predetermined natural number). As will be described later, the unit block B is a unit area that is the minimum unit for specifying an area in the target image. The magnitude of N is an arbitrary design value, but is set to about 10 to 50, for example. Since the edge image E and the target image G have the same size (vertical and horizontal pixels), it can be said that the unit block B is set for the target image G. The area specifying unit M110 sets the determination image BG corresponding to the setting of the unit block.

  FIG. 4 is a diagram illustrating an example of the determination image BG. Each pixel (determination pixel) BP constituting the determination image BG corresponds to each of the plurality of unit blocks B described above. As the pixel value of each pixel of the determination image BG, values representing various types of information (for example, area identification information described later) regarding the corresponding unit block B are used. At the stage where the determination image BG is set, the pixel values of all the pixels of the determination image BG are set to 0 (initialized).

  In step S400, the area specifying unit M110 selects a processing target determination block from the plurality of unit blocks B according to a predetermined selection order. Here, in this embodiment, one determination block is composed of four unit blocks of 2 rows × 2 columns. An example of the selection order will be described. The determination block U11 at the upper left corner in the edge image E is first selected (FIG. 3B). Then, a determination block U12 shifted rightward by one unit block B and a determination block U13 shifted rightward by one unit block B are sequentially selected (FIG. 3B). Then, as a result of selecting groups shifted sequentially by one unit block B in the right direction, when the determination block U1n at the upper right end in the edge image E is selected, next, one unit block B from the determination block U11. Thus, the determination block U21 shifted downward is selected (FIG. 3B). Then, determination blocks that are sequentially shifted by one unit block B in the right direction are selected. This process is repeated until the determination block Unn at the lower right corner (FIG. 3B) is sequentially selected. In this selection method, since one unit block B is included in a plurality of determination blocks, one unit block B can be selected a plurality of times. For example, the unit block B in the second block from the upper end and the second block from the left end shown by hatching in FIG. 3B is selected four times.

In step S500, the edge strength calculation unit M120 of the image processing unit M100 calculates the edge strength (region edge strength) of the determination block to be processed. Specifically, in this embodiment, the edge strength calculation unit M120 uses the determination block (4) as the feature value indicating the region edge strength.
The average values (ERave, EGave, EBave) of the component values (R, G, B) are calculated for all the pixels of the edge image E in one unit block B). Here, as the feature value indicating the region edge strength, instead of the average value of each component value, an intermediate value or mode value of each component value may be used, and at least one component value has a predetermined threshold value. A larger number of pixels may be used.

  In step S600, the area specifying unit M110 determines whether the processing target determination block (four unit blocks B) is a uniform block. A uniform block is a region whose region edge strength is smaller than a predetermined reference. In the embodiment, the region specifying unit M110 uses three characteristic values (ERave, EGave, EBave) indicating the above-described region edge strength of the determination block to be processed as reference values (ETr, ETg) determined for the respective characteristic values. , ETb). As a result of this comparison, when ERave <ETr, EGave <ETg, and EBave <ETb are satisfied, the area specifying unit M110 determines that the processing target determination block (four unit blocks B) is a uniform block. Judge that there is. On the other hand, if at least one of ERave ≧ ETr, EGave ≧ ETg, and EBave ≧ ETb is satisfied, the region specifying unit M110 uniformly determines the determination block (four unit blocks B) to be processed. It is not determined to be a block. If the image processing unit M100 determines that the determination block to be processed is a uniform block (step S600: YES), the image processing unit M100 proceeds to step S700 and does not determine that the determination block to be processed is a uniform block. In this case (step S600: NO), the process of step 700 is omitted, and the process proceeds to step 800.

  In step S700, region specifying unit M110 updates the pixel value of determination image BG based on the determination result in step S600. Specifically, the region specifying unit M110 sets the pixel value of the pixel corresponding to the unit block B determined to be a uniform block in the determination image BG to “1”.

  In step S800, the area specifying unit M110 determines whether all the determination blocks have been selected. If all the determination blocks have not been selected (step S800: NO), the area specifying unit M110 returns to step S400 and selects a new determination block to be processed. When all the determination blocks to be processed are selected (step S800: YES), the area specifying unit M110 moves the process to step S900.

  FIG. 4A shows an example of the determination image BG at the stage of shifting to step S900. Since one unit block B belongs to a plurality of determination blocks, the area specifying unit M110 can make a plurality of determinations (step S600) for one unit block B. The unit block B determined to be a uniform block even once is finally classified as a uniform block, and the unit block B not determined to be a uniform block is finally classified as a non-uniform block. At the stage of shifting to step S900, the pixel value of each pixel of the determination image BG is either “0” or “1”. The pixel value “0” indicates that the unit block B corresponding to the pixel is classified as a non-uniform block. The pixel value “1” indicates that the unit block B corresponding to the pixel is classified as a uniform block.

In step S900, the area specifying unit M110 classifies the unit blocks B into a plurality of groups, and executes a labeling process for attaching an identifier for each group. Specifically, in the determination image BG, the region specifying unit M110 sets a plurality of pixels that have a pixel value “0” and are adjacent to each other, and sets a plurality of pixels that have a pixel value “1” and are adjacent to each other. Same group. Then, the area specifying unit M110 updates the pixel value of the determination image BG so that pixels belonging to the same group have the same pixel value and pixels belonging to different groups have different pixel values. As a result, an image region composed of pixels having the same pixel value is specified in the determination image BG.

  FIG. 4B shows an example of the determination image BG after the labeling process. In the example of FIG. 4B, it can be seen that five image regions PG1 to PG5 are specified. As described above, an image area having the unit block B as a minimum unit is specified in the target image G in correspondence with the image area specified in the determination image BG. As can be understood from the above description, the pixel value of the determination image BG after the labeling process can be said to be an area identifier for identifying the image area specified in the target image G.

  In the example shown in FIGS. 3 and 4, as the image areas of the target image G corresponding to the image areas PG1 to PG5 (FIG. 4B) of the determination image BG, the background image Bg1, the first character object Ob2, the photograph Regions A1 to A5 (FIG. 3A) corresponding to the object Ob3, the second character object Ob4, and the drawing object Ob5 are specified. Note that the image area specified in the target image G is either a non-uniform area or a uniform area. The non-uniform region is a region corresponding to the unit block B classified as a non-uniform block. The uniform region is a region corresponding to the unit block B classified as a uniform block. In the example of FIG. 3A, the image area A1 corresponding to the background image Bg1 is a uniform area, and the image areas A2 to A5 corresponding to the objects Ob2 to Ob5 are non-uniform areas. As described above, non-uniform blocks adjacent to each other are specified as one image region (non-uniform region), and thus the non-uniform region is basically surrounded by the uniform region.

  As can be understood from the above description, the above-described processing from step S200 to step S900 performs region specification for specifying a non-uniform region and a uniform region positioned around (adjacent to) the non-uniform region in the target image G. It is processing.

  In step S1000, the image processing unit M100 selects one non-uniform area from the non-uniform areas (image areas A2 to A5) specified in the target image G as the target area. In step S1100, the region determination unit M130 of the image processing unit M100 executes region determination processing for determining the attribute of the non-uniform region selected as the target region.

  FIG. 5 is a flowchart showing the process steps of the area determination process. In step S1110, the region determination unit M130 identifies a uniform region (hereinafter also simply referred to as a peripheral region) located around the non-uniform region that is the target region. For example, when the non-uniform area that is the target area is the image area A2 (FIG. 3A), the area determination unit M130 specifies the entire image area A1 corresponding to the background image Bg1 as the surrounding area. Instead, the area determination unit M130 may specify a part of the image area A1, for example, a part of the area near the image area A2, which is the target area, as the surrounding area. For example, in the determination image BG in FIG. 4B, a predetermined width (for example, one image area corresponding to the area indicated by the symbol RG <b> 2, i.e., the image area A <b> 1 corresponding to the background image Bg <b> 1 surrounds the target area. An annular region having a width corresponding to the unit block B may be used as the surrounding region.

  In step S1120, the area determination unit M130 calculates the average value (ARave, AGave, ABave) of the color of the surrounding area specified in the target image G. That is, the region determination unit M130 calculates the average value of the component values (pixel values) of the RGB pixel data for all the pixels constituting the surrounding region. Instead of the average value, a mode value or an intermediate value may be adopted.

In step S1130, the binarization processing unit M131 of the region determination unit M130 performs binarization of the target region using the average value (ARave, AGave, ABave). In other words, the binarization processing unit M131 binarizes each pixel constituting the target area into either “0” or “1”, so that each pixel constituting the target area corresponds to “1”. And the background pixel corresponding to “0”. Specifically, when the component values (Ri, Gi, Bi) of the RGB pixel data of the pixel i constituting the target region satisfy all the following expressions (2) to (4), the pixel of the pixel i The value is set to “0”, and when any of the expressions (2) to (4) is not satisfied, the pixel value of the pixel i is set to “1”.
ARave−Th1 <Ri <ARave + Th1 (2)
AGave−Th1 <Gi <AGave + Th1 (3)
ABave−Th1 <Bi <ABave + Th1 (4)

  As understood from the equations (2) to (4), the absolute value of the difference between the average value (average pixel value) of the color of the surrounding area and the component value (pixel value) of the pixel i is smaller than the reference value Th1. In this case, the pixel i is classified as a background pixel. If the absolute value of the difference between the average color value (average pixel value) of the surrounding area and the component value (pixel value) of the pixel i is equal to or greater than the reference value Th1, the pixel i is classified as an object pixel. Is done.

In step S1140, the pixel density calculation unit M132 of the region determination unit M130 calculates the pixel density D of the object pixel. The pixel density D is a value indicating the proportion of object pixels in the target area. The pixel density D is expressed by the following formula (5).
D = No / Na = No / (No + Nb) (5)
Here, No represents the number of object pixels, Na represents the number of all pixels constituting the target area (total number of pixels in the target area), and Nb represents the number of background pixels. As apparent from the fact that the pixel density D can be expressed as 1 / {1+ (Nb / No)}, it can be said to be a feature value corresponding to the ratio between the number No of object pixels and the number Nb of background pixels. .

  In step S1150, the color distribution calculation unit M133 of the region determination unit M130 calculates the luminance distribution of the pixels constituting the target region. In the present embodiment, the color distribution calculation unit M133 calculates the luminance distribution for the object pixel among the pixels constituting the target region.

  FIG. 6 is a diagram illustrating an example of a histogram representing the luminance distribution. The histogram of FIG. 6A is obtained by taking the luminance value Y (256 gradations) on the horizontal axis and plotting the number of pixels having each luminance value on the vertical axis. The color distribution calculation unit M133 generates luminance distribution data represented by a histogram as shown in FIG. That is, the color distribution calculation unit M133 calculates the luminance value Y of all the object pixels, and counts the number of pixels having the luminance value Y for each gradation (for example, 256 gradations) that the luminance value Y can take. To do. The luminance value Y of the pixels constituting the target region is calculated by, for example, the following equation (6) using RGB pixel data (R, G, B). Luminance value Y = ((0.298912 × R) + (0.586611 × G) + (0.114478 × B)) (6)

  In step S1160, the color distribution calculation unit M133 calculates the distribution width W using the calculated luminance distribution (luminance distribution data). In the present embodiment, the distribution width W is a maximum value among the luminance values Y of the pixels having more pixels than the reference value Th2, and a minimum value of the luminance values Y of the pixels having more pixels than the reference value Th2. It is a difference (FIG. 6A). It can be said that the distribution width W is a feature value representing the feature of the luminance value distribution and a feature value indicating the width of the luminance value distribution range.

  In step S1170, the color distribution calculation unit M133 calculates the number of colors C using the calculated luminance distribution. As shown in FIG. 6A, the number of colors C is the number of luminance values Y that more pixels have than the reference value Th2 in this embodiment. If the luminance value Y is different, the color of the pixel is different. Therefore, the number of different luminance values Y (number of types of luminance values Y) represents the number of different colors (number of types of colors). That is, it can be said that the number of colors C is a feature value indicating the number of different luminance values Y.

In step S1180, the region determination unit M130 specifies the attribute of the target region based on the pixel density D, the distribution width W, and the number of colors C described above. For example, the region determination unit M130 determines whether each of the pixel density D, the distribution width W, and the number of colors C described above satisfies a predetermined condition. Specifically, the pixel density determination unit M134 of the region determination unit M130 determines whether the pixel density D of the object pixels in the target region is equal to or higher than the pixel density reference value Dth. Then, the color distribution determination unit M135 of the region determination unit M130 determines whether or not the above-described distribution width W is equal to or greater than the distribution width reference value Wth. Further, the color distribution determination unit M135 determines whether or not the above-described number of colors C is equal to or greater than the number of colors reference value Cth. The attribute specifying unit M136 of the region determination unit M130 uses the determination result regarding the pixel density D by the pixel density determination unit M134, the determination result regarding the distribution width W and the determination result regarding the number of colors C by the color distribution determination unit M135. Then, the attribute of the target area is specified.

  FIG. 7 is a diagram illustrating an example of the determination table 292. The attribute specifying unit M136 refers to the determination table 292 (FIG. 1) stored in the external storage device 290 when specifying the attribute of the target area. As illustrated in FIG. 7, the determination table 292 includes a target region for a combination of a determination result of a condition regarding the pixel density D, a determination result of a condition regarding the distribution width W, and a determination result of the condition regarding the number of colors C. The specific result to be specified as the attribute of is associated. When the attribute of the target area is determined according to the determination table 292, the area determination unit M130 ends the area determination process.

  FIG. 2: In step S1200, the region determination unit M130 determines whether all non-uniform regions have been selected as target regions. If there is a non-uniform region that has not been selected (step S1200: YES), the region determination unit M130 returns to step S1000. When all the non-uniform regions have been selected (step S1200: No), the image processing unit M100 ends the image processing. After this image processing, for example, image quality adjustment processing corresponding to the attribute is performed for each image region for which the attribute is specified. For example, the image area specified as “photo” is subjected to processing for adjusting white balance, brightness, etc., and the image area specified as “character” is subjected to processing for increasing sharpness and contrast. obtain.

  Here, a typical example of each image area having attributes of “character”, “drawing”, and “photo” will be described. FIG. 8 is a diagram for explaining a typical example of an image region whose attribute is “character”. FIG. 8A shows an image area A4 including the second character object Ob4 in the target image G. FIG. 8B shows a binary image BV4 obtained by binarizing the image area A4. FIG. 8C shows a histogram showing the luminance distribution of the object pixel OP in the image area A4.

  The image area A4 includes characters of three colors (FIG. 8A). The histogram of the image area A4 indicates that there are only three types of luminance value pixels corresponding to the three colors (FIG. 8C). As described above, since characters are often expressed by a small number of colors, an image having an attribute of “character” has a smaller number of colors C than an image having an attribute of “photo”, which will be described later. Tend to be. In the example shown in FIG. 8B, the area of the object pixel OP is smaller than the area of the background pixel BP. In this way, since characters are often composed of lines and rarely include a filled area, an image having an attribute of “character” has a proportion of the area of the object in the area of the entire area. The pixel density D of the object pixel OP tends to be smaller than that of an image that is relatively small and has an attribute “photograph” or “drawing”, which will be described later.

  In addition, in an image having the attribute “character”, the difference in luminance between the color of the background pixel BP and the color of the object pixel OP is easily increased from the viewpoint of readability. For example, when the luminance of the color of the background pixel BP is large (for example, a light color such as white), the luminance of the color of the object pixel OP of the character is often small (for example, a dark color such as black). Conversely, when the luminance of the color of the background pixel BP is low, the luminance of the color of the character object pixel OP is often high. As a result, the distribution width W of an image having the attribute “character” is often relatively small (FIG. 8C).

  FIG. 9 is a diagram for explaining a typical example of an image region whose attribute is “drawing”. FIG. 9A shows an image area A5 including the drawing object Ob5 in the target image G. FIG. 9B shows a binary image BV5 obtained by binarizing the image area A5. FIG. 9C shows a histogram showing the luminance distribution of the object pixel OP in the image area A5.

  The image area A5 includes four-color characters (FIG. 9A). The histogram of the image area A5 indicates that there are only four types of luminance value pixels corresponding to the four colors (FIG. 9C). As described above, an image having an attribute “drawing” tends to have a smaller number of colors C than an image having an attribute “photograph” to be described later. In the example shown in FIG. 9B, the area of the object pixel OP is larger than the area of the background pixel BP. Thus, since drawing often includes not only a line but also a filled area, an image having an attribute “drawing” is compared with an image having an attribute “character” as described above. The pixel density D of the pixel OP tends to increase.

  FIG. 10 is a diagram for explaining a typical example of an image area whose attribute is “photograph”. FIG. 10A shows an image area A3 including the photographic object Ob3 in the target image G. FIG. 10B shows a binary image BV3 obtained by binarizing the image area A3. FIG. 10C shows a histogram showing the luminance distribution of the object pixel OP in the image area A3.

  The image area A3 is rich in gradation and includes a large number of colors (FIG. 10A). For this reason, the histogram of the image area A3 indicates that there are pixels of other types of luminance values over a wide range of luminance values (FIG. 10C). As described above, an image having the attribute “photo” tends to have a larger number of colors C and a larger distribution width W than the above-described image having the attribute “character” or “drawing”. In the example shown in FIG. 10B, the area of the object pixel OP is larger than the area of the background pixel BP. As described above, since the region of the photograph is often filled with pixels of various colors, an image having the attribute “photo” is compared with the image having the attribute “character” in the object pixel OP. The pixel density D tends to increase.

As can be seen from the determination table 292, when the attribute of the target area is determined to be "character", there are the following two.
(1-1) Number of colors C <Cth and pixel density D <Dth
(1-2) Distribution width W <Wth, number of colors C ≧ Cth, and pixel density D <Dth
Further, when it is determined that the attribute of the target area is “drawing”, it is one of the following.
(2-1) Number of colors C <Cth and pixel density D ≧ Dth
When it is determined that the attribute of the target area is “photograph”, there are the following two.
(3-1) Distribution width W ≧ Wth and the number of colors C ≧ Cth
(3-2) Distribution width W <Wth, number of colors C ≧ Cth, and pixel density D ≧ Dth

  In this embodiment, in the case shown in (1-1), since the attribute of the target area is determined to be “character”, the above-described number of colors C is relatively small and the pixel density D is relatively small. The attribute of the image area (see FIG. 8) having typical character characteristics can be correctly specified.

  In this embodiment, in the case shown in (2-1), since the attribute of the target region is determined to be “drawing”, the above-described typical case where the number of colors C is relatively small and the pixel density D is relatively large. The attributes of the image area (see FIG. 9) having typical drawing characteristics can be correctly specified.

In the present embodiment, in the case of (3-1), since the attribute of the target region is determined to be “photograph”, the above-described distribution width W is relatively large and the number of colors C is relatively large. The attribute of the image area (see FIG. 10) having typical photographic characteristics can be correctly specified.

  Next, when the distribution width W is relatively small, but the number of colors C is relatively large, it cannot be said that the typical characteristics of “character” and “drawing” are provided in that the number of colors C is relatively large. It cannot be said that the typical feature of “photograph” is provided in that the distribution width W is relatively small. Such a non-typical case will be described.

  When the distribution width W is relatively small, the number of colors C is relatively large, and the pixel density D is relatively small, the distribution width W is relatively small and the pixel density D is relatively small 2 In point, it matches the characteristics of “character”. On the other hand, the point that matches the feature of “photo” is only the point where the number of colors C is relatively large, and the point that matches the feature of “draw” is not seen. In the present embodiment, in the case of (1-2) described above, since the attribute of the target region is determined to be “character”, it is not typical, but based on the number of feature matching points, The attribute of the target area can be specified with high accuracy.

  When the distribution width W is relatively small, the number of colors C is relatively large, and the pixel density D is relatively large, the number of colors C is relatively large and the pixel density D is relatively large 2 In point, it matches the characteristics of “photograph”. In the case of “photograph”, it can be assumed that the luminance distribution width W is relatively small, for example, a dark photo that is underexposed or a bright photo that is overexposed. On the other hand, the point that matches the feature of “character” is only the point that the distribution width W is relatively small, and the point that matches the feature of “draw” is only the point that the pixel density D is relatively large. is there. In the present embodiment, in the case of (3-2) described above, since the attribute of the target region is determined to be “photograph”, it is not typical, but based on the number of feature matching points and the like. The attribute of the target area can be specified with high accuracy.

  As can be understood from the above description, the image processing unit M100 according to the present embodiment relates to the pixel density D of the object pixels in addition to whether or not the conditions regarding the pixel values of the object pixels such as the distribution width W and the number of colors C are satisfied. Determine whether the condition is met. The image processing unit M100 can specify the attribute of the target image with high accuracy using these determination results. Specifically, as described above, when the number of colors C is relatively small, it is specified whether the attribute of the target area is “character” or “drawing” only by the condition relating to the number of colors C. However, it is possible to accurately determine whether the attribute of the target area is “character” or “drawing” by determining the condition regarding the pixel density D. Further, when the distribution width W is relatively small and the number of colors C is relatively large, it is difficult to specify the attributes of the target region only with the conditions relating to the distribution width W and the number of colors C. By determining the conditions regarding the pixel density D, it is possible to accurately determine whether the attribute of the target area is “character” or “photo”.

  Further, the region specifying unit M110 of the image processing unit M100 specifies a non-uniform region as a target region and a uniform region as a peripheral region located around the target region. Then, the binarization processing unit M131 of the image processing unit M100 classifies the plurality of pixels configuring the target region into object pixels and background pixels using the pixel values of the pixels configuring the surrounding region. Therefore, it is possible to correctly classify a plurality of pixels in the target region into object pixels and background pixels using the pixel values of the surrounding image.

More specifically, the area determination unit M130 calculates an average pixel value of a plurality of pixels constituting the surrounding image, that is, an average value (ARave, AGave, ABave) of the color of the surrounding area. Then, the binarization processing unit M131 determines that the absolute value of the difference between the average value of the color of the surrounding area and the component value (Ri, Gi, Bi) of the RGB pixel data of the pixel i constituting the target area is the reference value. If it is smaller than Th1, the pixel i is classified as a background pixel. Then, the binarization processing unit M131 classifies pixels other than the pixels classified as background pixels as object pixels. As a result, the color (pixel value) of the background pixel can be predicted from the color (pixel value) of the surrounding area. Based on the prediction, a plurality of pixels constituting the target area can be converted into the background pixel and the object pixel. Classify. As a result, for example, even when the difference between the background pixel and the object pixel is small, the plurality of pixels constituting the target region can be correctly classified into the background pixel and the object pixel.

  Further, the image processing unit M100 extracts a difference in pixel values between a plurality of pixels constituting the processing target unit block B using an edge extraction filter, and the edge strength (ERave, EGave, EBave)) is calculated. Then, the edge strength calculation unit M120 determines whether or not the unit block B to be processed is a uniform block by comparing the edge strength (ERave, EGave, EBave) with the reference values (ETr, ETg, ETb). doing. As a result, the non-uniform region as the target region and the uniform region as the surrounding region can be appropriately identified.

  Further, the multifunction device 200 is one of the determination result based on the condition regarding the pixel density D, the determination result based on the condition regarding the number of colors C and the distribution width W, and the attribute specification result (“character”, “draw”, “photo”) ) And an external storage device 290 that stores a determination table 292 associated with each other. Then, the attribute specifying unit M136 of the region determining unit M130 can easily specify whether or not the target region is a character region using the determination table 292.

  The area determination unit M130 can specify the attribute of the target area with high accuracy using the condition regarding the distribution width W and the condition regarding the number of colors C. As described above, when the pixel density D is relatively large, it is difficult to specify whether the attribute of the target region is “drawing” or “photograph” only by the condition relating to the pixel density D. The area determination unit M130 of the present embodiment can correctly determine the attribute of the target area by making a determination by adding the conditions of the distribution width W and the number of colors C. When the pixel density D is smaller than the pixel density reference value Dth, the area determination unit M130 basically determines that the attribute of the target area is “character”. Yes. However, if the distribution width W is equal to or greater than the distribution width reference value Wth and the color number C is equal to or greater than the color number reference value Cth, the region determination unit M130 does not depend on the value of the pixel density D but the target region. Is attributed to “photo”. As described above, the region determination unit M130 can specify the attribute of the target region with high accuracy using the conditions regarding the distribution width W and the number of colors C.

  The binarization processing unit M131 in this embodiment is an example of a classification unit in the claims, and the pixel density determination unit M134 in this embodiment is an example of the first determination unit in the claims. The color distribution determination unit M135 is an example of a second determination unit in the claims. The pixel density calculation unit M132 in the present embodiment is an example of the first feature value calculation unit in the claims. The color distribution calculation unit M133 in the present embodiment is an example of the second feature value calculation unit in the claims. .

B. Variation:
The present invention is not limited to the above-described embodiments, and can be implemented in various modes without departing from the scope of the invention.

(1) In the above embodiment, the color distribution calculation unit M133 generates luminance distribution data represented by a histogram as shown in FIG. 6A, but instead of this, FIG. Luminance distribution data represented by a histogram as shown may be generated. In this example, the gradation (for example, 256 gradations) that the luminance value Y can take is assigned to a predetermined number, specifically, for example, 20 luminance classes. One luminance class is assigned a continuous luminance value Y of 12 or 13 gradations. A smaller luminance value Y is assigned as the class number is smaller, and a larger luminance value Y is assigned as the class number is larger. For example, for a luminance class with a class number “1”, a luminance value 0-12, a luminance class with a class number “2”, a luminance value 13-25, and a luminance class with a class number “20” Luminance values 244 to 255 are respectively assigned. Then, for each luminance class, the color distribution calculation unit M133 counts the number of pixels having the luminance value Y belonging to the luminance class, and generates luminance distribution data.

  In this case, as shown in FIG. 6B, the distribution width W may be a value representing the width from the effective minimum class to the effective maximum class using the number of luminance classes. Here, the effective maximum class is a luminance class to which the largest luminance value Y is assigned among luminance classes to which more pixels than the reference value Th5 belong (also referred to as an effective luminance class). The effective minimum class is a luminance class to which the smallest luminance value Y is assigned among luminance classes to which more pixels than the reference value Th5 belong. Specifically, the distribution width W may be expressed as W = Wmax−Wmin + 1 using the class number Wmax of the effective maximum class and the class number Wmin of the effective minimum class. In this case, in the example shown in FIG. 6B, the distribution width W is 19−1 + 1 = 19. Further, the number of colors C may be represented by the number of effective luminance classes. In this case, the number of colors C is 15 in the example shown in FIG. Thus, the distribution width W is a feature value that represents the feature of the distribution of the luminance value Y, and is preferably a feature value that has a correlation with the width of the distribution range of the luminance value. It can also be said that the distribution width W is preferably an evaluation value that can evaluate the width of the distribution range of the luminance values. Similarly, the color number C is a feature value that represents a feature of the distribution of the luminance value Y, and is preferably a feature value that has a correlation with the number of different luminance values (number of types). The number of colors C is preferably an evaluation value that can evaluate the number of different luminance values. In such a case, by setting an appropriate distribution width reference value Wth and color number reference value Cth according to possible values of the distribution width W and the number of colors C, as in the embodiment, the attributes of the target image are set. Can be identified appropriately.

  The distribution width W and the number of colors C are not limited to the feature values for the luminance value Y, but may be feature values for the component values (R, G, B) of the pixel data, and may be an HSB color space or the like. It may be a feature value for brightness. That is, feature values for various values obtained based on pixel values of pixels or values obtained by converting pixel values can be used. In general, the distribution width W and the number of colors C are preferably feature values representing the characteristics of the pixel value distribution of the pixels.

(2) Instead of the pixel density D of the object pixel in the above embodiment, the pixel density BD of the background pixel may be adopted. Since the pixel density D of the object pixel and the pixel density BD of the background pixel are in a relationship of D = 1−BD, if the value of the pixel density reference value Dth and the inequality sign of the determination condition are appropriately set, the pixel density of the background pixel The BD can be used to make substantially the same determination as in the embodiment. Similarly, substantially the same determination as in the embodiment can be performed using a value obtained by dividing the number of object pixels by the number of background pixels or a value obtained by dividing the number of background pixels by the number of object pixels. In general, it is preferable to employ a feature value corresponding to the ratio between the number of object pixels and the number of background pixels.

  Furthermore, the determination condition for specifying the attribute is not limited to the condition relating to the ratio between the number of object pixels and the number of background pixels, such as the condition relating to the pixel density D, but relating to the relationship between the object pixel and the background pixel. Various conditions can be used. For example, the positional relationship between the object pixel and the background pixel in the target region, for example, whether or not the object pixel and the background pixel are alternately present along a certain direction can be employed.

(3) The area determination unit M130 may specify whether the attribute of the target image is “character” using only the pixel density D without using the number of colors C and the distribution width W. As described above, when the pixel density D is relatively small, there is a high possibility that the attribute of the target image is “character”. Therefore, using the pixel density D, the attribute of the target image is “character” with high accuracy. Whether or not there is can be specified.

(4) In the above embodiment, the classification accuracy is improved by classifying (binarizing) the pixels constituting the target area using the average value of the color of the surrounding area, but without using the surrounding area. Such classification may be performed. In this case, it is possible to classify the pixels constituting the target area with a low calculation load without specifying the surrounding area. Further, when there is no surrounding area, for example, even when the attribute of the target area is specified with the entire image as the target area, the attribute of the target area can be specified.

(5) In the above embodiment, the pixels constituting the target image are classified into object pixels and background pixels. Here, the object pixel constitutes an object that is to be an attribute determination target in the target image. Pixels, and background pixels are other pixels. For example, in the case where only a blue object is an attribute determination target, the pixels constituting the target image are classified as object pixels, and the pixels constituting the blue object are classified as object pixels. Pixels constituting the object, pixels constituting the background, noise components, and the like may be classified as background pixels.

(6) In the above-described embodiment, the region determination unit M130 specifies a uniform region as the surrounding region. However, the region determination unit M130 is not limited to the uniform region, and exists along the outer edge of the target region regardless of whether the region is uniform or non-uniform. A region having an arbitrary size may be specified as the surrounding region. In addition, the area specifying unit M110 uses the edge strength of the area to specify the target area (specify the non-uniform area) and determine whether the area is a uniform area or a non-uniform area. Not limited. For example, the target area and the surrounding area may be specified by, for example, receiving an area designation instruction from the user via the operation unit 270. Similarly, the determination of whether the region is uniform or non-uniform may be performed by accepting the determination result determined by the user through the operation unit 270 in the same manner.

(7) The determination table 292 is not limited to a table format, and may be determination information that can be referred to by the area determination unit M130. Further, instead of storing determination information such as the determination table 292 in the external storage device 290, for example, the determination information may be obtained by accessing an external server or the like.

(8) The device that realizes the function of the image processing unit M100 is not limited to the multifunction device 200, and various devices such as computers inside image-related devices such as printers, digital cameras, and scanners, or general-purpose personal computers. Or a computer such as a server. In addition, the image processing unit M100 is not limited to a single casing device, but may be a computer system including a plurality of computers arranged in a plurality of casings (for example, a distributed computer system that realizes so-called cloud computing). The function may be realized. In this case, the entire computer system that realizes the function of the image processing unit M100 corresponds to the image processing apparatus in the claims.

(9) In the above embodiment, a part of the configuration realized by hardware may be replaced with software, and conversely, a part of the configuration realized by software may be replaced by hardware. Also good.

200 ... MFP, 210 ... CPU, 240 ... Internal storage device, 250 ... Printer part, 260 ... Scanner part, 270 ... Operation part, 280 ... Communication part, 290 ... external storage device, 291 ... computer program, 292 ... determination table, M100 ... image processing unit, M
110 ... Area specifying unit, M120 ... Edge strength calculating unit, M130 ... Area determining unit, M13
DESCRIPTION OF SYMBOLS 1 ... Binarization process part, M132 ... Pixel density calculation part, M133 ... Color distribution calculation part, M134 ... Pixel density determination part, M135 ... Color distribution determination part, M136 ... Attribute specific part image area

However, a technique for determining whether or not the target region is a character with higher accuracy has been demanded. In addition, there is a need for a technique for accurately specifying the attributes of the target area.

The main advantage of the present invention is to provide a technique for determining whether or not the target area is a character with higher accuracy. Another advantage of the present invention is to provide a technique for accurately specifying attributes of a target area.

According to the above configuration, the plurality of pixels constituting the target area are classified into object pixels and background pixels using the pixel values of the pixels constituting the surrounding area, and the number of object pixels and the number of background pixels are calculated. It is determined whether or not a condition relating to the feature value according to the ratio is satisfied. Then, using the determination result, it can be accurately determined whether or not the target region is a character region.
Application Example 3 Using image data representing an image, a plurality of pixels constituting a target area in the image are changed into an object pixel constituting an object and a background pixel constituting a background excluding the object. A classifying unit for classifying, a first feature value calculating unit for calculating a first feature value corresponding to a ratio between the number of the object pixels and the number of the background pixels; A second feature value calculation unit that calculates a feature value of 2, a third feature value calculation unit that calculates a third feature value relating to the number of gradation values that more pixels have than the first reference value, A first determination unit that determines whether or not the target region satisfies a first condition related to the first feature value; and whether or not the target region satisfies a second condition related to the second feature value A second determination unit that determines whether or not the target region is A third determination unit that determines whether or not the third condition value relating to the third feature value is satisfied, a first determination result based on the first condition, and a second determination result based on the second condition And a determination information storage unit that stores determination information in which a specific result to be specified as an attribute of the target region is associated with a combination of the third determination result according to the third condition, Using the first determination result, the second determination result, the third determination result, and the determination information, whether or not the target area is the character area, An image processing apparatus comprising: an attribute specifying unit that specifies at least one of whether or not there is a drawing area.

Claims (4)

A classification unit that classifies a plurality of pixels constituting a target area in the image into object pixels constituting an object and background pixels constituting a background excluding the object, using image data representing the image; ,
A first determination unit that determines whether or not the target region satisfies a first condition regarding a relationship between the object pixel and the background pixel;
A second determination unit that determines whether or not the target region satisfies a second condition regarding pixel values of the plurality of pixels;
Using the determination result by the first determination unit and the determination result by the second determination unit, an attribute specifying unit for specifying whether or not the target region is a character region representing a character;
An image processing apparatus comprising: An area specifying unit for specifying a target area in the image and a peripheral area located around the target area using image data representing an image;
A classification unit that classifies a plurality of pixels constituting the target region into object pixels representing an object and background pixels representing a background excluding the object, using pixel values of pixels constituting the surrounding region. ,
A feature value calculation unit that calculates a feature value according to a ratio between the number of the object pixels and the number of the background pixels;
An attribute specifying unit that specifies whether or not the target area is a character area representing a character, based on whether or not the target area satisfies a condition regarding the feature value;
An image processing apparatus comprising: A classification function for classifying a plurality of pixels constituting a target area in the image into object pixels constituting an object and background pixels constituting a background excluding the object, using image data representing the image; ,
A first determination function for determining whether or not the target region satisfies a first condition relating to a relationship between the object pixel and the background pixel;
A second determination function for determining whether or not the target region satisfies a second condition regarding pixel values of the plurality of pixels;
Using the determination result by the first determination unit and the determination result by the second determination unit, an attribute specifying function for specifying whether the target region is a character region representing a character;
An image processing program that causes a computer to realize the above. An area specifying function for specifying a target area in the image and a peripheral area located around the target area using image data representing an image;
A classification function for classifying a plurality of pixels constituting the target area into object pixels representing an object and background pixels representing a background excluding the object, using pixel values of pixels constituting the surrounding area; ,
A feature value calculation function for calculating a feature value according to a ratio between the number of the object pixels and the number of the background pixels;
An attribute specifying function for specifying whether or not the target area is a character area representing a character based on whether or not the target area satisfies a condition regarding the feature value;
An image processing program for realizing a computer.