JP4716255B2 - Image processing apparatus, image processing method, and program thereof - Google Patents

Description

  The present invention relates to an image processing apparatus that enlarges an image.

  The image enlargement process is one of the basic processes for a system that performs image editing, filing, display, printing, and the like. In recent years, with the widespread use of image data mainly for display at display resolutions such as images on the Internet homepage and digital video, these low-resolution images are frequently printed by high-resolution printers. It has been broken. At the time of printing with this printer, it is desired to obtain a high-quality output result, and the importance of the high-quality enlargement process is increasing.

  Typical existing methods for enlarging images expressed in multi-tones including color (hereinafter referred to as multi-valued images) include nearest neighbor method, linear interpolation method, cubic convolution method, etc. There is. The nearest neighbor method is a method in which the pixel value of the pixel having the closest distance is used as each enlarged pixel value when the pixel is reversely mapped on the original image. Since this method has a small amount of calculation, it can be processed at high speed. However, since one pixel of the original image is directly expanded into a rectangular shape, if the difference between the pixel values of adjacent pixels is small, the degree of image quality degradation is small and has almost no effect. When the jagged edges and edge portions are conspicuous or the magnification is large, the degree of image quality deterioration is large, such as a mosaic image.

  In the linear interpolation method, it is assumed that pixel values between pixels change linearly, and pixel values of four pixels in the vicinity of a point obtained by inversely mapping the enlarged pixel are linearly interpolated to obtain a pixel value. is there. Although this method is heavier than the nearest neighbor method, the amount of calculation is relatively small and jaggies are not easily generated. On the other hand, there is a drawback that the entire image becomes blurred, centering on the edge portion that does not apply to the assumption that it changes linearly.

  The cubic convolution method defines an interpolation function that approximates a sinc function (sin (x) / x) based on the sampling theorem, and 16 pixels in the vicinity of a point obtained by inversely mapping the enlarged pixel (X and Y directions) This is a method for obtaining an enlarged pixel value by a convolution operation of each of 4 pixels) and the approximate interpolation function. This method has relatively good image quality compared to the above two methods, but has a drawback that the reference range is large and the amount of calculation is large. In addition, since the high frequency band is emphasized, there are disadvantages such as light jaggies occurring at the edge and noise components being emphasized.

As an attempt to solve the image quality problem of these enlarged images, for example, new systems such as Patent Document 1 and Patent Document 2 have been proposed.
The technique described in Patent Document 1 performs edge detection using an edge detection filter as a method for detecting the degree of change in an original image, and defines edge pixels based on the edge detection result. When this method is determined to be an edge pixel, enlargement is performed by the M-cubic method in which the cubic function shape of the cubic convolution method is adjusted. Otherwise, enlargement is performed by the nearest neighbor method. . However, since the edge portion having a large degree of change is performed by the M-cubic method adjusted so that the edge is emphasized rather than the cubic function shape of the cubic convolution method, jaggies are generated at the edge portion or the periphery of the edge portion. There is a disadvantage that the overshoot or undershoot increases.

  The technique described in Patent Document 2 is a technique for restoring and enlarging high-frequency components while repeatedly performing DCT forward and reverse transformations. Furthermore, in this method, in order to suppress ringing that occurs in the flat part, in the flat part region, an image obtained by interpolation enlargement of the original image by the bilinear method or cubic method is used as a reference image, and an error from the corresponding region of the reference image is detected. Is performed to limit the deviation to an allowable deviation. However, this patent is based on an iterative method of repeatedly performing DCT forward and reverse transformations, and has a large amount of computation such as a reference process for suppressing ringing in a flat portion that occupies most of the image.

  Further, as an attempt to suppress undershoot and overshoot at the edge portion of the image, the method of Patent Document 3 has been proposed. The technique described in Patent Document 3 uses the variance value as an index indicating the edge portion, and corrects the enhancement coefficient so that the enhancement coefficient becomes smaller at the edge portion where the variance value is larger than a certain threshold value. By doing so, excessive emphasis (occurrence of undershoot and overshoot) is suppressed. However, when this method is used for image enlargement processing, overshoot and undershoot can be suppressed to some extent, but blurring of the edge portion cannot be prevented.

JP 2000-188681 A Japanese Patent No. 3404138 Japanese Patent Laid-Open No. 9-93439

  The present invention has been made from the above-described background, and an object thereof is to provide an image processing apparatus that enlarges an image with high image quality.

  In order to achieve the above object, an image processing apparatus according to the present invention is an image processing apparatus that performs an enlargement process of an input image, and calculates a feature amount of an image area having a predetermined size including a target pixel. An enlarged image area generating means for generating an enlarged image area corresponding to the image area based on the feature quantity of the image area calculated by the area feature quantity calculating means, and the enlarged image area generating means; Applies a different enlargement technique to generate an enlarged image of the input image, and an image region that satisfies a predetermined condition for the feature amount calculated by the region feature amount calculation unit is superimposed. Based on the degree, the generation value of the enlarged image area generated by the enlarged image area generation unit is corrected, and based on the enlarged image area and the enlarged image generated by the enlargement processing unit. Te, and an enlarged image generating means for generating an enlarged image for the input image.

Preferably, the region feature amount calculating means calculates a gradation change amount in the image region based on each pixel value in the image region, and sets the gradation change amount as a feature amount of the image region.
Preferably, the region feature amount calculating means calculates a gradation change direction in the image region based on each pixel value in the image region, and sets the gradation change direction as a feature amount of the image region.

Preferably, the area feature quantity calculating means calculates a gradation change in the image area based on each pixel value in the image area, and when the gradation change corresponds to a predetermined pixel value pattern, A pixel value pattern is selected as a feature amount of the image area.
Preferably, the area feature value calculating unit calculates a feature value for each color component in the color space, selects one color component based on the calculated feature value, and calculates the feature value calculated for the color component. Is the feature quantity of the image area.

Preferably, the enlarged image region generation unit is configured to detect the feature amount of the image region and the neighborhood region of the image region when the feature amount of the image region calculated by the region feature amount calculation unit satisfies a predetermined condition. An enlarged image area is generated by using the feature amount and the pixel value in the neighboring area.
Preferably, the enlarged image area generation unit performs edge enhancement processing for enhancing gradation change according to the enlargement ratio, corrects the pixel value in the image area, and uses the corrected pixel value to enlarge the enlarged image. Create a region.

Preferably, the enlarged image region generation unit selects a pixel value in a neighborhood region of the image region according to the gradation change direction calculated by the region feature amount calculation unit, and uses the selected pixel value. To generate an enlarged image area.
Preferably, the enlarged image region generating unit generates an enlarged image region using a predetermined arithmetic expression corresponding to the feature amount calculated by the region feature amount calculating unit.

Preferably, the enlargement processing unit applies an enlargement method based on at least a nearest neighbor interpolation method.
Preferably, the enlargement processing unit applies an enlargement method based on at least a linear interpolation method.
Preferably, the enlargement processing unit generates an enlarged image for each input image or for every several lines in the input image.

Preferably, the enlarged image generation unit arranges the enlarged image region generated by the enlarged image region generation unit at a corresponding position on the enlarged image generated by the enlargement processing unit.
Preferably, the enlarged image generation unit uses a pixel value at a corresponding position on the enlarged image generated by the enlargement processing unit, using the generation value of the enlarged image region generated by the enlarged image region generation unit. To correct.

Preferably, the enlarged image generation unit includes a generation value of the enlarged image region generated by the enlarged image region generation unit and a pixel value at a corresponding position on the enlarged image generated by the enlargement processing unit. This pixel value is corrected using a value obtained by linearly summing at a predetermined ratio.
Preferably, the enlarged image generation means, when the feature amounts of the image areas calculated by the area feature quantity calculation means satisfy a predetermined condition in a plurality of image areas, and these image areas overlap each other, The ratio of the linear sum is determined based on the degree of overlapping of the image areas.

Preferably, the enlarged image generation unit is configured to select an edge portion of the input image based on the feature amount of the image region calculated by the region feature amount calculation unit and the feature amount of the image region superimposed on the image region. Edge periphery is determined.
Preferably, the enlarged image generation means, when the feature amounts of the image areas calculated by the area feature quantity calculation means satisfy a predetermined condition in a plurality of image areas, and these image areas overlap each other, An image portion in which the degree of image region superposition is equal to or greater than a predetermined value is determined as an edge portion.
Preferably, the enlarged image generation means, when the feature amounts of the image areas calculated by the area feature quantity calculation means satisfy a predetermined condition in a plurality of image areas, and these image areas overlap each other, An image portion in which the degree of image region superposition is equal to or less than a predetermined value is determined as an edge peripheral portion.

Preferably, the enlarged image generation means generates the generated value of the image portion corresponding to the edge portion in the enlarged image area generated by the enlarged image area generation means, and the enlarged image generated by the enlargement processing means. Placed in the corresponding position.
Preferably, the enlarged image generation means is an enlarged image area generated by the enlarged image area generation means, and a generation value of an image portion corresponding to an edge peripheral portion and an enlargement generated by the enlargement processing means. This pixel value is corrected using the pixel value at the corresponding position in the image.
Further preferably, the enlarged image generating means is an enlarged image area generated by the enlarged image area generating means, and an image portion that does not correspond to either an edge portion or an edge peripheral portion is processed by the enlargement processing means. The pixel value at the corresponding position on the generated enlarged image is used.

  Further, the edge detection measure according to the present invention is an edge detection device that detects an edge of an input image, and includes a region feature amount calculation unit that calculates a feature amount of an image region having a predetermined size including a target pixel, And determining means for determining an edge portion and an edge peripheral portion of the input image based on the feature amount of the image region calculated by the region feature amount calculating unit and the feature amount of the image region superimposed on the image region.

Preferably, the determination unit includes a plurality of image regions in which the feature amounts of the image regions calculated by the region feature amount calculation unit satisfy a predetermined condition and these image regions overlap each other. An image part in which the degree of superimposing is equal to or greater than a predetermined value is determined as an edge part.
Preferably, the determination unit includes a plurality of image regions in which the feature amounts of the image regions calculated by the region feature amount calculation unit satisfy a predetermined condition and these image regions overlap each other. An image portion in which the degree of superimposing is equal to or less than a predetermined value is determined as an edge peripheral portion.

  An image processing method according to the present invention is an image processing method that performs an enlargement process of an input image, calculates a feature amount of an image area having a predetermined size including a target pixel, and calculates the calculated image area. Based on the feature amount, an enlarged image region corresponding to the image region is generated, an enlarged method different from the generation is applied to generate an enlarged image of the input image, and the calculated feature amount is a predetermined condition. When an image area that satisfies the condition is superimposed, the generation value of the generated enlarged image area is corrected based on the degree of superimposition, and the input image is corrected based on the enlarged image area and the generated enlarged image. An enlarged image is generated for.

  An edge detection method according to the present invention is an edge detection method for detecting an edge of an input image, wherein a feature amount of an image area having a predetermined size including a target pixel is calculated, and the calculated image area Based on the feature amount and the feature amount of the image region superimposed on the image region, the edge portion and the edge peripheral portion of the input image are determined.

  A first program according to the present invention includes an area feature amount calculating step of calculating a feature amount of an image area having a predetermined size including a target pixel in an image processing apparatus that includes a computer and performs an enlargement process of an input image. An enlarged image region generating step for generating an enlarged image region corresponding to the calculated image region based on the calculated feature amount of the image region, and applying an enlargement method different from the generation to enlarge the input image When an enlargement processing step for generating an image and an image region where the calculated feature amount satisfies a predetermined condition are superimposed, the generation value of the generated enlarged image region is corrected based on the degree of overlap, An enlarged image generating step for generating an enlarged image for the input image based on the enlarged image area and the generated enlarged image is performed by the computer of the image processing apparatus. To be executed in.

  Furthermore, a second program according to the present invention is a region feature amount calculation for calculating a feature amount of an image region having a predetermined size including a target pixel in an edge detection apparatus that detects an edge of an input image including a computer. The edge detecting device comprising: a step of determining an edge portion and an edge peripheral portion of the input image based on the calculated feature amount of the image region and the feature amount of the image region superimposed on the image region. To run on a computer.

  According to the image processing apparatus of the present invention, a high-quality enlarged image can be obtained at high speed with a light processing load.

[Hardware configuration]
A hardware configuration of the image processing apparatus 2 in the present embodiment will be described.
FIG. 1 is a diagram illustrating a hardware configuration of an image processing apparatus 2 to which an image processing method according to the present invention is applied, centering on a control apparatus 20.
As illustrated in FIG. 1, the image processing apparatus 2 includes a control device 20 including a CPU 202 and a memory 204, a communication device 22, a storage device 24 such as an HDD / CD device, an LCD display device or a CRT display device, and a keyboard. A user interface device (UI device) 26 including a touch panel and the like is included.
The image processing apparatus 2 is, for example, a general-purpose computer in which an image enlargement program 4 (described later with reference to FIG. 2) is installed. The image processing apparatus 2 acquires image data via the communication device 22 or the storage device 24. Enlarge the image data according to the output resolution. For example, the image processing apparatus 2 converts the resolution to 600 dpi or 2400 dpi when outputting image data to the printer apparatus 10, and 75 dpi or the like when outputting image data to the UI apparatus 26. Convert to the resolution.

[Image enlargement program]
FIG. 2 is a diagram showing a functional configuration of the image enlargement program 4 which is executed by the control device 20 (FIG. 1) and realizes the image processing method according to the present invention.
As illustrated in FIG. 2, the image enlargement program 4 includes a storage unit 400, an image block setting unit 410, an image block feature amount calculation unit 420, a high-quality image block generation unit 430, a high-speed enlargement processing unit 440, and an enlarged image generation unit 450. Have The image block feature amount calculation unit 420 includes an edge strength calculation unit 422, an edge direction estimation unit 424, and an edge pattern selection unit 426. The enlarged image generation unit 450 includes a block superimposition degree determination unit 452 and an output value correction unit. 454.
Note that all or part of the image enlargement program 4 may be realized by hardware such as an ASIC.

  The storage unit 400 temporarily stores image data until the image data is enlarged, and temporarily stores the enlarged image data that has undergone resolution conversion or enlargement processing until it is output to an output device (not shown). The image data is data described in an image format (for example, BMP, TIFF, PNG, etc.) that can be processed by the image processing apparatus 2, and is captured by a digital camera (not shown) or a scanner (not shown). Or image data created by an application program that is created or edited in a personal computer (image processing apparatus 2 or the like). Enlarged image data (image data after being enlarged) is also data in the same image format.

  The image block setting unit 410 sets a predetermined image block size required in the processing of the image block feature amount calculation unit 420 and the high quality image block generation unit 430, respectively. In addition, the image block setting unit 410 sequentially cuts out image blocks having a set block size from the input image data stored in the storage unit 400 (for example, in raster scan order), and converts each image block having the block size into an image. The block feature value calculation unit 420 and the high-quality image block generation unit 430 are each output.

  The image block feature amount calculation unit 420 calculates the image feature amount in the attention area, which is at least a part of the image blocks sequentially input from the image block setting section 410, in the image block including the attention area or the peripheral portion of the attention area. Calculation is performed based on each pixel value. The image feature amount is, for example, edge strength (gradation change amount) or edge angle (direction of gradation change) of the attention area. However, the image feature amount is not limited to these. For example, the image block feature amount calculation unit 420 calculates an average value of each pixel value of the attention area, and uses a value (for example, standard deviation or variance) representing a variation of each pixel value of the attention area with respect to the average value as the image feature amount. May be calculated as

In addition, the image block feature quantity calculation unit 420 compares the image feature quantity calculated for each attention area and the reference value, and performs the attention area separation. By distinguishing the attention area, an image block having a characteristic (for example, an image block including an edge) and an image block having a small feature (for example, an image block having a small change in pixel value) are separated. More specifically, the image block feature value calculation unit 420 determines an image block whose calculated image feature value is equal to or greater than a reference value as a feature block, and an image block whose calculated image feature value is less than the reference value. Are determined as non-characteristic blocks.
In this example, the image block feature amount calculation unit 420 calculates the edge strength (numerical value) of an image block as one of the image feature amounts, and an image block whose calculated edge strength is a reference value or more is a feature block. The image block having the edge strength less than the reference value is determined to be a non-feature block. Note that the feature quantity used for dividing the feature block is not limited to the edge strength, and is, for example, a standard deviation, a variance, or other feature quantity that is a value representing variation of each pixel value of the attention area with respect to the average value of the attention area. May be.

  The high image quality image block generation unit 430 generates an enlarged image block corresponding to the attention area using the image feature amount calculated by the image block feature amount calculation unit 420. For the enlargement processing in the high-quality image block generation unit 430, it is desirable to apply an enlargement method that retains the features included in the region of interest. In addition, the high-quality image block generation unit 430 performs the enlargement process only on the image block (that is, the feature block) in which the feature amount calculated by the image block feature amount calculation unit 420 is greater than or equal to the reference value.

  The high-speed enlargement processing unit 440 enlarges the input image data stored in the storage unit 400. The high-speed enlargement processing unit 440 applies an enlargement algorithm that has a smaller processing load than the enlargement process performed by the high-quality image block generation unit 430. Specifically, the high-speed enlargement processing unit 440 enlarges the input image data by applying, for example, a nearest neighbor interpolation enlargement method or a linear interpolation enlargement method. Also, the high-speed enlargement processing unit 440 performs enlargement processing in units of input images or in units of several lines of input images, rather than processing for each image block.

  The enlarged image generation unit 450 uses the enlarged image block enlarged by the high-quality image block generation unit 430 and the enlarged image enlarged by the high-speed enlargement processing unit 440 to generate an enlarged image for the original image. More specifically, when the image block (feature block) in which the image feature amount calculated by the image block feature amount calculation unit 420 is equal to or greater than the reference value overlaps, the enlarged image generation unit 450 follows the number of overlaps. By correcting the pixel value of each of the enlarged image blocks enlarged by the high-quality image block generation unit 430 based on the pixel value at the position corresponding to the enlarged image enlarged by the high-speed enlargement processing unit 440, the enlarged image is Generate.

[Image block feature value calculation unit]
Next, the image block feature value calculation unit 420 will be described in more detail. A case where the attention area is a 2 × 2 pixel size block and the peripheral area including the attention area is a 4 × 4 pixel size block will be described as a specific example.
The edge strength calculation unit 422 calculates the edge strength G of the attention area in the image block cut out by the image block setting unit 410 using the following equation (1).

gx = (a + c−b−d) / 2
gy = (a + b−c−d) / 2
G = gx × gx + gy × gy (1)

In Expression (1), a, b, c, and d are pixel values of each pixel in the attention area as illustrated in FIG. gx represents the amount of change in the pixel value in the main scanning direction (left and right direction in FIG. 6), and gy represents the amount of change in the pixel value in the sub scanning direction (up and down direction in FIG. 6).
Note that the edge strength may be calculated by the following formula (2) or the like instead of the value calculated by the above formula (1) alone.
G = | gx | + | gy | (2)
That is, the edge strength G may be calculated as the sum of the absolute value of gx and the absolute value of gy.

Further, the edge strength calculation unit 422 compares the calculated edge strength G with a reference value (predetermined threshold value Th) to determine whether the attention area is a feature block or a non-feature block, The determination result is output to the high-quality image block generation unit 430 and the like.
When the edge strength G of the attention area is smaller than the threshold Th, the edge direction estimation unit 424 and the edge pattern selection unit 426 do not perform processing on the attention area.

FIG. 3 illustrates an area determined as a feature block by the edge strength calculation unit 422 by comparing the edge strength G with a reference value.
FIG. 4 illustrates feature blocks that overlap one another.
As illustrated in FIG. 3, the feature blocks are concentrated near the edges in the original image. Further, when the image block feature amount calculation unit 420 calculates the image feature amount of the attention area while shifting the pixel feature amount by one pixel in the main scanning direction and the sub scanning direction, as illustrated in FIG. 4, in the vicinity of the edge in the original image. The feature blocks overlap each other.

FIG. 5 illustrates the number of overlapping feature blocks (block superposition degree).
As illustrated in FIG. 5, the range of the block superimposition degree is, for example, 1 to 4, and the edge strength calculation unit 422 also calculates the block superposition degree, and the calculation result is displayed as the high-quality image block generation unit 430 and the enlarged image. The data is output to the generation unit 450.
When the block superposition degree is calculated, the edge region and the edge peripheral region are distinguished in the vicinity of the edge in the original image. More specifically, a region with a low block superposition degree is distinguished from an edge peripheral region, and a region with a high block superposition degree is distinguished as an edge region.

The edge direction estimation unit 424 will be described.
FIG. 6 is an explanatory diagram of a specific example of the attention area and the peripheral area, and an example of the edge direction of the attention area. FIG. 6A illustrates the attention area and the peripheral area, and FIG. It is a figure which illustrates the edge direction estimated about this attention area.
As illustrated in FIG. 6A, the attention area (image area) has a 2 × 2 rectangular area (two in the main scanning direction and two in the sub scanning direction), and the peripheral area is 4 × 4. It has rectangular areas (four each in the main scanning direction and the sub-scanning direction). Each rectangle corresponds to a pixel, and each numeral in the rectangle indicates a pixel value. That is, the attention area is the pixel {a, b, c, d} = {15, 104, 86, 203} near the center.
Hereinafter, the edge direction estimation processing by the edge direction estimation unit 424 will be described using the attention area illustrated in FIG. 6A as a specific example.

FIG. 7 is a flowchart showing edge direction estimation processing (S10) by the edge direction estimation unit 424.
FIG. 8 is a diagram illustrating a reference region selected in the process of S102 of the edge direction estimation process (S10). The hatched portion in FIG. 8 corresponds to the region of interest shown in FIG.
As shown in FIG. 7, in step 100 (S100), the edge direction estimation unit 424 calculates the edge angle Θ of the attention area exemplified in FIG. 6A by the following equation (3).
Θ = arctan (gy / gx) (3)
In FIG. 6 (A), the pixel value of the attention area is {a, b, c, d} = {15, 104, 86, 203}.
gx = −103
gy = −85
And substituting these into equation (3),
Θ = -140.5 °
It becomes.
The direction of the edge angle Θ corresponds to the direction of the broken line shown in FIG.

  Further, the edge direction estimation unit 424 determines which of the angle ranges in the directions (eight directions) divided every 22.5 ° is the calculated edge angle Θ. In this example, the angle range around the edge angle Θ of 0 ° or ± 180 ° is “direction 0”, the angle range around 22.5 ° or −157.5 ° is “direction 1”, An angle range centered on 45 ° or −135 ° is “direction 2”, an angle range centered on 67.5 ° or −112.5 ° is “direction 3”, and centered on 90 ° or −90 °. An angle range centered on 112.5 ° or −67.5 ° is “direction 5”, and an angle range centered on 135 ° or −45 ° is “direction 6”. ", And an angle range centered on 157.5 ° or -22.5 ° is" direction 7 ". These angle ranges are ± 11.25 ° from their respective centers. Since the edge angle Θ (= −140.5 °) in the above specific example is included in the range of −135 ° ± 11.25 °, the edge angle is “direction 2”.

In step 102 (S102), the edge direction estimation unit 424 estimates the edge direction from the peripheral area (outside the thick line frame) shown in FIG. 6A according to the calculated edge angle Θ of the attention area. Select the reference area to be used for. More specifically, the edge direction estimation unit 424 selects the reference area so as to include a pixel that may be adjacent to the attention area in the direction of the calculated edge angle Θ.
For example, the edge direction estimation unit 424 selects the reference area (two areas surrounded by a thick line) illustrated in FIG. 8A when the edge angle calculated for the attention area is included in “direction 0”. When the calculated edge angle is included in “direction 4”, the reference area (two areas surrounded by a thick line) illustrated in FIG. 8B is selected, and the calculated edge angle is other than the above. Are included in the directions (directions 1 to 3 and directions 5 to 7), reference areas (four areas surrounded by thick lines) illustrated in FIG. 8C are selected. In the specific example shown in FIG. 6, since the direction of the edge angle is “direction 2”, the four reference areas shown in FIG. 8C are candidates for selection.
Note that the reference region is not limited to the region illustrated in FIG. 8. For example, in the case of FIG. 8C, the number of reference regions may be eight or may be set according to each direction.

In step 104 (S104), the edge direction estimation unit 424 calculates the edge angle Θ for each of the selected reference regions according to the equations (1) and (3), as in the process of S100.
In step 106 (S106), the edge direction estimation unit 424 compares the edge angle calculated for each reference area with the edge angle calculated for the attention area, and the difference between these is set in advance. It is determined whether it is smaller than Θth. If the edge angle difference is smaller than the threshold Θth, the edge direction estimation unit 424 determines that the edge angle of the reference region is an appropriate edge angle, and proceeds to the processing of S108. The edge angle difference is equal to or greater than the threshold Θth. If it is, it is determined that the edge angle of the reference region is not an appropriate edge angle, and the process proceeds to S110.

In step 108 (S108), the edge direction estimation unit 424 increments the angle reference number. That is, the edge direction estimation unit 424 increments the angle reference number only when it is determined that the edge angle calculated for the reference region is an appropriate edge angle.
The angle reference number is a variable for counting the reference number of the edge angle, and is initialized to “angle reference number 1” for each region of interest.

In step 110 (S110), the edge direction estimation unit 424 determines whether or not the edge angles have been calculated for all the selected reference areas. In other cases, the process returns to S104 to calculate the edge angle for the next reference area.
In step 112 (S112), the edge direction estimation unit 424 calculates the sum of the edge angle of the attention area and the edge angle of the reference area determined as an appropriate edge angle, and calculates the sum of the calculated edge angles as an angle. The average edge angle divided by the reference number is set as the estimated edge direction of the attention area.

  In the specific example shown in FIG. 6, the edge angle ΘU is ΘU = -149.4 ° from the upper reference region {86, 203, 171, 211}, and the edge angle ΘL is the left reference region { ΘL = −131.2 ° from 10, 15, 20, 86}, and the edge angle ΘD becomes ΘD = −175.2 ° from the lower reference region {1,102,15,104}, and the edge angle ΘR is From the reference region {104, 215, 203, 219} on the right side, ΘR = −141.0 °. The edge angle Θ = −140.5 ° of the attention area is compared with the edge angle of each reference area, and the number of reference areas whose difference is smaller than the threshold Θth is counted as the angle reference number.

  FIG. 9 is an explanatory diagram of an example of the estimated edge direction in the region of interest shown in FIG. For example, in the above-described specific example, if the difference between the edge angle of the attention area for all the reference areas is smaller than the threshold Θth, the sum of the edge angles obtained from the attention area and the four reference areas is −737.3 °. By dividing by the angle reference number 5, the average edge angle is determined to be -147.5 °. Also in this case, the edge direction estimation unit 434 determines whether the average edge angle is included in any of the eight directions, for example, in the same manner as the edge direction of the region of interest described above. In this example, since the average edge angle is −147.5 °, it is included in “direction 1”, and “direction 1” is the estimated edge direction.

In the present embodiment, a grayscale image that is one color element per pixel is described as a specific example, but the present invention is not limited to this. For example, when a color image in the RGB color space of three color elements per pixel is input, the color space data selected by the strength of the edge strength Gr, Gg, Gb in the data of each color component The edge direction estimation process may be performed. More specifically, the image block feature value calculation unit 420 calculates the edge strength for each color component, selects the color component that maximizes the calculated edge strengths Gr, Gg, and Gb. The feature amount is calculated only for the color component. In this way, it is possible to suppress deterioration in image quality such as color shift at the edge of enlarged image data in a color image.
In this example, for the attention area and the reference area, the edge angle calculated by the expression (1) is classified into any one of the eight directions, but the classification is not limited to the eight directions, and the edge has higher accuracy. If a direction is required, the classification may be done for a larger number of angular regions, such as 12 directions (every 15.0 °), 16 directions (every 12.25 °).

The edge pattern selection unit 426 will be described.
FIG. 10 is a diagram illustrating an edge pattern table used by the edge pattern selection unit 426.
As illustrated in FIG. 10, the edge pattern selection unit 426 includes an edge pattern table in which the estimated edge direction and the edge pattern are associated with each other. In the edge pattern table, one or more edge patterns are registered for each estimated edge direction (for example, eight directions) corresponding to the pattern size of the region of interest.
The edge pattern selection unit 426 refers to the edge pattern table and selects an edge pattern corresponding to the estimated edge direction estimated for each region of interest by the edge direction estimation unit 424.
In this example, as illustrated in FIG. 9, since the estimated edge direction with respect to the region of interest is estimated to be “direction 1” by the edge direction estimating unit 424, the edge pattern selecting unit 426 According to (direction 1), four edge patterns from “pattern 0” to “pattern 3” corresponding to direction 1 are selected from the edge pattern table shown in FIG. Candidate.

The edge pattern selection unit 426 selects one edge pattern from one or more edge patterns that are edge pattern candidates based on the pixel value of the region of interest. A specific method for selecting an edge pattern will be described with reference to FIG.
FIG. 11 is a diagram for explaining a method of selecting an edge pattern corresponding to the attention area shown in FIG.
In this example, since the estimated edge direction is “direction 1”, four edge patterns from “pattern 0” to “pattern 3” are selected as the edge pattern candidates as illustrated in FIG. 11A. Has been. These edge patterns are expressed as bit patterns as illustrated in FIG. Specifically, the edge pattern is formed into a bit pattern with the white portion being 0 and the others being 1, and bit patterns from “bit pattern 0” to “bit pattern 3” are generated. Note that these bit patterns may be registered in advance in the edge pattern table shown in FIG. 10 as bit tables.

  The edge pattern selection unit 426 determines a bit pattern corresponding to the attention area. Specifically, the edge pattern selection unit 426 calculates an average pixel value in the attention area according to the following equation (4), subtracts the average value from each pixel value in the attention area, and uses the sign to pay attention. The pixel value pattern of the region is used.

Mean = (a + b + c + d) / 4
a_sign = a-Mean
b_sign = b-Mean
c_sign = c-Mean
d_sign = d-Mean (4)

In this example, as shown in FIG. 11C, Mean = (15 + 104 + 86 + 203) / 4 = 102, a_sign = −87, b_sign = 2, c_sign = −16, and d_sign = 101. Therefore, the edge pattern selection unit 426 determines these positive and negative signs, and generates a bit pattern (1010) of the region of interest as shown in FIG.
The edge pattern selection unit 426 performs pattern matching between the bit pattern corresponding to the edge pattern candidate illustrated in FIG. 11B and the bit pattern of the attention area illustrated in FIG. Is determined as the selection pattern. The selected edge pattern is applied to enlarged image block generation processing in a high-quality image block generation unit 430 described later.

  Note that the edge pattern is not limited to the edge pattern illustrated in FIG. 10. For example, the edge pattern selection unit 426 switches the edge pattern table according to the type of input image data and applies a different edge pattern. Also good. The edge pattern selection unit 426 may increase or decrease the number of edge pattern candidates at each angle.

[High-quality image block generator]
Next, the high-quality image block generation unit 430 will be described in more detail.
The high-quality image block generation unit 430 performs the edge pattern and estimation for the attention area obtained by the image block feature amount calculation unit 420 with respect to the attention area determined by the image block feature amount calculation unit 420 as the feature block. Enlarged image block generation processing is performed based on the edge direction.
First, the high-quality image block generation unit 430 uses the size and coefficient emphasis kernel according to the enlargement magnification of the enlargement process, and the image of the attention area and the surrounding area in the image block cut out by the image block setting unit 410 Emphasize the contrast of data.

FIG. 12 is a diagram illustrating an enhancement kernel 432 (edge enhancement kernel) used by the high-quality image block generation unit 430.
As illustrated in FIG. 12, the first enhancement kernel 432a enhances contrast using the weighting factors “1.60” and “−0.15”, and the second enhancement kernel 432b includes the weighting factor “1”. .20 ”and“ −0.05 ”to enhance contrast. These enhancement kernels are associated with the enlargement magnification of the enlargement process already performed on the target image, and refer to pixel values at different positions using different weighting coefficients.

As illustrated in FIG. 12A, the first enhancement kernel 432a refers to the pixel a, the right pixel b, the pixel c, and the pixel d of the pixel of interest P by referring to the pixels of these pixels. Each value is multiplied by a weighting coefficient (−0.15) and added to the pixel value of the pixel of interest P multiplied by the weighting coefficient (1.60), and the sum is set as the pixel value of the pixel of interest P.
The second enhancement kernel 432b is applied to an image having a larger magnification than the first enhancement kernel 432a, and as illustrated in FIG. 12B, the lower pixel a that is separated by one pixel from the target pixel P. , The right pixel b, the upper pixel c, and the left pixel d, the pixel value of these pixels is multiplied by a weighting coefficient (−0.05), and the pixel of interest multiplied by the weighting coefficient (1.20) The pixel value of P is added together, and the added value is set as the pixel value of the target pixel P.
For example, when applying the first enhancement kernel 432a, the pixel value P ′ after contrast enhancement is calculated according to the following equation (5).
Pixel value P ′ = 1.60 × P−0.15 × (a + b + c + d) (5)

As described above, the enhancement kernel differs depending on the enlargement magnification of the enlargement processing that has already been performed. Therefore, when the image is enlarged in order of 4 times and 8 times, the pixels having the characteristics of the original image are 2 The pixel is a pixel at a position distant from four images. Therefore, as illustrated in FIG. 12, the high-quality image block generation unit 430 performs enhancement processing with reference to pixels that are further away as the enlargement magnification is higher. For example, the high-quality image block generation unit 430 applies the first enhancement kernel 432a (see FIG. 5) in the first double enlargement process when the double enlargement process is applied twice in succession to realize the quadruple enlargement. 12 (A)) and the second enhancement kernel 432b (FIG. 12B) is applied in the second double enlargement process. The same applies to magnifications of 8 times, 16 times and later.
Further, the position of the pixel to be referred to is not limited to the vertical and horizontal directions as shown in FIG. For example, the high-quality image block generation unit 430 performs contrast enhancement with reference to pixels in an oblique direction, performs contrast enhancement with reference to further distant pixels, or the type and size of the processing target image data The emphasis kernel to be applied may be switched by.

  Next, the high-quality image block generation unit 430 generates the edge pattern and the estimated edge direction with respect to the region of interest obtained by the image block feature amount calculation unit 420, the region of interest subjected to the contrast enhancement processing as described above, and its surroundings. An enlarged image block for the region of interest is generated using the pixel value of the region.

  FIG. 13 is an explanatory diagram of a specific example of enlarged image block generation processing in the high-quality image block generation unit 430. First, the high-quality image block generation unit 430 calculates a pixel value corresponding to a 3 × 3 image block using a pixel value subjected to contrast enhancement processing based on the edge pattern of the region of interest and the estimated edge direction. FIG. 13A illustrates an example of the attention area and the peripheral area illustrated in FIG. As described above, in the image block feature value calculation unit 420, this attention area corresponds to the edge pattern 1010 and the estimated edge direction is determined to be “direction 1”. In the case of the combination of (edge pattern 1010) − (estimated edge direction “1”), the high-quality image block generation unit 430 calculates each pixel corresponding to the 3 × 3 image block as shown in FIG. Assuming p0 to p8, the pixel values of p0 to p8 are calculated by the following formula based on the pixel values {a, b, c, d} of the attention area shown in FIG. The combination of the edge pattern and the estimated edge direction is expressed as “(edge pattern) − (estimated edge direction)”.

p0 = a
p1 = (a + b) / 2
p2 = b
p3 = (a + c) / 2
p4 = (b + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (b + b) / 2
p8 = d

  These calculation formulas are uniquely determined by a combination of (edge pattern) − (estimated edge direction), and pixel values corresponding to 3 × 3 image blocks are calculated.

FIG. 14 is an explanatory diagram of an example of a calculation formula used in the case of another (edge pattern)-(estimated edge direction) combination.
FIG. 14A shows a calculation formula used in the case of (edge pattern 1000) − (estimated edge direction “1”), and pixel values corresponding to 3 × 3 image blocks are calculated by the following formula.

p0 = a
p2 = b
p3 = a
p4 = (b + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
p1 = (p4 + c) / 2

  FIG. 14B shows a calculation formula used in the case of (edge pattern 1100) − (estimated edge direction “5”), and pixel values corresponding to 3 × 3 image blocks are calculated by the following formula.

p0 = a
p1 = (a + b) / 2
p2 = b
p4 = (a + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
p3 = (p4 + c) / 2
p5 = (p4 + b) / 2

  FIG. 14C shows a calculation formula used in the case of (edge pattern 1100) − (estimated edge direction “2”), and a pixel value corresponding to a 3 × 3 image block is calculated by the following formula.

p0 = a
p1 = a
p2 = b
p3 = a
p4 = (b + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d

  FIG. 14D shows a calculation formula used in the case of (edge pattern 0101) − (estimated edge direction “7”), and a pixel value corresponding to a 3 × 3 image block is calculated by the following formula.

p0 = a
p2 = b
p3 = (a + c) / 2
p4 = (a + d) / 2
p5 = (b + d) / 2
p6 = c
p8 = d
p1 = (p4 + b) / 2
p7 = (p4 + c) / 2

  In the case of other edge patterns as well, pixel values corresponding to 3 × 3 image blocks can be calculated by performing calculation according to the calculation formula corresponding to each edge pattern.

  Next, the high-quality image block generation unit 430 generates a plurality of reference pixels in the peripheral area selected based on the pixel value equivalent to the 3 × 3 image block calculated as described above and the estimated edge direction of the attention area. Are used to generate 4 × 4 image blocks.

  FIG. 15 is an explanatory diagram of a method of selecting the reference pixels r0 to r13 based on the estimated edge direction in the attention area. When the estimated edge direction of the attention area is from direction 1 (22.5 °) to direction 3 (67.5 °), as shown in FIG. 15A, the reference pixels r0 to r5 are shown in FIG. A) is selected so that there are 3 pixels from the upper left to the lower and 3 pixels from the lower right to the upper as surrounded by a thick line frame. Further, when the estimated edge direction of the attention area is from the direction 5 (112.5 °) to the direction 7 (157.5 °), as shown in FIG. 3 pixels from the top to the top and 3 pixels from the top right to the bottom. For the reference pixels r6 to r13, four pixels are selected on the upper and lower sides, respectively, as shown in FIGS. 15A and 15B, regardless of the estimated edge direction. In this way, the reference pixel is selected based on the estimated edge direction in the attention area. Of course, the selection of reference pixels is not limited to the selection from two patterns as shown in FIG. 15, and more reference pixel selection patterns may be prepared according to the estimated edge direction. The reference pixel to be selected may be changed according to the estimated edge direction.

  FIG. 16 is an explanatory diagram of a specific example of generation processing of a 4 × 4 pixel enlarged image block. As shown in FIG. 16, using the calculated pixel values p0 to p8 corresponding to the 3 × 3 image block and the reference pixels r0 to r13 selected based on the estimated edge direction in the region of interest, according to the following calculation formula: Pixel values (s0 to s15) corresponding to the 4 × 4 pixel enlarged image block are calculated to generate a 4 × 4 enlarged image block.

s0 = 0.2 × r6 + 0.16 × r0 + 0.64 × p0
s1 = 0.2 × r7 + 0.32 × p0 + 0.48 × p1
s2 = 0.2 × r8 + 0.48 × p1 + 0.32 × p2
s3 = 0.2 × r9 + 0.64 × p2 + 0.16 × r1
s4 = 0.08 * r0 + 0.32 * p0 + 0.12 * r2 + 0.48 * p3
s5 = 0.16 × p0 + 0.24 × p1 + 0.24 × p3 + 0.36 × p4
s6 = 0.24 × p1 + 0.16 × p2 + 0.36 × p4 + 0.24 × p5
s7 = 0.32 × p2 + 0.08 × r1 + 0.48 × p5 + 0.12 × r3
s8 = 0.12 * r2 + 0.48 * p3 + 0.08 * r4 + 0.32 * p6
s9 = 0.24 × p3 + 0.36 × p4 + 0.16 × p6 + 0.24 × p7
s10 = 0.36 × p4 + 0.24 × p5 + 0.24 × p7 + 0.16 × p8
s11 = 0.48 × p5 + 0.12 × r3 + 0.32 × p8 + 0.08 × r5
s12 = 0.16 × r4 + 0.64 × p6 + 0.2 × r10
s13 = 0.32 × p6 + 0.48 × p7 + 0.2 × r11
s14 = 0.48 × p7 + 0.32 × p8 + 0.2 × r12
s15 = 0.64 × p8 + 0.16 × r5 + 0.2 × r13

  In this way, 4 × 4 pixel enlarged image blocks (s0 to s15) are generated for the attention area determined as the feature block by the image block feature amount calculation unit 420.

  As described above, the high-quality image block generation unit 430 performs the enlarged image block generation process in accordance with the image feature amount calculated by the image block feature amount calculation unit 420, whereby high-quality enlargement processing with reduced jaggies is performed. Can do.

[Enlarged image generator]
Next, the enlarged image generation unit 450 will be described in more detail.
The enlarged image generation unit 450 generates an enlarged image for the original image using the enlarged image block for the region of interest generated by the high-quality image block generation unit 430 and the enlarged image output from the high-speed enlargement processing unit 440. .

  In the enlarged image generation unit 450, the block superimposition degree determination unit 452 performs edge peripheral processing on an area determined as an edge portion in the image based on the block superposition degree of the feature block calculated by the edge strength calculation unit 422. It is determined whether the region (edge peripheral portion) is the most probable edge region (edge portion). That is, an area having a small block superposition degree is considered as a boundary area between a non-feature block (flat part) and a feature block (edge part) in the image, and is determined as an edge peripheral area. In addition, an area having a high degree of block superposition is determined to be the most probable edge area because most of the surrounding areas are feature blocks. For example, areas with block superimposition degrees 1 and 2 are determined as edge peripheral areas, and areas with block superimposition degrees 3 and 4 are determined as the most probable edge areas. The determination criterion is not limited to the above-described example, and for example, a determination criterion for classifying the edge more finely for each level (level 1 to level 4) of the block superimposition degree may be provided.

The output value correction unit 454 corrects the pixel value in the enlarged image block for the attention area generated by the high-quality image block generation unit 430 according to the determination result based on the block superposition degree by the block superposition degree determination unit 452.
FIG. 17 is a diagram for specifically explaining pixel value correction in an enlarged image block according to a determination result based on the block superposition degree.
In FIG. 17, as a specific example of the determination in the block superimposition degree determination unit 452, the areas with the block superimposition degrees 1 and 2 are determined as edge peripheral areas, and the areas with the block superimposition degrees 3 and 4 are determined as the most probable edge areas. An example is shown.
As illustrated in FIG. 17, the pixel value in the enlarged image block generated by the high-quality image block generation unit 430 corresponding to the region determined as the most probable edge region by the block superimposition degree determination unit 452 is used as it is. Is output. On the other hand, the pixel value in the enlarged image block corresponding to the area determined as the edge peripheral area in the block superimposition degree determination unit 452 is corrected and output.

FIG. 18 is a diagram specifically explaining the pixel value correction.
As shown in FIG. 18, the pixel values in the enlarged image block corresponding to the area determined as the edge peripheral area are expressed as follows using the pixel values at the corresponding positions on the enlarged image output from the high-speed enlargement processing unit 440. Correction is performed by equation (6).
Correction value = α × s + (1−α) × v (6)
Here, s is a pixel value in the enlarged image block, v is a pixel value on the enlarged image output from the high-speed enlargement processing unit 440, and α is a variable (0 ≦ α ≦ 1). Here, the variable α may be a predetermined value, or may be variable based on the block superposition degree.

  As described above, the image enlargement program 4 classifies the edge portion in the image as the edge peripheral region and the most probable edge region based on the degree of block superposition, and the most probable edge region has a high-quality image block. The pixel value of the enlarged image block generated by the generation unit 430 is output as it is, and the pixel value of the enlarged image block generated by the high-quality image block generation unit 430 is output to the edge peripheral region as the high-speed enlargement processing unit 440. By correcting with the pixel value of the enlarged image, the cause of image quality deterioration such as undershoot and overshoot in the edge peripheral region can be suppressed while maintaining the sharpness and smoothness of the edge.

[Overall operation]
The overall operation (image enlargement process) of the image processing apparatus 2 will be described.
FIG. 19 is a flowchart (S20) showing the overall operation of the image processing apparatus 2.
As shown in FIG. 19, in step 200 (S200), the image block setting unit 410 sets the default image block size required for the processing in the image block feature value calculation unit 420 and the high-quality image block generation unit 430. An image block having a set block size is cut out sequentially (for example, in raster scan order) from the processing target image data stored in the storage unit 400 and set, and each cut out image block is converted into an image block feature amount calculation unit. 420 and the high-quality image block generation unit 430, respectively.

In step 202 (S202), the image block feature value calculation unit 420 calculates the edge strength G of the attention area in the input image block using Expression (1).
In addition, {a, b, c, d} is each pixel value in the attention area as illustrated in FIG. When the input image data is not a grayscale image but a color image in, for example, an RGB color space, the image block feature value calculation unit 420 performs an image block for each color component in each of the R, G, and B color spaces with respect to the region of interest. For each, the edge strengths Gr, Gg, and Gb are calculated using the formula (1), and the image component of the color component that is the maximum edge strength among the Gr, Gg, and Gb is selected, and the edge strength is selected as the attention area Edge strength (common to all color components).

In step 204 (S204), the image block feature amount calculation unit 420 determines whether the attention area is a feature block or a non-feature block based on the edge strength G of the attention area. Specifically, the image block feature amount calculation unit 420 calculates the attention area when the calculated edge strength G is equal to or larger than a predetermined threshold Th (that is, when the gradation change amount in the attention area is large). When the calculated edge strength G is less than a predetermined threshold Th (that is, when the gradation change amount in the attention area is small), the attention area is determined as a non-feature block.
When it is determined that the attention area is a feature block, the image processing apparatus 2 proceeds to the process of S206 to perform an enlargement process for storing the image feature of the attention area, and the attention area is a non-feature block. If it is determined, the process returns to S200 and the next image block is processed.

  In step 206 (S206), if the image block feature amount calculation unit 420 overlaps an image region determined to be a feature block with an image region that has already been determined to be a feature block, block superimposition is performed. Calculate the degree.

In step 208 (S208), the image block feature value calculation unit 420 uses the expression (3) for the attention area (feature block) and the edge angle Θ of the reference area in one or more peripheral areas including the attention area. To calculate.
Note that gx and gy are values calculated in the equation (1). Then, the image block feature quantity calculation unit 420 estimates the edge direction θ of the region of interest from the calculated plurality of edge angles Θ. For example, the image block feature quantity calculation unit 420 calculates the average value of the obtained plurality of edge angles Θ, and sets the calculated average value as the estimated edge direction θ.

  In step 210 (S210), the image block feature quantity calculation unit 420 selects an edge pattern using the estimated edge direction θ and the pixel distribution pattern of the region of interest (feature block). The edge pattern is selected from a pattern table (FIG. 10) prepared in advance according to the edge direction and the pixel distribution pattern.

  In step 212 (S212), the high-quality image block generation unit 430 uses the enhancement kernel 432 (FIG. 12) in which the size and the weighting coefficient are set according to the enlargement ratio, and the image cut out by the image block setting unit 410 Contrast emphasis processing is performed on the image data of the attention area (feature block) and its peripheral area in the block.

  In step 214 (S214), the high-quality image block generation unit 430 performs contrast enhancement in S212, and the edge pattern of the attention area and the estimated edge direction θ of the attention area obtained by the image block feature amount calculation section 420. The pixel values (p0 to p8) corresponding to the 3 × 3 image block are calculated using the calculation formulas (FIGS. 13 and 14) corresponding to the edge pattern and the estimated edge direction θ using the pixel values in the attention area and the peripheral area. ) Is generated. Furthermore, the high-quality image block generation unit 430 uses the reference pixels (r0 to r13) selected based on the generated pixel values (p0 to p8) and the estimated edge direction θ, and the calculation formula illustrated in FIG. Thus, an enlarged image block for the attention area is generated.

  In step 216 (S216), the image processing apparatus 2 determines whether or not the processing from S200 to S214 has been completed for all input image data, and if it is determined that the processing has not been completed, S200. Returning to the process, the process for the next pixel block is performed, and if it is determined that the process has been completed for all the input image data, the process proceeds to S218.

In step 218 (S218), the high-speed enlargement processing unit 440 inputs the input image data stored in the storage unit 400 for each image or every several lines, and performs enlargement processing by the nearest neighbor interpolation method.
In this example, the high-speed enlargement processing unit 440 performs the enlargement process after the enlargement process (the processes from S200 to S214) by the high-quality image block generation unit 430 is completed. The enlargement process may be performed in parallel with the enlargement process by 430, or the enlargement process may be performed before the enlargement process by the high-quality image block generation unit 430.

In step 220 (S220), the enlarged image generation unit 450 uses the enlarged image generated by the high-quality image block generation unit 430 and the enlargement enlarged by the high-speed enlargement processing unit 440 based on the block superimposition degree calculated in S206. Merge images.
As an enlargement pixel integration method, the enlarged image generation unit 450 outputs the value of the enlarged image generated by the high-quality image block generation unit 430 as an output value for an area where the block superposition degree is large, and the block superposition degree is small. For the region, the value calculated by Expression (6) is used as the output value using the enlarged image generated by the high-quality image block generation unit 430 and the enlarged image generated by the high-speed enlargement processing unit 440. .

  As described above, the image processing apparatus 2 according to the present embodiment calculates the feature amount of an image region having a predetermined size including the target pixel, and the calculated feature amount satisfies the predetermined condition. When an enlarged image area is generated, and an enlarged image of the input image is generated by a different enlargement method, and image areas where the calculated feature amount satisfies a predetermined condition are overlapped, the generated enlargement is performed based on the number of overlaps. The generation value of the image area is corrected, and an enlarged image for the input image is generated based on the corrected enlarged image area and the enlarged images generated by different enlargement methods.

  Through such enlargement processing, the image processing apparatus 2 according to the present embodiment applies the high-quality enlargement processing only to characteristic portions (feature blocks), thereby suppressing the overall processing load and Since the high-quality enlargement process is applied to the portion, the feature amount of the characteristic portion can be stored, and a high-quality enlarged image in which image quality defects such as blur and jaggy are suppressed can be obtained at high speed. Further, the image processing apparatus 2 according to the present embodiment can suppress image quality defects such as undershoot and overshoot around the edge portion while maintaining the sharpness and smoothness of the edge.

It is a figure which illustrates the hardware constitutions of the image processing apparatus 2 with which the image processing method concerning this invention is applied centering on the control apparatus 20. FIG. It is a figure which shows the function structure of the image expansion program 4 which is performed by the control apparatus 20 and implement | achieves the image processing method concerning this invention. The edge strength calculation unit 422 exemplifies a region determined as a feature block by comparing the edge strength G with a reference value. The feature block which mutually overlaps is illustrated. The number of overlaps between feature blocks (block superposition degree) is illustrated. FIG. 6A is an explanatory diagram of a specific example of a region of interest and a peripheral region, and an example of an edge direction of the region of interest. FIG. 6A illustrates the region of interest and the peripheral region, and FIG. It is a figure which illustrates the estimated edge direction. It is a flowchart which shows the edge direction estimation process (S10) by the edge direction estimation part 424. FIG. It is a figure which illustrates the reference field selected in processing of S102 of edge direction presumption processing (S10). It is explanatory drawing of an example of the estimated edge direction in the attention area | region shown in FIG. It is a figure which illustrates the edge pattern table used by the edge pattern selection part 426. It is a figure explaining the selection method of the edge pattern corresponding to the attention area shown in FIG. It is a figure which illustrates the emphasis kernel 432 (edge emphasis kernel) used by the high quality image block generation part 430. It is explanatory drawing of the specific example of the production | generation process of the enlarged image block in the high quality image block production | generation part 430. It is explanatory drawing of an example of the calculation formula used in the case of the combination of an edge pattern and an estimated edge direction. It is explanatory drawing of the selection method of the reference pixels r0-r13 based on the estimated edge direction in an attention area. It is explanatory drawing of the specific example of a production | generation process of the enlarged image block of 4x4 pixel. It is a figure which demonstrates concretely pixel value correction | amendment in the enlarged image block according to the determination result based on a block superimposition degree. It is a figure explaining pixel value correction concretely. It is a flowchart (S20) which shows the whole operation | movement of the image processing apparatus 2.

Explanation of symbols

2 ... Image processing device 10 ... Printer device 20 ... Control device 202 ... CPU
204 ... Memory 22 ... Communication device 24 ... Storage device 26 ... UI device 4 ... Image enlargement program 400 ... Storage unit 410 ... Image block setting unit 420 ... Image block Feature amount calculation unit 422 ... Edge strength calculation unit 424 ... Edge direction estimation unit 426 ... Edge pattern selection unit 430 ... High quality image block generation unit 432a, 432b ... Enhancement kernel 434 ... Edge direction estimation unit 440 ... high-speed enlargement processing unit 450 ... enlarged image generation unit 452 ... block superposition degree determination unit 454 ... output value correction unit

Claims (14)

An image processing apparatus for enlarging an input image,
An area setting means for sequentially setting an attention area of a predetermined size for the input image so that the attention area to be set overlaps with other attention areas;
Area feature amount calculating means for calculating edge strength from the amount of change in the pixel value in the attention area as the feature amount of the attention area set by the area setting means ;
Regarding the overlapped portion in the setting of the attention region by the region setting means, the number of attention regions whose edge intensity calculated by the region feature amount calculation means is equal to or greater than a reference value among the attention regions including the overlapped portion is 2 A determination means for determining the overlapped portion as an edge portion when the value is equal to or greater than the predetermined value, and determining the overlapped portion as an edge peripheral portion when the number of the attention regions is 1 or more and less than the predetermined value When,
An enlarged image that generates an enlarged image region corresponding to the region of interest for the region of interest whose edge intensity calculated by the region feature quantity calculating means is equal to or greater than a reference value by using an enlargement method that maintains the edge of the region of interest. Region generation means;
Applying another enlargement method having a smaller processing load than the enlargement method used in the enlarged image region generation unit , and generating an enlarged image of the input image;
The portion determined as the edge peripheral portion by the determination unit is generated by the enlarged image region generation unit using a first generation value that is a generation value corresponding to the portion generated by the enlargement processing unit. Correction that corrects the second generation value, which is the generation value corresponding to the portion of the enlarged image region, to be a value between the first generation value and the second generation value before correction Means,
The enlarged image region corrected by the correcting unit, the enlarged image region generated by the enlarged image region generating unit at the portion determined as the edge by the determining unit, and the enlarged image generated by the expanding processing unit And an enlarged image generating means for generating an enlarged image for the input image. The enlarged image area generation means uses the gradation change direction of the region of interest, the gradation change direction in the vicinity region of the noted region, and the default pixel value pattern, and a pixel value of the neighboring region, expanding The image processing device according to claim 1, wherein an image region is generated. The enlarged image area generating means performs edge enhancement processing for enhancing gradation changes in accordance with an enlargement ratio, corrects pixel values in the image area, and generates an enlarged image area using the corrected pixel values. The image processing apparatus according to claim 1. The enlarged image area generation unit in accordance with the gradation change direction of the target region, selects the pixel value of the neighboring region of the region of interest, to claim 1 for generating an enlarged image area by using the selected pixel values The image processing apparatus described. 2. The image processing according to claim 1, wherein the enlarged image area generation unit generates an enlarged image area by applying a calculation formula uniquely determined from a combination of a gradation change direction of a region of interest and a predetermined pixel value pattern. apparatus. The image processing apparatus according to claim 1, wherein the enlargement processing unit applies an enlargement method based on at least a nearest neighbor interpolation method. The image processing apparatus according to claim 1, wherein the enlargement processing unit applies an enlargement method based on at least a linear interpolation method. The image processing apparatus according to claim 1, wherein the enlargement processing unit generates an enlarged image for each input image or for every several lines in the input image. The image processing apparatus according to claim 1, wherein the enlarged image generation unit arranges and integrates the enlarged image region at a corresponding position on the enlarged image generated by the enlargement processing unit. It said correction means, wherein a first generation values before correction of the a second generation value, the image processing apparatus according to claim 1 which is corrected using a value obtained by linear sum in a predetermined ratio. The ratio of the linear sum according to the number of regions of interest whose edge strength calculated by the region feature amount calculation unit is equal to or greater than a reference value among the regions of interest to be overlapped in the portion determined as the edge peripheral portion by the determination unit The image processing apparatus according to claim 10 . The enlarged image generating means includes an enlarged image area generated by the enlarged image area generating means, and an image portion that does not correspond to either an edge portion or an edge peripheral portion on the enlarged image generated by the enlargement processing means. The image processing apparatus according to claim 1 , wherein a pixel value at a position corresponding to is used. An image processing method for enlarging an input image,
Set attention areas of default size for the input image sequentially so that the attention area to be set overlaps with other attention areas,
The edge strength is calculated from the amount of change of the pixel value in the attention area as the feature amount of the attention area that has been set,
For the overlapped part in the attention area setting, if the number of attention areas whose calculated edge strength is equal to or greater than a reference value among the attention areas including the overlapped part is equal to or greater than a predetermined value that is equal to or greater than 2, the overlap is performed. The determined portion is determined as an edge portion, and when the number of the attention areas is 1 or more and less than a predetermined value, the overlapped portion is determined as an edge peripheral portion,
For an attention area whose calculated edge strength is equal to or greater than a reference value, an enlarged image area corresponding to the attention area is generated using an enlargement method that keeps an edge of the attention area ,
Applying another enlargement method that has a lower processing load than the enlargement method used to generate the enlarged image region, generates an enlarged image of the input image,
The determination portion and the edge peripheral portion, using the first generation value is generated values corresponding to partial in the generated enlarged image, the generated values corresponding to the partial of the generated enlarged image area A certain second generation value is corrected so as to be a value between the first generation value and the second generation value before correction;
An image processing method for generating an enlarged image for an input image by integrating a corrected enlarged image region, an enlarged image region in which a portion determined to be an edge portion is generated, and the generated enlarged image. In an image processing apparatus that includes a computer and performs an enlargement process of an input image,
An area setting step for sequentially setting an attention area of a predetermined size for the input image so that the attention area to be set overlaps with other attention areas;
As the feature quantity of the set region of interest, the region feature calculation step of calculating an edge strength from the amount of change in pixel value in the target region,
For the overlapped part in the attention area setting, if the number of attention areas whose calculated edge strength is equal to or greater than a reference value among the attention areas including the overlapped part is equal to or greater than a predetermined value that is equal to or greater than 2, the overlap is performed. A determination step of determining the overlapped portion as an edge portion, and determining the overlapped portion as an edge peripheral portion when the number of the attention regions is 1 or more and less than a predetermined value;
An enlarged image region generation step for generating an enlarged image region corresponding to the attention region with respect to the attention region whose calculated edge strength is equal to or greater than a reference value , using an enlargement method that keeps the edge of the attention region ;
Applying another enlargement method that has a smaller processing load than the enlargement method used for generating the enlarged image region, and generating an enlarged image of the input image; and
The determination portion and the edge peripheral portion, using the first generation value is generated values corresponding to partial in the generated enlarged image, the generated values corresponding to the partial of the generated enlarged image area A correction step of correcting a second generated value to be a value between the first generated value and the second generated value before correction;
An enlarged image generation step of integrating the corrected enlarged image area, the enlarged image area in which the portion determined to be an edge portion is generated, and the generated enlarged image to generate an enlarged image for the input image; For causing the computer of the image processing apparatus to execute.

Images