JP4730525B2 - Image processing apparatus and program thereof - Google Patents

Description

  The present invention relates to an image processing apparatus for enlarging an image expressed in multiple gradations including color.

  The image enlargement process is one of the basic processes for a system that performs image editing, filing, display, printing, and the like. In recent years, with the widespread use of image data mainly for display at display resolutions such as images on the Internet homepage and digital video, these low-resolution images are frequently printed by high-resolution printers. It has been broken. At the time of printing with this printer, it is desired to obtain a high-quality output result, and the importance of the high-quality enlargement process is increasing.

  Typical existing methods for enlarging images expressed in multi-tones including color (hereinafter referred to as multi-valued images) include nearest neighbor method, linear interpolation method, cubic convolution method, etc. There is. The nearest neighbor method is a method in which the pixel value of the pixel having the closest distance is used as each enlarged pixel value when the pixel is reversely mapped on the original image. Since this method has a small amount of calculation, it can be processed at high speed. However, since one pixel of the original image is directly expanded into a rectangular shape, if the difference between the pixel values of adjacent pixels is small, the degree of image quality degradation is small and has almost no effect. When the jagged edges and edge portions are conspicuous or the magnification is large, the degree of image quality deterioration is large, such as a mosaic image.

  In the linear interpolation method, it is assumed that pixel values between pixels change linearly, and pixel values of four pixels in the vicinity of a point obtained by inversely mapping the enlarged pixel are linearly interpolated to obtain a pixel value. is there. Although this method is heavier than the nearest neighbor method, the amount of calculation is relatively small and jaggies are not easily generated. On the other hand, there is a drawback that the entire image becomes blurred, centering on the edge portion that does not apply to the assumption that it changes linearly.

  The cubic convolution method defines an interpolation function that approximates a sinc function (sin (x) / x) based on the sampling theorem, and 16 pixels in the vicinity of a point obtained by inversely mapping the enlarged pixel (X and Y directions) This is a method for obtaining an enlarged pixel value by a convolution operation of each of 4 pixels) and the approximate interpolation function. This method has relatively good image quality compared to the above two methods, but has a drawback that the reference range is large and the amount of calculation is large. In addition, since the high frequency band is emphasized, there are disadvantages such as light jaggies occurring at the edge and noise components being emphasized.

As an attempt to solve the image quality problem of these enlarged images, for example, new systems such as Patent Document 1 and Patent Document 2 have been proposed.
The technique described in Patent Document 1 performs edge detection using an edge detection filter as a method for detecting the degree of change in an original image, and defines edge pixels based on the edge detection result. When this method is determined to be an edge pixel, enlargement is performed by the M-cubic method in which the cubic function shape of the cubic convolution method is adjusted. Otherwise, enlargement is performed by the nearest neighbor method. . However, since the edge portion having a large degree of change is performed by the M-cubic method adjusted so that the edge is emphasized rather than the cubic function shape of the cubic convolution method, jaggies are generated at the edge portion or the periphery of the edge portion. There is a disadvantage that the overshoot or undershoot increases.

  The technique described in Patent Document 2 uses bilinear interpolation for areas where roughness is noticeable when the sharpness intensity is increased, such as human skin or the sky in a landscape, and M-cubic interpolation for other areas where sharpness is required. Use to enlarge the image. However, since the technique described in Patent Document 2 also performs interpolation using the M-cubic method for the portion that requires sharpness in the same manner as Patent Document 1 described above, Furthermore, it is considered that overshoot and undershoot around the edge portion occur.

JP 2000-188681 A Japanese Patent No. 2000-151990

  The present invention has been made from the above-described background, and an object thereof is to provide an image processing apparatus that enlarges an image expressed in multiple gradations including color with high image quality and at a high speed with a light processing load. To do.

In order to achieve the above object, an image processing apparatus according to the present invention is an image processing apparatus that performs an enlargement process of an input image, and a feature amount calculated for an image region of a predetermined size including a target pixel is a predetermined amount For an image area that is equal to or greater than a reference value, if the relationship between the color elements of each pixel in the image area is the same, the image area is determined to be the same color block, and the color element that each pixel in the image area has If the relationship is different to the image area and the boundary block and determining an image area determination means between, with the image area determined as the same color block by the image area determination unit, the gradation change direction and default of the image area A first enlarged image region generating means for generating an enlarged image region corresponding to the image region by calculating a pixel value by applying a first arithmetic expression corresponding to the pixel value pattern ; In order to make the edge of the image area stand out from the enlargement method used in the enlarged image area generation means, the gradation change direction and the predetermined pixels in the image area are determined for the image area determined as the boundary block by the image area determination means. A second enlargement that generates an enlarged image area corresponding to the image area by calculating a pixel value by applying a second arithmetic expression obtained by changing the first arithmetic expression for the pixel position according to the value pattern Image region generating means.

Preferably, the image area determination means uses a gradation change amount in the image area calculated based on each pixel value in the image area as a feature quantity of the image area, and the feature quantity is equal to or greater than a predetermined reference value. The image area is the object of determination .
Preferably, the first enlarged image area generation unit and the second enlarged image area generation unit generate an enlarged image area based on a gradation change direction in the image area calculated from each pixel value in the image area. Generate .

Preferably, the first enlarged image region generating unit and the second enlarged image region generating unit are configured to use a predetermined pixel value pattern corresponding to a gradation change in the image region calculated from each pixel value in the image region. An enlarged image area is generated based on
Preferably, the first enlarged image area generation unit and said second enlarged image area generation unit, among the plurality of color components in a color space, based-out is selected in the feature amount calculated for each of the color components An enlarged image region is generated based on the feature amount further calculated from the one color component.

Preferably, the image area determination means determines the image area based on the magnitude relationship between the color component values of each pixel in the image area.

Preferably, the first enlarged image area generation unit and said second enlarged image area generation means, the feature amount of the image area, and the characteristic quantity of the neighborhood area of the image area, the pixel values of the neighboring region Are used to generate an enlarged image region.
Preferably, the first enlarged image region generating unit and the second enlarged image region generating unit perform edge enhancement processing for emphasizing a change in gradation according to an enlargement ratio, and calculate a pixel value in the image region. Correction is performed, and an enlarged image region is generated using the corrected pixel value.
Preferably, the first enlarged image region generation unit and the second enlarged image region generation unit select a pixel value in the vicinity region of the image region according to a gradation change direction, and the selected pixel An enlarged image area is generated using the value.

Preferably, the second enlarged image region generation unit selects a pixel in the enlarged image region based on the feature amount calculated for the image region, and uses a predetermined calculation method to select the selected pixel. Correct the pixel value.

Preferably, the enlargement processing unit applies an enlargement method based on at least a nearest neighbor interpolation method.
Preferably, the enlargement processing unit applies an enlargement method based on at least a linear interpolation method.
Preferably, the enlargement processing unit generates an enlarged image for each input image or for every several lines in the input image.

Preferably, the enlarged image generating means generates the enlarged image area generated by the first enlarged image area generating means and the second enlarged image area generating means by the enlarged processing means. Place it at the corresponding position in the image.
Preferably, the enlarged image generating means converts each pixel value on the enlarged image generated by the enlargement processing means to the first enlarged image area generating means and the second enlarged image area generating means. Replace with each pixel value of the generated enlarged image area.
Furthermore, it is preferable that the enlarged image generation unit enlarges the enlarged image regions generated by the first enlarged image region generation unit and the second enlarged image region generation unit when they overlap each other. An average value of a plurality of pixel values corresponding to the same pixel is calculated for the overlapping portion of the image area.

In addition, the program according to the present invention includes an image processing apparatus that includes a computer and performs an enlargement process of an input image. When the relationship between the color elements possessed by each pixel in the image region is the same for the region, the image region is determined as the same color block, and the relationship between the color elements possessed by each pixel in the image region is different an image area determining step of determining the boundary block the image area with the image area determined as the same color block, a first operational expression corresponding to the gradation change direction and default pixel value pattern of the image areas To calculate a pixel value, thereby generating a first enlarged image region generating step for generating an enlarged image region corresponding to the image region, and the first enlarged image region generating step. In order to make the edge of the image area stand out more than the enlargement method used in the image area, for the image area determined as the boundary block, the pixel position corresponding to the gradation change direction in the image area and the predetermined pixel value pattern A second enlarged image region generating step of generating an enlarged image region corresponding to the image region by calculating a pixel value by applying a second arithmetic equation obtained by changing the first arithmetic equation; Let the computer of the device execute.

  According to the image processing apparatus of the present invention, an image expressed in multiple gradations including color can be enlarged at high speed with high image quality and a light processing load.

[Hardware configuration]
A hardware configuration of the image processing apparatus 2 in the present embodiment will be described.
FIG. 1 is a diagram illustrating a hardware configuration of an image processing apparatus 2 to which an image processing method according to the present invention is applied, centering on a control apparatus 20.
As illustrated in FIG. 1, the image processing apparatus 2 includes a control device 20 including a CPU 202 and a memory 204, a communication device 22, a storage device 24 such as an HDD / CD device, an LCD display device or a CRT display device, and a keyboard. A user interface device (UI device) 26 including a touch panel and the like is included.
The image processing apparatus 2 is, for example, a general-purpose computer in which an image enlargement program 4 (described later with reference to FIG. 2) is installed. The image processing apparatus 2 acquires image data via the communication device 22 or the storage device 24. Enlarge the image data according to the output resolution. For example, the image processing apparatus 2 converts the resolution to 600 dpi or 2400 dpi when outputting image data to the printer apparatus 10, and 75 dpi or the like when outputting image data to the UI apparatus 26. Convert to the resolution.

[Image enlargement program]
FIG. 2 is a diagram showing a functional configuration of the image enlargement program 4 which is executed by the control device 20 (FIG. 1) and realizes the image processing method according to the present invention.
As illustrated in FIG. 2, the image enlargement program 4 includes a storage unit 400, an image block setting unit 410, an image block feature amount calculation unit 420, a high-quality image block generation unit 430, a high-speed enlargement processing unit 440, and an enlarged image generation unit 450. Have The image block feature quantity calculation unit 420 includes an edge strength calculation unit 422, a boundary block determination unit 424 (image area determination unit), an edge direction estimation unit 426, and an edge pattern selection unit 428, and includes a high quality image block generation unit. 430 includes a boundary block processing unit 432.
Note that all or part of the image enlargement program 4 may be realized by hardware such as an ASIC.

  The storage unit 400 temporarily stores image data until the image data is enlarged, and temporarily stores the enlarged image data that has undergone resolution conversion or enlargement processing until it is output to an output device (not shown). The image data is data described in an image format (for example, BMP, TIFF, PNG, etc.) that can be processed by the image processing apparatus 2, and is captured by a digital camera (not shown) or a scanner (not shown). Or image data created by an application program that is created or edited in a personal computer (image processing apparatus 2 or the like). Enlarged image data (image data after being enlarged) is also data in the same image format.

  The image block setting unit 410 sets a predetermined image block size required in the processing of the image block feature amount calculation unit 420 and the high quality image block generation unit 430, respectively. In addition, the image block setting unit 410 sequentially cuts out image blocks having a set block size from the input image data stored in the storage unit 400 (for example, in raster scan order), and converts each image block having the block size into an image. The block feature value calculation unit 420 and the high-quality image block generation unit 430 are each output.

  The image block feature amount calculation unit 420 calculates the image feature amount in the attention area, which is at least a part of the image blocks sequentially input from the image block setting section 410, in the image block including the attention area or the peripheral portion of the attention area Calculation is performed based on each pixel value. The image feature amount is, for example, edge strength (gradation change amount), edge angle (gradation change direction), or color information of the region of interest. However, the image feature amount is not limited to these. For example, the image block feature value calculation unit 420 calculates an average value of each pixel value of the attention area, and uses a value (for example, standard deviation or variance) representing a variation of each pixel value of the attention area with respect to the average value as the image feature value. May be calculated as

  In addition, the image block feature quantity calculation unit 420 compares the image feature quantity calculated for each attention area and the reference value, and performs the attention area separation. By distinguishing the attention area, an image block having a characteristic (for example, an image block including an edge) and an image block having a small feature (for example, an image block having a small change in pixel value) are separated. More specifically, the image block feature value calculation unit 420 determines an image block whose calculated image feature value is equal to or greater than a reference value as a feature block, and an image block whose calculated image feature value is less than the reference value. Are determined as non-characteristic blocks.

  In this example, the image block feature amount calculation unit 420 calculates the edge strength (numerical value) of an image block as one of the image feature amounts, and an image block whose calculated edge strength is a reference value or more is a feature block. The image block having the edge strength less than the reference value is determined to be a non-feature block. Note that the feature quantity used for the feature block segmentation is not limited to the edge strength, and may be the above-described color information, standard deviation, variance, or other feature quantities.

  Further, the image block feature amount calculation unit 420 calculates the color tone information of each pixel in the block for the block determined as the feature block, and all the pixels in the block have the same color tone or pixels having different color tones. In the latter case, it is determined that the block is a boundary block.

The high image quality image block generation unit 430 generates an enlarged image block corresponding to the attention area using the image feature amount calculated by the image block feature amount calculation unit 420. For the enlargement processing in the high-quality image block generation unit 430, it is desirable to apply an enlargement method that retains the features included in the region of interest. In addition, the high-quality image block generation unit 430 performs the enlargement process only on the image block (that is, the feature block) in which the feature amount calculated by the image block feature amount calculation unit 420 is greater than or equal to the reference value.
Also, the high-quality image block generation unit 430 applies different methods to the feature blocks determined to be the same color block by the image block feature value calculation unit 420 and the feature blocks determined to be the boundary block, and enlarges them. Generate an image block.

  The high-speed enlargement processing unit 440 enlarges the input image data stored in the storage unit 400. The high-speed enlargement processing unit 440 applies an enlargement algorithm that has a smaller processing load than the enlargement process performed by the high-quality image block generation unit 430. Specifically, the high-speed enlargement processing unit 440 enlarges the input image data by applying, for example, a nearest neighbor interpolation enlargement method or a linear interpolation enlargement method. Also, the high-speed enlargement processing unit 440 performs enlargement processing in units of input images or in units of several lines of input images, rather than processing for each image block.

  The enlarged image generation unit 450 uses the enlarged image block enlarged by the high-quality image block generation unit 430 and the enlarged image enlarged by the high-speed enlargement processing unit 440 to generate an enlarged image for the original image. More specifically, the enlarged image generation unit 450 has a high-quality image block generation unit 430 only for image blocks (feature blocks) in which the image feature amount calculated by the image block feature amount calculation unit 420 is greater than or equal to a reference value. For the image block (non-feature block) in which the calculated image feature amount is less than the reference value, the enlarged image enlarged by the high-speed enlargement processing unit 440 is applied to the Generate an enlarged image.

[Image block feature value calculation unit]
Next, the image block feature value calculation unit 420 will be described in more detail. A case where the attention area is a 2 × 2 pixel size block and the peripheral area including the attention area is a 4 × 4 pixel size block will be described as a specific example.
The edge strength calculation unit 422 calculates the edge strength G of the attention area in the image block cut out by the image block setting unit 410 using the following equation (1).

gx = (a + c−b−d) / 2
gy = (a + b−c−d) / 2
G = gx × gx + gy × gy (1)

In Expression (1), a, b, c, and d are pixel values of each pixel in the region of interest as illustrated in FIG. gx represents the amount of change in the pixel value in the main scanning direction (left and right direction in FIG. 5), and gy represents the amount of change in the pixel value in the sub scanning direction (up and down direction in FIG. 5).
Note that the edge strength may be calculated by the following formula (2) or the like instead of the value calculated by the above formula (1) alone.
G = | gx | + | gy | (2)
That is, the edge strength G may be calculated as the sum of the absolute value of gx and the absolute value of gy.

Further, the edge strength calculation unit 422 compares the calculated edge strength G with a reference value (predetermined threshold value Th) to determine whether the attention area is a feature block or a non-feature block, The determination result is output to the high-quality image block generation unit 430 and the like.
When the edge strength G of the attention area is smaller than the threshold value Th, the edge direction estimation unit 426 and the edge pattern selection unit 428 do not process the attention area.

FIG. 3 illustrates an area determined as a feature block by the edge strength calculation unit 422 by comparing the edge strength G with a reference value.
As illustrated in FIG. 3, the feature blocks are concentrated near the edges in the original image.
4 is a diagram illustrating edge portions, feature blocks, and pixel values of the feature blocks in the image. FIG. 4A is a diagram illustrating edge portions in the image, and FIG. ) Exemplifies image regions determined as feature blocks (feature block 1, feature block 2 and feature block 3) in the image exemplified in FIG. 4A, and FIG. 4C illustrates the feature block. It is a figure which illustrates each pixel value.

  The boundary block determination unit 424 performs the processing in the block on the image block (the feature block 1, the feature block 2 and the feature block 3 illustrated in FIG. 4B) determined as the feature block by the edge strength calculation unit 422. The magnitude relationship of pixel values of color elements (for example, R, G, B) of each pixel is determined. More specifically, the boundary block determination unit 424 determines the magnitude relationship between the R, G, and B pixel values of each pixel illustrated in FIG. For example, in the feature block 1 in FIG. 4B, since the magnitude relationship of R, G, and B of each pixel is R ≧ G ≧ B for all pixels, the boundary block determination unit 424 selects such a feature block. It is determined as the same color block.

  On the other hand, in the feature block 2 and the feature block 3 in FIG. 4B, the magnitude relationship of R, G, and B of the pixels surrounded by the frame in FIG. 4C is different from the other pixels. That is, in the feature block 2, the magnitude relationship is G ≧ R ≧ B in the pixels surrounded by the frame, while R ≧ G ≧ B in the other three pixels. In the feature block 3, the magnitude relationship is R ≧ G ≧ B in the pixels surrounded by the frame, while G ≧ R ≧ B in the other three pixels. The boundary block determination unit 424 determines a feature block including pixels having different RGB magnitude relationships in the block as described above as a boundary block.

The boundary block determination unit 424 outputs the determination result of the same color block and the boundary block to the high quality image block generation unit 430 and the like.
The method for determining the same color block and the boundary block is not limited to the above-described method, and any method may be used as long as the method can calculate the hue change. Furthermore, in the specific example shown in FIG. 4, the feature block 2 and the feature block 3 are determined as boundary blocks. For example, “a feature block that changes from a same color block to a boundary block is determined as a boundary block. The determination method such as determining the feature block 2 as a boundary block may be used as a determination criterion for the boundary block.
In this way, the boundary block determination unit 424 can detect the edge portion of the input image.

Next, the edge direction estimation unit 426 will be described.
FIG. 5 is an explanatory diagram of a specific example of the attention area and the peripheral area, and an example of the edge direction of the attention area. FIG. 5A illustrates the attention area and the peripheral area, and FIG. It is a figure which illustrates the edge direction estimated about this attention area.
As illustrated in FIG. 5A, the region of interest (image region) has a 2 × 2 rectangular region (two each in the main scanning direction and the sub-scanning direction), and the peripheral region is 4 × 4. It has rectangular areas (four each in the main scanning direction and the sub-scanning direction). Each rectangle corresponds to a pixel, and each numeral in the rectangle indicates a pixel value. That is, the attention area is the pixel {a, b, c, d} = {15, 104, 86, 203} near the center.
Hereinafter, the edge direction estimation processing by the edge direction estimation unit 426 will be described using the attention area illustrated in FIG. 5A as a specific example.

FIG. 6 is a flowchart showing edge direction estimation processing (S10) by the edge direction estimation unit 426.
FIG. 7 is a diagram illustrating a reference region selected in the process of S102 of the edge direction estimation process (S10). The hatched portion in FIG. 7 corresponds to the region of interest shown in FIG.
As shown in FIG. 6, in step 100 (S100), the edge direction estimation unit 426 calculates the edge angle Θ of the attention area exemplified in FIG. 5A by the following equation (3).
Θ = arctan (gy / gx) (3)
In FIG. 5A, the pixel value of the attention area is {a, b, c, d} = {15, 104, 86, 203}, and from the equation (1),
gx = −103
gy = −85
And substituting these into equation (3),
Θ = -140.5 °
It becomes.
The direction of the edge angle Θ corresponds to the broken line direction shown in FIG.

  Furthermore, the edge direction estimation unit 426 determines which of the angle ranges in the directions (eight directions) divided every 22.5 ° is the calculated edge angle Θ. In this example, the angle range around the edge angle Θ of 0 ° or ± 180 ° is “direction 0”, the angle range around 22.5 ° or −157.5 ° is “direction 1”, An angle range centered on 45 ° or −135 ° is “direction 2”, an angle range centered on 67.5 ° or −112.5 ° is “direction 3”, and centered on 90 ° or −90 °. An angle range centered on 112.5 ° or −67.5 ° is “direction 5”, and an angle range centered on 135 ° or −45 ° is “direction 6”. ", And an angle range centered on 157.5 ° or -22.5 ° is" direction 7 ". These angle ranges are ± 11.25 ° from their respective centers. Since the edge angle Θ (= −140.5 °) in the above specific example is included in the range of −135 ° ± 11.25 °, the edge angle is “direction 2”.

In step 102 (S102), the edge direction estimation unit 426 estimates the edge direction from the peripheral area (outside the thick line frame) shown in FIG. 5A according to the calculated edge angle Θ of the attention area. Select the reference area to be used for. More specifically, the edge direction estimation unit 426 selects the reference region so as to include a pixel that may be adjacent to the attention region in the direction of the calculated edge angle Θ.
For example, the edge direction estimation unit 426 selects the reference areas (two areas surrounded by a thick line) illustrated in FIG. 7A when the edge angle calculated for the attention area is included in “direction 0”. When the calculated edge angle is included in “direction 4”, the reference area (two areas surrounded by a thick line) illustrated in FIG. 7B is selected, and the calculated edge angle is other than the above. 7 (directions 1 to 3 and directions 5 to 7) are selected, the reference areas (four areas surrounded by thick lines) illustrated in FIG. 7C are selected. In the specific example shown in FIG. 5, since the direction of the edge angle is “direction 2”, the four reference areas shown in FIG. 7C are candidates for selection.
Note that the reference area is not limited to the area illustrated in FIG. 7. For example, in the case of FIG. 7C, the number of reference areas may be 8 or may be set according to each direction.

In step 104 (S104), the edge direction estimation unit 426 calculates the edge angle Θ for each of the selected reference areas in accordance with the equations (1) and (3) as in the processing of S100.
In step 106 (S106), the edge direction estimation unit 426 compares the edge angle calculated for each reference area with the edge angle calculated for the attention area, and the difference between these is set in advance. It is determined whether it is smaller than Θth. If the edge angle difference is smaller than the threshold value Θth, the edge direction estimation unit 426 determines that the edge angle of the reference region is an appropriate edge angle and proceeds to the processing of S108, and the edge angle difference is equal to or greater than the threshold value Θth. If it is, it is determined that the edge angle of the reference region is not an appropriate edge angle, and the process proceeds to S110.

In step 108 (S108), the edge direction estimation unit 426 increments the angle reference number. That is, the edge direction estimation unit 426 increments the angle reference number only when it is determined that the edge angle calculated for the reference region is an appropriate edge angle.
The angle reference number is a variable for counting the reference number of the edge angle, and is initialized to “angle reference number 1” for each region of interest.

In step 110 (S110), the edge direction estimation unit 426 determines whether or not the edge angle has been calculated for all the selected reference areas, and when the edge angle has been calculated for all the reference areas, the edge direction is calculated in S112. In other cases, the process returns to S104 to calculate the edge angle for the next reference area.
In step 112 (S112), the edge direction estimation unit 426 calculates the sum of the edge angle of the attention area and the edge angle of the reference area determined as an appropriate edge angle, and calculates the sum of the calculated edge angles as an angle. The average edge angle divided by the reference number is set as the estimated edge direction of the attention area.

  In the specific example shown in FIG. 5, the edge angle ΘU is ΘU = -149.4 ° from the upper reference region {86, 203, 171, 211}, and the edge angle ΘL is the left reference region { ΘL = −131.2 ° from 10, 15, 20, 86}, and the edge angle ΘD becomes ΘD = −175.2 ° from the lower reference region {1,102,15,104}, and the edge angle ΘR is From the reference region {104, 215, 203, 219} on the right side, ΘR = −141.0 °. The edge angle Θ = −140.5 ° of the attention area is compared with the edge angle of each reference area, and the number of reference areas whose difference is smaller than the threshold Θth is counted as the angle reference number.

  FIG. 8 is an explanatory diagram of an example of the estimated edge direction in the region of interest shown in FIG. For example, in the above-described specific example, if the difference between the edge angle of the attention area for all the reference areas is smaller than the threshold Θth, the sum of the edge angles obtained from the attention area and the four reference areas is −737.3 °. By dividing by the angle reference number 5, the average edge angle is determined to be -147.5 °. Also in this case, the edge direction estimation unit 426 determines whether the average edge angle is included in any of the eight directions, for example, in the same manner as the edge direction of the region of interest described above. In this example, since the average edge angle is −147.5 °, it is included in “direction 1”, and “direction 1” is the estimated edge direction.

In FIG. 8, a gray scale image that is one color element per pixel is described as a specific example, but the present invention is not limited to this. For example, when a color image in the RGB color space of three color elements per pixel is input, the color space data selected by the strength of the edge strength Gr, Gg, Gb in the data of each color component The edge direction estimation process may be performed. More specifically, the image block feature value calculation unit 420 calculates the edge strength for each color component, selects the color component that maximizes the calculated edge strengths Gr, Gg, and Gb. The feature amount is calculated only for the color component. In this way, it is possible to suppress deterioration in image quality such as color shift at the edge of enlarged image data in a color image.
In this example, for the attention area and the reference area, the edge angle calculated by the expression (1) is classified into any one of the eight directions, but the classification is not limited to the eight directions, and the edge has higher accuracy. If a direction is required, the classification may be done for a larger number of angular regions, such as 12 directions (every 15.0 °), 16 directions (every 12.25 °).

The edge pattern selection unit 428 will be described.
FIG. 9 is a diagram illustrating an edge pattern table used by the edge pattern selection unit 428.
As illustrated in FIG. 9, the edge pattern selection unit 428 has an edge pattern table in which the estimated edge direction and the edge pattern are associated with each other. In the edge pattern table, one or more edge patterns are registered for each estimated edge direction (for example, eight directions) corresponding to the pattern size of the region of interest.
The edge pattern selection unit 428 refers to the edge pattern table and selects an edge pattern corresponding to the estimated edge direction estimated for each region of interest by the edge direction estimation unit 426.
In this example, as illustrated in FIG. 8, since the estimated edge direction with respect to the region of interest is estimated to be “direction 1” by the edge direction estimating unit 426, the edge pattern selecting unit 428 performs this estimated edge direction. According to (direction 1), four edge patterns from “pattern 0” to “pattern 3” corresponding to direction 1 are selected from the edge pattern table shown in FIG. Candidate.

The edge pattern selection unit 428 selects one edge pattern from one or more edge patterns that are edge pattern candidates based on the pixel value of the region of interest. A specific method for selecting an edge pattern will be described with reference to FIG.
FIG. 10 is a diagram for explaining a method of selecting an edge pattern corresponding to the attention area shown in FIG.
In this example, since the estimated edge direction is “direction 1”, four edge patterns from “pattern 0” to “pattern 3” are selected as the edge pattern candidates as illustrated in FIG. 10A. Has been. These edge patterns are expressed as bit patterns as illustrated in FIG. Specifically, the edge pattern is formed into a bit pattern with the white portion being 0 and the others being 1, and bit patterns from “bit pattern 0” to “bit pattern 3” are generated. Note that these bit patterns may be registered in advance as a bit table in the edge pattern table shown in FIG.

  The edge pattern selection unit 428 determines a bit pattern corresponding to the attention area. Specifically, the edge pattern selection unit 428 calculates an average pixel value in the attention area according to the following equation (4), subtracts the average value from each pixel value in the attention area, and uses the sign as the attention. The pixel value pattern of the region is used.

Mean = (a + b + c + d) / 4
a_sign = a-Mean
b_sign = b-Mean
c_sign = c-Mean
d_sign = d-Mean (4)

In this example, as shown in FIG. 10C, Mean = (15 + 104 + 86 + 203) / 4 = 102, and a_sign = −87, b_sign = 2, c_sign = −16, and d_sign = 101. Therefore, the edge pattern selection unit 428 determines these positive and negative signs, and generates a bit pattern (1010) of the region of interest as shown in FIG.
The edge pattern selection unit 428 performs pattern matching between the bit pattern corresponding to the edge pattern candidate illustrated in FIG. 10B and the bit pattern of the region of interest illustrated in FIG. Is determined as the selection pattern. The selected edge pattern is applied to enlarged image block generation processing in a high-quality image block generation unit 430 described later.

  Note that the edge pattern is not limited to the edge pattern shown in FIG. 9. For example, the edge pattern selection unit 428 switches the edge pattern table according to the type of input image data and applies a different edge pattern. Also good. Further, the edge pattern selection unit 428 may increase or decrease the number of edge pattern candidates at each angle.

[High-quality image block generator]
Next, the high-quality image block generation unit 430 will be described in more detail.
The high-quality image block generation unit 430 generates an edge for the attention area obtained by the image block feature value calculation unit 420 with respect to the attention area determined by the image block feature value calculation unit 420 as the feature block and the same color block. An enlarged image block generation process is performed based on the pattern and the estimated edge direction. In addition, the high-quality image block generation unit 430 applies the above-described method to the attention area determined as the feature block and the boundary block by the image block feature amount calculation unit 420 via the boundary block processing unit 432. A different method is applied to generate an enlarged image block, and a pixel value of a predetermined pixel among the generated pixels is corrected.

Hereinafter, the enlarged image block generation process for the attention area determined to be the same color block will be described.
First, the high-quality image block generation unit 430 uses the size and coefficient emphasis kernel according to the enlargement magnification of the enlargement process, and the image of the attention area and the surrounding area in the image block cut out by the image block setting unit 410 Emphasize the contrast of data.
FIG. 11 is a diagram illustrating an enhancement kernel 434 (edge enhancement kernel) used by the high-quality image block generation unit 430.
As illustrated in FIG. 11, the first enhancement kernel 434 a enhances contrast using weighting factors “1.60” and “−0.15”, and the second enhancement kernel 434 b includes a weighting factor “1”. .20 ”and“ −0.05 ”to enhance contrast. These enhancement kernels are associated with the enlargement magnification of the enlargement process already performed on the target image, and refer to pixel values at different positions using different weighting coefficients.

As illustrated in FIG. 11A, the first enhancement kernel 434a refers to the pixel a, the right pixel b, the pixel c, and the pixel d of the pixel of interest P with reference to the pixels of these pixels. Each value is multiplied by a weighting coefficient (−0.15) and added to the pixel value of the pixel of interest P multiplied by the weighting coefficient (1.60), and the sum is set as the pixel value of the pixel of interest P.
The second enhancement kernel 434b is applied to an image having a larger magnification than the first enhancement kernel 434a, and, as illustrated in FIG. 11B, the lower pixel a that is separated by one pixel from the target pixel P. , The right pixel b, the upper pixel c, and the left pixel d, the pixel value of these pixels is multiplied by a weighting coefficient (−0.05), and the pixel of interest multiplied by the weighting coefficient (1.20) The pixel value of P is added together, and the added value is set as the pixel value of the target pixel P.
For example, when applying the first enhancement kernel 434a, the pixel value P ′ after contrast enhancement is calculated according to the following equation (5).
Pixel value P ′ = 1.60 × P−0.15 × (a + b + c + d) (5)

As described above, the enhancement kernel differs depending on the enlargement magnification of the enlargement processing that has already been performed. Therefore, when the image is enlarged in order of 4 times and 8 times, the pixels having the characteristics of the original image are 2 The pixel is a pixel at a position distant from four images. Therefore, as illustrated in FIG. 11, the high-quality image block generation unit 430 performs enhancement processing with reference to pixels that are further away as the enlargement magnification is higher. For example, the high-quality image block generation unit 430 applies the first enhancement kernel 434a (see FIG. 5) in the first double enlargement process when the double enlargement process is applied twice in succession to realize the quadruple enlargement. 11 (A)) and the second enhancement kernel 434b (FIG. 11B) is applied in the second double enlargement process. The same applies to magnifications of 8 times, 16 times and later.
Further, the position of the pixel to be referred to is not limited to the vertical and horizontal directions as shown in FIG. For example, the high-quality image block generation unit 430 performs contrast enhancement with reference to pixels in an oblique direction, performs contrast enhancement with reference to further distant pixels, or the type and size of the processing target image data The emphasis kernel to be applied may be switched by.

  Next, the high-quality image block generation unit 430 generates the edge pattern and the estimated edge direction with respect to the region of interest obtained by the image block feature amount calculation unit 420, the region of interest subjected to the contrast enhancement processing as described above, and its surroundings. An enlarged image block for the region of interest is generated using the pixel value of the region.

  FIG. 12 is an explanatory diagram of a specific example of enlarged image block generation processing in the high-quality image block generation unit 430. First, the high-quality image block generation unit 430 calculates a pixel value corresponding to a 3 × 3 image block using a pixel value subjected to contrast enhancement processing based on the edge pattern of the region of interest and the estimated edge direction. FIG. 12A illustrates an example of the attention area and the peripheral area illustrated in FIG. As described above, in the image block feature value calculation unit 420, this attention area corresponds to the edge pattern 1010 and the estimated edge direction is determined to be “direction 1”. In the case of the combination of (edge pattern 1010) − (estimated edge direction “1”), the high-quality image block generation unit 430 calculates each pixel corresponding to the 3 × 3 image block as shown in FIG. Assuming p0 to p8, the pixel values of p0 to p8 are calculated by the following formula based on the pixel values {a, b, c, d} of the attention area shown in FIG. The combination of the edge pattern and the estimated edge direction is expressed as “(edge pattern) − (estimated edge direction)”.

p0 = a
p1 = (a + b) / 2
p2 = b
p3 = (a + c) / 2
p4 = (b + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (b + b) / 2
p8 = d

  These calculation formulas are uniquely determined by a combination of (edge pattern) − (estimated edge direction), and pixel values corresponding to 3 × 3 image blocks are calculated.

FIG. 13 is an explanatory diagram of an example of a calculation formula used in the case of another (edge pattern)-(estimated edge direction) combination.
FIG. 13A shows a calculation formula used in the case of (edge pattern 1000) − (estimated edge direction “1”), and pixel values corresponding to 3 × 3 image blocks are calculated by the following formula.

p0 = a
p2 = b
p3 = a
p4 = (b + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
p1 = (p4 + c) / 2

  FIG. 13B shows a calculation formula used in the case of (edge pattern 1100) − (estimated edge direction “5”), and pixel values corresponding to 3 × 3 image blocks are calculated by the following formula.

p0 = a
p1 = (a + b) / 2
p2 = b
p4 = (a + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
p3 = (p4 + c) / 2
p5 = (p4 + b) / 2

  FIG. 13C shows a calculation formula used in the case of (edge pattern 1100) − (estimated edge direction “2”), and pixel values corresponding to 3 × 3 image blocks are calculated by the following formula.

p0 = a
p1 = a
p2 = b
p3 = a
p4 = (b + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d

  FIG. 13D shows a calculation formula used in the case of (edge pattern 0101) − (estimated edge direction “7”), and a pixel value corresponding to a 3 × 3 image block is calculated by the following formula.

p0 = a
p2 = b
p3 = (a + c) / 2
p4 = (a + d) / 2
p5 = (b + d) / 2
p6 = c
p8 = d
p1 = (p4 + b) / 2
p7 = (p4 + c) / 2

  In the case of other edge patterns as well, pixel values corresponding to 3 × 3 image blocks can be calculated by performing calculation according to the calculation formula corresponding to each edge pattern.

  Next, the high-quality image block generation unit 430 generates a plurality of reference pixels in the peripheral area selected based on the pixel value equivalent to the 3 × 3 image block calculated as described above and the estimated edge direction of the attention area. Are used to generate 4 × 4 image blocks.

  FIG. 14 is an explanatory diagram of a method of selecting the reference pixels r0 to r13 based on the estimated edge direction in the attention area. When the estimated edge direction of the attention area is from direction 1 (22.5 °) to direction 3 (67.5 °), as shown in FIG. 14A, reference pixels r0 to r5 are shown in FIG. A) is selected so that there are 3 pixels from the upper left to the lower and 3 pixels from the lower right to the upper as surrounded by a thick line frame. Further, when the estimated edge direction of the attention area is from the direction 5 (112.5 °) to the direction 7 (157.5 °), as shown in FIG. 3 pixels from the top to the top and 3 pixels from the top right to the bottom. As shown in FIGS. 14A and 14B, four reference pixels r6 to r13 are selected, regardless of the estimated edge direction. In this way, the reference pixel is selected based on the estimated edge direction in the attention area. Of course, the selection of reference pixels is not limited to selection from two patterns as shown in FIG. 14, and more reference pixel selection patterns may be prepared according to the estimated edge direction. The reference pixel to be selected may be changed according to the estimated edge direction.

  FIG. 15 is an explanatory diagram of a specific example of generation processing of a 4 × 4 pixel enlarged image block. As shown in FIG. 15, using the calculated pixel values p0 to p8 corresponding to 3 × 3 image blocks and the reference pixels r0 to r13 selected based on the estimated edge direction in the region of interest, according to the following calculation formula: Pixel values (s0 to s15) corresponding to the 4 × 4 pixel enlarged image block are calculated to generate a 4 × 4 enlarged image block.

s0 = 0.2 × r6 + 0.16 × r0 + 0.64 × p0
s1 = 0.2 × r7 + 0.32 × p0 + 0.48 × p1
s2 = 0.2 × r8 + 0.48 × p1 + 0.32 × p2
s3 = 0.2 × r9 + 0.64 × p2 + 0.16 × r1
s4 = 0.08 * r0 + 0.32 * p0 + 0.12 * r2 + 0.48 * p3
s5 = 0.16 × p0 + 0.24 × p1 + 0.24 × p3 + 0.36 × p4
s6 = 0.24 × p1 + 0.16 × p2 + 0.36 × p4 + 0.24 × p5
s7 = 0.32 × p2 + 0.08 × r1 + 0.48 × p5 + 0.12 × r3
s8 = 0.12 * r2 + 0.48 * p3 + 0.08 * r4 + 0.32 * p6
s9 = 0.24 × p3 + 0.36 × p4 + 0.16 × p6 + 0.24 × p7
s10 = 0.36 × p4 + 0.24 × p5 + 0.24 × p7 + 0.16 × p8
s11 = 0.48 × p5 + 0.12 × r3 + 0.32 × p8 + 0.08 × r5
s12 = 0.16 × r4 + 0.64 × p6 + 0.2 × r10
s13 = 0.32 × p6 + 0.48 × p7 + 0.2 × r11
s14 = 0.48 × p7 + 0.32 × p8 + 0.2 × r12
s15 = 0.64 × p8 + 0.16 × r5 + 0.2 × r13

In this way, 4 × 4 pixel enlarged image blocks (s0 to s15) are generated for the attention area determined as the feature block and the same color block by the image block feature amount calculation unit 420.
As described above, the high-quality image block generation unit 430 performs the enlarged image block generation process according to the image feature amount calculated by the image block feature amount calculation unit 420, thereby achieving high-quality enlargement processing with reduced jaggies. Can do.

Further, the high-quality image block generation unit 430 generates an enlarged image block via the boundary block processing unit 432 for the attention area determined as the feature block and the boundary block by the image block feature amount calculation unit 420. .
The boundary block processing unit 432 will be described below.
The boundary block processing unit 432 calculates the 3 × 3 enlarged image block shown in FIG. 12 and FIG. 13 for the attention area that is determined as the boundary block by the image block feature amount calculation unit 420. An enlarged image block is generated using a calculation formula different from the formula. Further, the boundary block processing unit 432 calculates the edge pattern and the estimated edge of the attention area from the pixel values (s0 to s15) corresponding to the enlarged image block of 4 × 4 pixels generated using the calculation formula illustrated in FIG. One or more pixels are selected based on the direction and the pixel values are corrected.

FIG. 16 is an explanatory diagram of a specific example of enlarged image block generation processing and pixel value correction processing in the boundary block processing unit 432. FIG. 16A shows a 3 × 3 enlarged image in the case of the combination of (edge pattern 1010) − (estimated edge direction “1”) of a region of interest that is a feature block but not a boundary block, as shown in FIG. The calculation formula of a block is shown. FIG. 16B shows a calculation formula for a combination (edge pattern 1010) − (estimated edge direction “1”) similar to FIG. 16A when the region of interest is a boundary block.
As shown in FIG. 16B, when the region of interest is a boundary block, the boundary block processing unit 432 uses the following calculation formula different from the calculation formula shown in FIG. Calculate the pixel value of ~ p8. Note that in FIG. 16B, a calculation formula different from the calculation formula of FIG. 16A is surrounded by a dotted line.

p0 = a
p1 = (a + b) / 2
p2 = b
p3 = (a + c) / 2
p4 = (b + c) / 2
p5 = d
p6 = c
p7 = d
p8 = d

As shown in FIG. 16C, the boundary block processing unit 432 is generated using the calculation formula of FIG. 15 in the case of the combination of (edge pattern 1010) − (estimated edge direction “1”). For example, six correction pixels (s1, s4, s7, s8, s11, s14) are selected from the pixel values (s0 to s15) corresponding to the enlarged image block of 4 × 4 pixels, and the calculation formula of FIG. 15 is used. Each pixel value obtained in this way is corrected.
The boundary block processing unit 432 calculates a correction value using the following calculation formula.

s1 ′ = s1−α × s1
s4 ′ = s4-α × s4
s7 ′ = s7 + α × s7
s8 ′ = s8−α × s8
s11 ′ = s11 + α × s11
s14 ′ = s14 + α × s14

  Here, α is a parameter for adjusting the edge sharpness of the enlarged image block, and may be variable to obtain a desired edge sharpness, or may be a predetermined value in advance.

  In the specific example described above, for the combination of “(edge pattern 1010) − (estimated edge direction“ 1 ”)”, the calculation formula of the 3 × 3 enlarged image block is changed, the correction pixel is selected, and the correction value is calculated. For example, in the case of “(edge pattern) − (estimated edge direction)” shown in FIG. 13 for example, the boundary block processing unit 432 also uses the same color block based on the edge pattern and the estimated edge direction. A change calculation formula, a correction pixel selection, and a correction value calculation formula that are different from the calculation formula for calculating the correction value are uniquely determined.

As described above, the change of the calculation formula for generating the enlarged image block and the selection and correction of the generated pixel value in the boundary block processing unit 432 are the characteristics of the enlarged image block with respect to the attention area determined as the boundary block (in this example, the edge ) Is made more prominent.
Therefore, the calculation formula changing method and the correction pixel selection and correction calculation method described above are not limited to the above-described methods and the like, and the calculation formula changing method and the pixel Any method may be used as long as it is a selection and correction calculation method.

[Enlarged image generator]
Next, the enlarged image generation unit 450 will be described in more detail.
The enlarged image generation unit 450 integrates the enlarged image block for the region of interest generated by the high-quality image block generation unit 430 and the enlarged image output from the high-speed enlargement processing unit 440.
FIG. 17 is an explanatory diagram of a specific example in which the enlarged image block of 4 × 4 pixels generated by the high-quality image block generation unit 430 and the enlarged image output from the high-speed enlargement processing unit 440 are integrated.
As illustrated in FIG. 17, the enlarged image generation unit 450 sequentially arranges the sequentially generated enlarged image block 0 and enlarged image block 1 at corresponding positions on the enlarged image output from the high-speed enlargement processing unit 440. To integrate. At this time, the magnified image generation unit 450 may be arranged to replace each pixel value on the magnified image with each pixel value (S0 to S15) of the magnified block, or each pixel value on the magnified image and the magnified image. You may arrange | position so that each pixel value (S0-S15) of a block may overlap.

  When the corresponding positions on the enlarged image of the enlarged image blocks (enlarged image block 0 and enlarged image block 1 in FIG. 17) overlap, the enlarged image generation unit 450 sets the overlapping pixels of the enlarged image blocks. The average value of the pixel values is set as the pixel value of this pixel. The enlarged image generation unit 450 may calculate each pixel value by calculating the sum of the overlapping pixels and dividing the sum of the pixel values by the number of overlaps.

[Overall operation]
The overall operation (image enlargement process) of the image processing apparatus 2 will be described.
FIG. 18 is a flowchart (S20) showing the overall operation of the image processing apparatus 2.
As shown in FIG. 18, in step 200 (S200), the image block setting unit 410 sets the default image block size required for the processing in the image block feature amount calculation unit 420 and the high-quality image block generation unit 430. An image block having a set block size is cut out sequentially (for example, in raster scan order) from the processing target image data stored in the storage unit 400 and set, and each cut out image block is converted into an image block feature amount calculation unit. 420 and the high-quality image block generation unit 430, respectively.

In step 202 (S202), the image block feature value calculation unit 420 calculates the edge strength G of the attention area in the input image block using Expression (1).
In addition, {a, b, c, d} is each pixel value in the attention area as illustrated in FIG. When the input image data is not a grayscale image but a color image in, for example, an RGB color space, the image block feature value calculation unit 420 performs an image block for each color component in each of the R, G, and B color spaces with respect to the region of interest. For each, the edge strengths Gr, Gg, and Gb are calculated using the formula (1), and the image component of the color component that is the maximum edge strength among the Gr, Gg, and Gb is selected, and the edge strength is selected as the region of interest. Edge strength (common to all color components).

In step 204 (S204), the image block feature amount calculation unit 420 determines whether the attention area is a feature block or a non-feature block based on the edge strength G of the attention area. Specifically, the image block feature amount calculation unit 420 calculates the attention area when the calculated edge strength G is equal to or larger than a predetermined threshold Th (that is, when the gradation change amount in the attention area is large). When the calculated edge strength G is less than a predetermined threshold Th (that is, when the gradation change amount in the attention area is small), the attention area is determined as a non-feature block.
When it is determined that the attention area is a feature block, the image processing apparatus 2 proceeds to the process of S206 to perform an enlargement process for storing the image feature of the attention area, and the attention area is a non-feature block. If it is determined, the process returns to S200 and the next image block is processed.

  In step 206 (S206), the image block feature quantity calculation unit 420 determines the magnitude relationship between the pixel values of the color elements of each pixel in the block for the attention area determined as the feature block, and the feature block is the same color block. Determine whether it is a boundary block or not.

In step 208 (S208), the image block feature value calculation unit 420 uses the expression (3) for the attention area (feature block) and the edge angle Θ of the reference area in one or more peripheral areas including the attention area. To calculate.
Note that gx and gy are values calculated in the equation (1). Then, the image block feature quantity calculation unit 420 estimates the edge direction θ of the region of interest from the calculated plurality of edge angles Θ. For example, the image block feature quantity calculation unit 420 calculates the average value of the obtained plurality of edge angles Θ, and sets the calculated average value as the estimated edge direction θ.

  In step 210 (S210), the image block feature quantity calculation unit 420 selects an edge pattern using the estimated edge direction θ and the pixel distribution pattern of the region of interest (feature block). The edge pattern is selected from a pattern table (FIG. 9) prepared in advance according to the edge direction and the pixel distribution pattern.

  In step 212 (S212), the high-quality image block generation unit 430 uses the enhancement kernel 434 (FIG. 11) in which the size and the weighting coefficient are set according to the enlargement ratio, and the image cut out by the image block setting unit 410 Contrast emphasis processing is performed on the image data of the attention area (feature block) and its peripheral area in the block.

  In step 214 (S214), the high-quality image block generation unit 430, for the same color block determined by the image block feature quantity calculation unit 420, the edge pattern of the attention area obtained by the image block feature quantity calculation unit 420 and the attention pattern. Using the estimated edge direction θ of the region and the pixel values in the attention region and the peripheral region subjected to contrast enhancement in S212, the calculation formulas (FIGS. 12 and 13) corresponding to the edge pattern and the estimated edge direction θ are used. Pixel values (p0 to p8) corresponding to 3 × 3 image blocks are generated. Further, the high-quality image block generation unit 430 uses the reference pixels (r0 to r13) selected based on the generated pixel values (p0 to p8) and the estimated edge direction θ, and the calculation formula illustrated in FIG. Thus, an enlarged image block for the attention area is generated.

  In step 216 (S216), the high-quality image block generation unit 430 calculates a calculation formula (FIG. 12) different from the calculation formula (FIG. 12, FIG. 13) for the same color block for the boundary block determined by the image block feature value calculation unit 420. 16 (B)), pixel values corresponding to 3 × 3 image blocks are generated. Further, the high-quality image block generation unit 430 generates an enlarged image block using the calculation formula shown in FIG. 15 and corrects the pixel values of one or more pixels.

  In step 218 (S218), the image enlargement program 4 determines whether or not the processing from S200 to S216 has been completed for all input image data, and if it is determined that the processing has not been completed, S200 Returning to the process, the process for the next pixel block is performed, and if it is determined that the process has been completed for all input image data, the process proceeds to S220.

In step 220 (S220), the high-speed enlargement processing unit 440 inputs the input image data stored in the storage unit 400 for each image or every several lines, and performs enlargement processing by the nearest neighbor interpolation method.
In this example, the high-speed enlargement processing unit 440 performs the enlargement process after the enlargement process (the processes from S200 to S216) by the high-quality image block generation unit 430 is completed. The enlargement process may be performed in parallel with the enlargement process by 430, or the enlargement process may be performed before the enlargement process by the high-quality image block generation unit 430.
In step 222 (S222), the enlarged image generation unit 450 integrates the enlarged image block generated by the high-quality image block generation unit 430 and the enlarged image enlarged by the high-speed enlargement processing unit 440.

  As described above, the image processing apparatus 2 according to the present embodiment calculates the feature amount of an image area having a predetermined size including the target pixel, and the calculated feature amount of the image area satisfies a predetermined condition. The image area is determined based on the color information of each pixel in the image area, and the enlarged image area corresponding to the image area is determined based on the calculated feature value of the image area and the determination result for the image area. And generating an enlarged image of the input image by applying an enlargement method different from this generation, and generating an enlarged image for the input image based on the generated enlarged image region and the generated enlarged image .

  Through such enlargement processing, the image processing apparatus 2 according to the present embodiment applies the high-quality enlargement processing only to characteristic portions (feature blocks), thereby suppressing the overall processing load and Since the high-quality enlargement process is applied to the portion, the feature amount of the characteristic portion can be stored, and a high-quality enlarged image in which image quality defects such as blur and jaggy are suppressed can be obtained at high speed. Further, the image processing apparatus 2 according to the present embodiment can suppress image quality defects such as undershoot and overshoot around the edge portion while maintaining the sharpness and smoothness of the edge.

It is a figure which illustrates the hardware constitutions of the image processing apparatus 2 with which the image processing method concerning this invention is applied centering on the control apparatus 20. FIG. It is a figure which shows the function structure of the image expansion program 4 which is performed by the control apparatus 20 and implement | achieves the image processing method concerning this invention. The edge strength calculation unit 422 exemplifies a region determined as a feature block by comparing the edge strength G with a reference value. FIG. 4A is a diagram illustrating an edge portion in an image, a feature block, and pixel values of the feature block. FIG. 4A is a diagram illustrating an edge portion in the image, and FIG. 4A illustrates an image area determined as a feature block (feature block 1, feature block 2, and feature block 3) in the image illustrated in FIG. 4A, and FIG. 4C illustrates pixel values of the feature blocks. FIG. FIG. 5A is a diagram illustrating a specific example of a region of interest and a peripheral region and an example of an edge direction of the region of interest. FIG. 5A illustrates the region of interest and the peripheral region, and FIG. It is a figure which illustrates the estimated edge direction. It is a flowchart which shows the edge direction estimation process (S10) by the edge direction estimation part 426. It is a figure which illustrates the reference field selected in processing of S102 of edge direction presumption processing (S10). It is explanatory drawing of an example of the estimated edge direction in the attention area | region shown in FIG. It is a figure which illustrates the edge pattern table used by the edge pattern selection part 428. It is a figure explaining the selection method of the edge pattern corresponding to the attention area | region shown in FIG. It is a figure which illustrates the emphasis kernel 434 (edge emphasis kernel) used by the high quality image block generation part 430. It is explanatory drawing of the specific example of the production | generation process of the enlarged image block in the high quality image block production | generation part 430. It is explanatory drawing of an example of the calculation formula used in the case of the combination of an edge pattern and an estimated edge direction. It is explanatory drawing of the selection method of the reference pixels r0-r13 based on the estimated edge direction in an attention area. It is explanatory drawing of the specific example of a production | generation process of the enlarged image block of 4x4 pixel. It is explanatory drawing of the specific example of the production | generation process of the enlarged image block in the boundary block process part 432, and a pixel value correction process. 4 is an explanatory diagram of a specific example in which an enlarged image block of 4 × 4 pixels generated by a high-quality image block generation unit 430 and an enlarged image output from a high-speed enlargement processing unit 440 are integrated. FIG. It is a flowchart (S20) which shows the whole operation | movement of the image processing apparatus 2.

Explanation of symbols

2 ... Image processing device 10 ... Printer device 20 ... Control device 202 ... CPU
204 ... Memory 22 ... Communication device 24 ... Storage device 26 ... UI device 4 ... Image enlargement program 400 ... Storage unit 410 ... Image block setting unit 420 ... Image block Feature amount calculation unit 422 ... Edge strength calculation unit 424 ... Boundary block determination unit 426 ... Edge direction estimation unit 428 ... Edge pattern selection unit 430 ... High quality image block generation unit 432 ... Boundary block processing units 434a, 434b ... enhancement kernel 440 ... high-speed enlargement processing unit 450 ... enlarged image generation unit

Claims (18)

An image processing apparatus for enlarging an input image,
For an image area in which the feature amount calculated for an image area of a predetermined size including the target pixel is equal to or greater than a predetermined reference value, the relationship between the color elements of each pixel in the image area is the same An image area determination unit that determines an area as the same color block, and determines that the image area is a boundary block when the relationship between the color elements of each pixel in the image area is different;
For the image area determined as the same color block by the image area determination unit, to calculate the pixel values by applying a first operational expression corresponding to the gradation change direction and default pixel value pattern of the image areas A first enlarged image area generating means for generating an enlarged image area corresponding to the image area;
The gradation change direction in the image area for the image area determined as the boundary block by the image area determination means so that the edge of the image area stands out from the enlargement method used in the first enlarged image area generation means And applying a second arithmetic expression obtained by changing the first arithmetic expression to a pixel position corresponding to a predetermined pixel value pattern, thereby generating a magnified image area corresponding to the image area An image processing apparatus comprising: a second enlarged image region generation unit. An enlargement processing unit that generates an enlarged image of an input image by applying an enlargement method that has a smaller processing load than the enlargement method used in the first enlarged image region generation unit and the second enlarged image region generation unit;
Generating an enlarged image for the input image based on the enlarged image region generated by the first enlarged image region generating unit and the second enlarged image region generating unit and the enlarged image generated by the enlargement processing unit The image processing apparatus according to claim 1, further comprising: The image area determination means determines an image area having a feature amount equal to or greater than a predetermined reference value, using the gradation change amount in the image area calculated based on each pixel value in the image area as a feature amount of the image area. The image processing apparatus according to claim 1. The first enlarged image region generating unit and the second enlarged image region generating unit generate an enlarged image region based on a gradation change direction in the image region calculated from each pixel value in the image region. The image processing apparatus according to 1. The first enlarged image area generating means and the second enlarged image area generating means are enlarged based on a predetermined pixel value pattern corresponding to a gradation change in the image area calculated from each pixel value in the image area. The image processing device according to claim 1, wherein an image region is generated. The first enlarged image region generating unit and the second enlarged image region generating unit are configured to select one color component selected from a plurality of color components in a color space based on a feature amount calculated for each color component. The image processing apparatus according to claim 1, further generating an enlarged image area based on the calculated feature amount. The image processing apparatus according to claim 1, wherein the image area determination unit determines the image area based on a magnitude relationship between color component values of pixels in the image area. The first enlarged image region generation unit and the second enlarged image region generation unit use the feature amount of the image region, the feature amount of the neighborhood region of the image region, and the pixel value in the neighborhood region. The image processing apparatus according to claim 1, wherein an enlarged image region is generated. The first enlarged image area generating unit and the second enlarged image area generating unit perform edge enhancement processing for emphasizing a change in gradation according to an enlargement ratio to correct a pixel value in the image area and perform correction. The image processing apparatus according to claim 1, wherein an enlarged image region is generated using the pixel values that have been set. The first enlarged image region generation unit and the second enlarged image region generation unit select a pixel value in the vicinity region of the image region according to the gradation change direction, and use the selected pixel value. The image processing apparatus according to claim 4, wherein an enlarged image area is generated. The second enlarged image area generation unit selects a pixel in the enlarged image area based on the feature amount calculated for the image area, and corrects the pixel value of the selected pixel using a predetermined calculation method. The image processing apparatus according to claim 1. The image processing apparatus according to claim 2, wherein the enlargement processing unit applies an enlargement method based on at least a nearest neighbor interpolation method. The image processing apparatus according to claim 2, wherein the enlargement processing unit applies an enlargement method based on at least a linear interpolation method. The image processing apparatus according to claim 2, wherein the enlargement processing unit generates an enlarged image for each input image or for every several lines in the input image. The magnified image generating means sets the magnified image area generated by the first magnified image area generating means and the second magnified image area generating means to a position corresponding to the magnified image generated by the magnification processing means. The image processing apparatus according to claim 2. The enlarged image generation means converts each pixel value on the enlarged image generated by the enlargement processing means to an enlarged image area generated by the first enlarged image area generation means and the second enlarged image area generation means. The image processing apparatus according to claim 15 , wherein the pixel value is replaced with each pixel value. The enlarged image generating unit is configured to detect a plurality of enlarged image regions generated by the first enlarged image region generating unit and the second enlarged image region generating unit when overlapping portions of the enlarged image regions. The image processing apparatus according to claim 2, wherein an average value of a plurality of pixel values corresponding to the same pixel is calculated. In an image processing apparatus that includes a computer and performs an enlargement process of an input image,
For an image area in which the feature amount calculated for an image area of a predetermined size including the target pixel is equal to or greater than a predetermined reference value, the relationship between the color elements of each pixel in the image area is the same An image area determination step for determining an area as the same color block, and determining that the image area is a boundary block if the relationship between the color elements of each pixel in the image area is different;
For the determined image areas with the same color block, by calculating the pixel values by applying a first operational expression corresponding to the gradation change direction and default pixel value pattern of the image area, to the image area A first enlarged image region generating step for generating a corresponding enlarged image region;
The gradation change direction in the image area and the predetermined pixel value pattern for the image area determined to be a boundary block so as to make the edge of the image area stand out from the enlargement method used in the first enlarged image area generation step. A second enlarged image area that generates an enlarged image area corresponding to the image area by calculating a pixel value by applying a second arithmetic expression obtained by changing the first arithmetic expression for a pixel position corresponding to A program for causing a computer of the image processing apparatus to execute the generating step.

Images