JP2010212782A - Image sharpening device and image sharpening method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To sharpen a defocused image by suppressing emphasis on ringing. <P>SOLUTION: A decision unit 101 determines whether each pixel of an input image belongs to an edge area. A range calculation unit 103 sets pixel positions determined to belong to the edge area as positions of interest in order, and calculates a limitation range from a distribution of peripheral pixel values of a temporarily sharpened image at the positions of interest. A conversion unit 104 sets positions of pixels determined to belong to the edge area of the input image as positions of interest and pixel values as reference pixel values, and multiplies the peripheral pixel values of the temporarily sharpened image at the positions of interest by a weight coefficient corresponding to a distance to calculate trial pixel values of the reference pixel values. The conversion unit 104 also corrects the peripheral pixel values of the temporarily sharpened image at the positions of interest so that differences between the reference pixel values and trial pixel values are small and fall within a limitation range. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image sharpening device that sharpens an image and an image sharpening method.

  An image taken with a digital camera or a moving image displayed on a television may be blurred due to defocusing or encoding processing. In order to convert a blurred image or video in an easy-to-view manner, the blur is removed by image enhancement processing or original image restoration processing. For example, a technique is disclosed in which image data is classified into three types of pixels, contour pixels, noise pixels, and non-edge pixels, and corresponding processing is performed. For example, Patent Document 1 discloses a method of performing edge enhancement processing by determining edge enhancement coefficients corresponding to the number of contour pixels.

Japanese Patent No. 2790604

  However, the above-described conventional technique has a problem that the ringing generated around the edge is emphasized along with the edge enhancement processing.

  In order to solve the above problems, the present invention provides an image sharpening device and an image sharpening method performed by the image sharpening device.

  According to the image sharpening device, the image sharpening method, and the image display device of the present invention, it is possible to sharpen a blurred image while suppressing ringing around the edge.

1 is a block diagram of an image processing apparatus according to a first embodiment. 2 is a flowchart of the image processing apparatus according to the first embodiment. The figure explaining the method of calculating a restriction | limiting range. The block diagram of the image processing apparatus of 2nd Embodiment. 6 is a flowchart of an image processing apparatus according to a second embodiment. The block diagram of the image processing apparatus of 3rd Embodiment. The figure explaining the self-congruity of an image. 10 is a flowchart of an image processing apparatus according to a third embodiment. The block diagram of the image processing apparatus of 4th Embodiment. 10 is a flowchart of an image processing apparatus according to a fourth embodiment. The block diagram of the image processing apparatus of 5th Embodiment. 10 is a flowchart of an image processing apparatus according to a fifth embodiment. The figure explaining the image display apparatus of 6th Embodiment.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same code | symbol is attached | subjected to the structure and process which mutually perform the same operation | movement, and the overlapping description is abbreviate | omitted.

[First Embodiment]
FIG. 1 is a block diagram of an image sharpening apparatus according to this embodiment. The image sharpening apparatus according to the present embodiment includes a determination unit 101, a storage unit 102, a range calculation unit 103, and a conversion unit 104.

  The storage unit 102 stores an input image and a temporarily sharpened image that uses each pixel value of the input image as an initial value. The storage unit 102 is a readable / writable storage medium such as a DRAM (memory) of a PC (personal computer), an HDD (hard disk), or an LSI frame memory. A pixel of the temporarily sharpened image corresponding to the pixel of the input image is set as a corresponding pixel. “Corresponding” indicates a relationship between pixels that are relatively in the same position in the image space.

  The determination unit 101 sequentially selects each pixel of the input image stored in the storage unit 102, and to which type of region the pixel is classified based on a change in the pixel value in the peripheral region including the selected pixel. Determine. As the type of the area to be classified, for example, it is determined whether the selected pixel belongs to an edge area where a pixel value change occurs linearly in the surrounding area or a non-edge area other than the edge area. . In addition, a method of determining an edge region, a texture region having a large number of pixels having a large change in pixel value in the local region, and a flat region having a small change in pixel value in the local region may be used. The area information that is the determination result is sent to the range calculation unit 103 and the conversion unit 104.

  The range calculation unit 103 limits the pixel value of the temporarily sharpened image included in the peripheral region of the corresponding pixel of the temporarily sharpened image corresponding to the edge pixel determined to belong to the edge region as a reference. The lower limit value of the range is set, and the upper limit value of the limit range is set based on the maximum pixel value. Pixel values included between the lower limit value and the upper limit value are pixel values that fall within the limit range.

  The conversion unit 104 repeats the sharpening process for each pixel of the temporarily sharpened image, and sequentially rewrites the pixel values of the temporarily sharpened image stored in the storage unit 102. The following sharpening process is performed on the pixels included in the peripheral area of the corresponding pixel of the temporarily sharpened image corresponding to the edge pixel of the input image. In the sharpening process, first, (a) a trial calculation pixel value is calculated by multiplying each pixel value of each pixel included in the peripheral area of the corresponding pixel by a weighting factor corresponding to the distance from the corresponding pixel. The weighting factor is a predetermined weighting factor according to the distance determined by the degree of blur, the imaging system, and the like, and uses, for example, a Gaussian function. Next, (b1) the pixel value of the edge pixel and the pixel value after conversion of each pixel included in the peripheral area of the corresponding pixel, rather than the difference between the pixel value of the edge pixel and the trial calculation pixel value calculated in (a) The pixel value after conversion of each pixel included in the peripheral area of the corresponding pixel is obtained so that the difference amount from the trial calculation pixel value obtained in the above calculation is smaller. Next, (b2) the pixel value stored in the storage unit 102 is rewritten to the pixel value corrected so that the calculated pixel value of each pixel is within the limit range. In the correction in (b2), among the converted pixel values derived in (b1), for the pixel values exceeding the limit range, the upper limit value or the lower limit value of the limit range is clipped to the closer pixel value. Process.

  When the determination unit 101 also determines whether the image belongs to the texture area, the conversion unit 104 determines whether the pixel included in the peripheral area of the corresponding pixel of the temporary sharpened image corresponding to the texture pixel of the input image. The following sharpening process is performed. The conversion unit 104 performs a trial calculation by multiplying the pixel value of the pixel included in the peripheral region of the corresponding pixel of the temporarily sharpened image corresponding to the texture pixel determined to belong to the texture region by a weighting factor corresponding to the distance from the texture pixel. Pixel value is calculated. Next, it is obtained when the pixel value of the texture pixel and the pixel value after conversion of each pixel included in the peripheral area of the corresponding pixel are obtained from the difference amount between the pixel value of the texture pixel and the calculated trial pixel value. The converted pixel value of each pixel included in the peripheral region of the corresponding pixel is obtained so that the difference amount from the trial calculation pixel value becomes smaller.

  The corrected pixel value is obtained by adding and subtracting each pixel value around the corresponding pixel of the temporarily sharpened image, and the sharpening process is repeatedly performed on the corresponding pixel peripheral region of the edge pixels that are sequentially set, and is stored in the storage unit 102 Conversion processing of the pixel value of the temporarily sharpened image is performed.

  As a method of conversion to a pixel value that makes the difference between the estimated pixel value and the pixel value of the input image smaller, for example, the POCS method or the Iterative Back-Projection method (S. Park, et.al. “Super -Resolution Image Reconstruction: A Technical Overview, "IEEE Signal Processing Magazine, USA, IEEE, May 2003, p. 29, p. 31). Any other method for solving the minimization problem may be used.

  FIG. 2 is a flowchart for explaining an operation in which the image sharpening device of the present embodiment outputs a sharpened output image from an input image.

  First, the memory | storage part 102 memorize | stores the pixel value of each pixel of an input image (step S200).

  Next, the determination unit 101 sequentially selects each pixel of the input image, and determines to which type of region the pixel is classified based on a change in surrounding pixel values including the pixel (step S201). The determination unit 101 holds area information of the determination result.

  Next, the initial value of the temporarily sharpened image is set in the storage unit 102. As an initial value, an image having the same pixel value as the input image stored in S200 is copied and stored (step S202).

  Next, the range calculation unit 103 sequentially sets the edge pixels determined to belong to the edge region by the determination unit 101 as processing targets, and the peripheral pixel values in the temporarily sharpened image of the corresponding pixels of the set edge pixels From the distribution, a limit range that is a range of values that can be taken as pixel values at the time of pixel value correction in the subsequent stage is calculated (step S203).

  FIG. 3 is a diagram illustrating the distribution of pixel values around the corresponding pixel of the temporarily sharpened image. The vertical axis indicates the size of the pixel value, and the horizontal axis indicates the pixel position. FIG. 3A is a diagram for explaining a method of calculating the limit range. In FIG. 3A, the pixel having the maximum pixel value among the pixel values of the pixels included in the peripheral region 303 of the pixel position 302 of the corresponding pixel 301 of the edge pixel determined to belong to the edge region by the determination unit 101. The pixel value of 304 is set as the upper limit value. Further, the pixel value of the pixel 305 having the minimum pixel value is set as the lower limit value. The limit range 306 is from the lower limit value to the upper limit value. As will be described later, in order to calculate the corrected pixel value so as to be within the limit range, the conversion unit 104 corrects the corrected pixel value to a pixel value that exceeds the ringing around the edge (maximum pixel value / minimum pixel value). This prevents the ringing from being emphasized.

  FIG. 3B is a diagram for explaining a method for calculating the limit range by a method different from that in FIG. 3B, among the pixel values of the pixels included in the peripheral region 309 of the pixel position 308 of the corresponding pixel 307 of the edge pixel determined to belong to the edge region by the determination unit 101, the upper M-th (maximum pixel value) The upper limit value is the pixel value of the pixel 310 having the pixel value that is the Mth largest pixel value from. In addition, the pixel value of the pixel 311 having the lower Nth pixel value (the pixel value Nth smallest from the smallest pixel value) is set as the lower limit value. The limit range 312 is from the lower limit value to the upper limit value. According to the method of FIG. 3B, even if ringing occurs in the peripheral region 309 of the corresponding pixel 307 by providing a limited range in the same manner as in FIG. The effect of suppressing the enhancement of ringing in the sharpening process at 104 can be obtained. In addition, there is an effect of suppressing ringing itself included in the input image.

  Next, the conversion unit 104 converts the pixel value of the temporarily sharpened image stored in the storage unit 102 (step S204). Specifically, the edge pixels of the input image are sequentially set as pixels to be processed according to the area information from the determination unit 101, and the following processing is performed. First, (a) a calculated pixel value of a sum obtained by multiplying each pixel value of each pixel included in the peripheral area of the corresponding pixel by a weighting coefficient corresponding to the distance from the corresponding pixel is calculated. The weighting factor is a predetermined weighting factor according to the distance determined by the degree of blur, the imaging system, and the like, and uses, for example, a Gaussian function. Next, (b1) the pixel value of the edge pixel and the pixel value after conversion of each pixel included in the peripheral area of the corresponding pixel, rather than the difference between the pixel value of the edge pixel and the trial calculation pixel value calculated in (a) The pixel value after conversion of each pixel included in the peripheral area of the corresponding pixel is obtained so that the difference amount from the trial calculation pixel value obtained in the above calculation is smaller. Next, (b2) the pixel value stored in the storage unit 102 is rewritten to the pixel value corrected so that the calculated pixel value of each pixel is within the limit range. In the correction in (b2), among the converted pixel values derived in (b1), for the pixel values exceeding the limit range, the upper limit value or the lower limit value of the limit range is clipped to the closer pixel value. Process. A process of converting the pixel value of the temporarily sharpened image held in the storage unit 102 into the pixel value of the pixel included in the calculated peripheral region of the corresponding pixel is performed. Sequentially, edge pixels are set as processing targets, and the above conversion processing is repeated.

  In addition, when the determination unit 101 also determines whether the determination unit 101 belongs to the texture region, the conversion unit 104 has a periphery of the corresponding pixel of the temporarily sharpened image corresponding to the texture pixel determined to belong to the texture region. A trial calculation pixel value is calculated by multiplying the pixel value of the pixel included in the region by a weighting factor corresponding to the distance from the texture pixel. Next, it is obtained when the pixel value of the texture pixel and the pixel value after conversion of each pixel included in the peripheral area of the corresponding pixel are obtained from the difference amount between the pixel value of the texture pixel and the calculated trial pixel value. The converted pixel value of each pixel included in the peripheral region of the corresponding pixel is obtained so that the difference amount from the trial calculation pixel value becomes smaller. A process of converting the pixel value of the temporarily sharpened image held in the storage unit 102 into the pixel value of the pixel included in the calculated peripheral region of the corresponding pixel is performed. Sequentially, edge pixels and texture pixels are set as processing targets, and the above conversion processing is repeated.

  Next, the conversion unit 104 determines whether the conversion process has been performed for a predetermined number of repetitions (step S205). Specifically, the conversion unit 104 determines whether the conversion process using the edge pixels has been completed a predetermined number of times. If the determination result is NO, the process returns to step S203, and the range calculation unit 103 calculates the limited range from the converted temporarily sharpened image again, and converts the temporarily sharpened image for the corresponding pixel of the edge pixel set first. Process. If YES, the conversion process ends, and the temporarily sharpened image in the storage unit 102 is output as a sharpened image.

  In this way, it is determined whether the pixel belongs to the edge region from the degree of change in the pixel value around each pixel of the input image, and the distribution of the pixel values around the corresponding pixel of the temporarily sharpened image corresponding to the edge pixel. By setting the pixel value limit range during pixel value correction and performing sharpening processing, it is possible to obtain a sharpened image while suppressing ringing in the edge region.

[Second Embodiment]
FIG. 4 is a block diagram of the image sharpening apparatus of the present embodiment.

  The image sharpening apparatus includes a determination unit 101, a storage unit 102, a range calculation unit 403, and a conversion unit 104. The method of setting the limit range is different from the image sharpening device of the first embodiment shown in FIG. The image sharpening device in FIG. 1 calculates the limited range from the distribution of pixel values in the peripheral area of the corresponding pixel of the temporarily sharpened image. The image sharpening device according to the present embodiment is different in that the limit range is calculated from the distribution of pixel values in the peripheral region of the edge pixel of the input image.

  The range calculation unit 403 sequentially sets the edge pixels determined to belong to the edge region by the determination unit 101 as a processing target, and the limited range based on the minimum pixel value of the pixels of the input image included in the peripheral region of the edge pixel The lower limit value is set, and the upper limit value of the limit range is set based on the maximum pixel value.

  It is a flowchart for demonstrating the operation | movement in which the image sharpening apparatus of this embodiment outputs the output image sharpened from the input image. Since the processing excluding step S503 is the same as the processing in steps S200, S201, 2020, 204, and 205 in FIG. 2, details thereof are omitted.

  First, the storage unit 102 stores an input image (step S500).

  Next, the determination unit 101 sequentially selects each pixel of the input image, and determines to which type of region the pixel is classified based on a change in surrounding pixel values including the pixel (step S501). The determination unit 101 holds area information of the determination result.

Next, the initial value of the temporarily sharpened image is set in the storage unit 102. As an initial value, an image having the same pixel value as the input image stored in S500 is copied and stored (step S502).
Next, the range calculation unit 403 sequentially sets the edge pixels determined to belong to the edge region by the determination unit 101, and the restriction at the time of correcting the pixel value from the distribution of the pixel values of the input image around the set edge pixel The range is calculated (step S503). The limit range is set by the same method as in the example shown in FIG.

  Next, the conversion unit 104 converts the pixel value of the temporarily sharpened image (step S504). In step S504, the same conversion process as in step S204 in FIG. 2 is performed.

  Next, the conversion unit 104 determines whether the conversion process has been performed for a predetermined number of repetitions (step S505). Specifically, the conversion unit 104 determines whether or not the conversion process for the edge pixel has been completed a predetermined number of times, and if NO, the process returns to step S504 (corresponding to step S203 in the first embodiment). Then, the temporarily sharpened image is changed for the edge pixel set first. If YES, the conversion process ends, and the temporarily sharpened image in the storage unit 102 is output as a sharpened image.

  In this way, by performing the sharpening process by setting the restriction range at the time of pixel value correction from the distribution of the pixel values of the input image around the edge pixel determined to belong to the edge region, the edge region is faster. An image obtained by sharpening a blurred image can be output while suppressing ringing.

[Third Embodiment]
FIG. 6 is a block diagram of the image sharpening apparatus of the present embodiment.

  The image sharpening apparatus includes a determination unit 101, a storage unit 102, a range calculation unit 603, a conversion unit 604, and a position calculation unit 605. The image sharpening apparatus according to the first embodiment illustrated in FIG. 1 is different from the image sharpening apparatus according to the first embodiment in that the position calculation unit 605 is included in the processing performed by the range calculation unit 603 and the conversion unit 604.

  The position calculation unit 605 sequentially sets the edge pixels determined to belong to the edge region by the determination unit 101 as processing targets. In the input image, the corresponding position of the pixel decimal point accuracy, in which the change in the pixel value in the peripheral region of the set edge pixel and the change in the pixel value in the periphery thereof are estimated to be similar, is obtained. In other words, the position calculation unit 605 uses the “self-congruity” in the screen space (in the input image), and determines the corresponding position that can be estimated to have the same pixel value as the edge pixel with a decimal precision smaller than the pixel interval. calculate.

  The “self-congruity” will be described with reference to FIG. FIG. 7 is a diagram showing actual digital image data in which the horizontal axis indicates the abscissa indicating the position of the pixel in the screen space, and the vertical axis indicates the pixel value indicating the luminance. Different five rows of data in the edge region in the same image are indicated by different broken lines. As shown in FIG. 7, it can be seen that even in different rows in the same image, there are portions having very similar luminance changes. In this way, the property of an image having a similar luminance change within the same frame is defined as “self-congruity”. The position calculation unit 605 searches for a corresponding position with decimal precision (subpixel precision) smaller than the pixel interval around the edge pixel in the input image using self-congruity. A matching error interpolation method or an image interpolation method is used to search for a corresponding position with subpixel accuracy.

  The range calculation unit 603 sequentially sets the position of the temporarily sharpened image, which is the same position as the corresponding position in the input image, as the position of interest. The range calculation unit 603 sets the lower limit value of the limit range based on the minimum pixel value of the pixels included in the peripheral region of the target position in the temporarily sharpened image corresponding to the corresponding position, and sets the maximum pixel value as the reference The upper limit value of the limit range is set as Note that, similarly to the image sharpening device of the first embodiment in FIG. 1, the limit range is calculated and set for the edge pixels determined by the determination unit 101 to belong to the edge region.

  The conversion unit 604 performs the following processing in addition to the conversion processing for edge pixels. First, (c) a trial calculation pixel value is calculated by multiplying a pixel value of a pixel around the target position in the temporarily sharpened image corresponding to the corresponding position by a weighting coefficient corresponding to the distance from the target position. Next, (d1) a trial calculation pixel obtained when the pixel value of the edge pixel and the pixel value after conversion of each pixel around the target position are calculated based on the difference amount between the pixel value of the edge pixel and the trial calculation pixel value The pixel value after conversion of each pixel around the corresponding pixel is obtained so that the difference amount from the value becomes smaller. Next, (d2) the pixel value corrected so that the calculated pixel value of each pixel is within the limit range is written in the pixel value of the temporarily sharpened image.

  Correction pixel values are obtained by adding and subtracting the pixel values of the temporarily sharpened image around the position of interest, and the pixel values of the temporarily sharpened image held in the storage unit 102 are converted. The above conversion process is repeated for the attention positions set sequentially.

FIG. 8 is a flowchart for explaining an operation in which the image sharpening device of the present embodiment outputs a sharpened output image from an input image.
First, the memory | storage part 102 memorize | stores the pixel value of each pixel of an input image (step S800).

  Next, the determination unit 101 sequentially selects each pixel of the input image, and determines to which type of region the pixel is classified based on a change in surrounding pixel values including the pixel (step S801). The determination unit 101 holds area information of the determination result.

  Next, the initial value of the temporarily sharpened image is set in the storage unit 102. As an initial value, an image having the same pixel value as the input image stored in S800 is copied and stored (step S802).

Next, the determination unit 101 sequentially selects each pixel of the input image, and determines to which type of region the pixel is classified based on a change in surrounding pixel values including the pixel (step S801).
Next, a temporarily sharpened image having each pixel value of the input image as an initial value is stored in the storage unit 102 (step S802).
Next, the position calculation unit 605 sequentially sets the edge pixels determined to belong to the edge region by the determination unit 101 as processing targets, and it is estimated that the change in pixel value is similar to the first region including the target pixel. In the second area in the input image, the corresponding position of the decimal point precision of the pixel estimated to have the same pixel value as the target pixel is calculated (step S803).

  Next, in addition to the range calculation processing set by the range calculation unit 603 for the edge pixel determined to belong to the edge region by the determination unit 101, the temporary sharpness corresponding to the corresponding position calculated by the position calculation unit 605 is used. The attention position of the digitized image is sequentially set, and the restriction range at the time of pixel value correction is calculated from the distribution of pixel values around the attention position and set (step S804).

  Next, the conversion unit 604 converts the pixel value of the temporarily sharpened image stored in the storage unit 102 (step S805). The conversion unit 604 performs a sharpening process on the temporarily sharpened image corresponding to the edge pixel, as in step S204 of the first embodiment. Further, (c) a trial calculation pixel value is calculated by multiplying a pixel value of a pixel around the target position in the temporarily sharpened image corresponding to the corresponding position by a weighting coefficient corresponding to the distance from the target position. Next, (d1) a trial calculation pixel obtained when the pixel value of the edge pixel and the pixel value after conversion of each pixel around the target position are calculated based on the difference amount between the pixel value of the edge pixel and the trial calculation pixel value The pixel value after conversion of each pixel around the corresponding pixel is obtained so that the difference amount from the value becomes smaller. Next, (d2) the pixel value corrected so that the calculated pixel value of each pixel is within the limit range is written in the pixel value of the temporarily sharpened image.

  Next, the conversion unit 604 determines whether the conversion process has been performed for a predetermined number of repetitions (step S806). If a determination result is NO, it will return to Step S804. If YES, the conversion process ends, and the temporarily sharpened image in the storage unit 102 is output as a sharpened image.

  In this way, from the degree of change in the pixel value around each pixel of the input image, it is determined whether the pixel belongs to the edge region, and the corresponding position with decimal precision is calculated by self-congruity in the pixel in the edge region. Blur while suppressing ringing and jaggies in the edge area by setting the limiting range at the time of pixel value correction from the surrounding pixel value distribution in the temporarily sharpened image of the pixel position and corresponding position, and performing sharpening processing An image with a sharpened image can be output.

[Fourth Embodiment]
FIG. 9 is a block diagram of the image sharpening apparatus of this embodiment.

  The image sharpening apparatus includes a determination unit 101, a storage unit 902, a range calculation unit 103, a conversion unit 104, and a temporary enlargement unit 905. The image sharpening apparatus shown in FIG. 1 is different from the image sharpening apparatus in that a temporary enlargement unit 905 is provided.

  The temporary enlargement unit 905 calculates a pixel value of a temporary high-resolution image having a number of pixels larger than the number of pixels included in the input image by interpolation processing based on the pixel value of the input image. For example, a general linear filter process such as a cubic convolution interpolation method or a bilinear interpolation method is used.

  The storage unit 902 stores a temporarily sharpened image that uses the pixel value of the temporary high resolution image calculated by the temporary enlargement unit 905 as an initial value. Thereafter, the provisionally sharpened image in which the provisional high-resolution image is stored and the input image have the same image size although the number of pixels is different. That is, the temporarily sharpened image has pixels with higher density than the input image, and the pixel interval of the temporarily sharpened image is smaller than the pixel interval of the input image.

FIG. 10 is a flowchart for explaining the operation of the image sharpening apparatus of this embodiment.
First, the storage unit 902 stores an input image (step S1000).

  Next, the determination unit 101 sequentially selects each pixel of the input image, and determines which type of region the pixel is classified from based on changes in peripheral pixel values including the pixel (step S1001). The determination unit 101 holds area information of the determination result.

  Next, the temporary enlargement unit 905 calculates a pixel value of a temporary high-resolution image having a larger number of pixels than the number of pixels included in the input image by interpolation processing based on the pixel value of the input image (step S1002).

  Next, the initial value of the temporarily sharpened image is set in the storage unit 902. As an initial value, the temporary enlarged image calculated by the temporary enlargement unit 905 in S1002 is stored (step S1003).

  Next, the range calculation unit 103 sequentially sets the edge pixels determined to belong to the edge region by the determination unit 101 as processing targets, and from the distribution of pixel values around the position of the temporarily sharpened image corresponding to the edge pixels. A limit range at the time of pixel value correction is calculated and set (step S1004).

Next, the conversion unit 104 converts the pixel value of the temporarily sharpened image (step S1005).
Next, the conversion unit 104 determines whether the conversion process has been performed for a predetermined number of repetitions (step S1006). If a determination result is NO, it will return to Step S1004. If YES, the conversion process ends and the temporarily sharpened image in the storage unit 902 is output as a sharpened image.

  In this way, an image obtained by interpolating the input image is used as a temporarily sharpened image, and a sharpening process is performed by setting a restriction range at the time of pixel value correction. An image obtained by sharply enlarging the image can be generated.

[Fifth Embodiment]
FIG. 11 is a block diagram of the image sharpening apparatus of the present embodiment.

  The image sharpening apparatus includes a determination unit 101, a storage unit 1102, a range calculation unit 1103, a conversion unit 1104, a temporary enlargement unit 1105, and a position calculation unit 1106.

  The temporary enlargement unit 1105 calculates a pixel value of a temporary high-resolution image having a larger number of pixels than the number of pixels included in the input image by interpolation processing based on the pixel value of the input image.

  The storage unit 1102 stores a temporary sharpened image having the initial value of the pixel value of the temporary high resolution image calculated by the temporary enlargement unit 1105. Thereafter, the provisionally sharpened image in which the provisional high-resolution image is stored and the input image have the same image size although the number of pixels is different. That is, the temporarily sharpened image has pixels with higher density than the input image, and the pixel interval of the temporarily sharpened image is smaller than the pixel interval of the input image.

  The position calculation unit 1106 sequentially sets pixels determined to belong to the edge region by the determination unit 101 as a processing target, and in the input image in which the change in pixel value is estimated to be similar to the first region including the target pixel. In the second region, the corresponding position of the decimal point precision of the pixel estimated to have the same pixel value as the target pixel is calculated.

  The range calculation unit 1103, in addition to the restriction range setting process set from the edge pixels determined to belong to the edge region by the determination unit 101, from the corresponding position calculated by the position calculation unit 1106, the temporary position corresponding to the corresponding position. A restriction range at the time of pixel value correction is calculated and set from the distribution of pixel values around the position in the sharpened image.

  The conversion unit 1104 performs the following processing in addition to the conversion processing for edge pixels as the sharpening processing. (C) A trial calculation pixel value is calculated by multiplying a pixel value of a pixel around the target position in the temporarily sharpened image corresponding to the corresponding position by a weighting coefficient corresponding to the distance from the target position. Next, (d1) a trial calculation pixel obtained when the pixel value of the edge pixel and the pixel value after conversion of each pixel around the target position are calculated based on the difference amount between the pixel value of the edge pixel and the trial calculation pixel value The pixel value after conversion of each pixel around the corresponding pixel is obtained so that the difference amount from the value becomes smaller. Next, (d2) the pixel value corrected so that the calculated pixel value of each pixel is within the limit range is written in the pixel value of the temporarily sharpened image.

  FIG. 12 is a flowchart for explaining an operation in which the image sharpening apparatus of the present embodiment outputs an output image sharpened from an input image. First, the storage unit 1102 stores an input image (step S1200).

  Next, the determination unit 101 sequentially selects each pixel of the input image, and determines which type of region the pixel is classified from based on a change in surrounding pixel values including the pixel (step S1201). The determination unit 101 holds area information of the determination result.

  Next, the temporary enlargement unit 1105 calculates a pixel value of a temporary high-resolution image having a larger number of pixels than the number of pixels included in the input image by interpolation processing based on the pixel value of the input image (step S1202).

Next, the initial value of the temporary sharpened image is set in the storage unit 1102. As an initial value, the temporary enlarged image calculated by the temporary enlargement unit 1105 in S1002 is stored (step S1203).
Next, the position calculation unit 1106 sequentially sets pixels determined to belong to the edge region by the determination unit 101 as processing targets, and the input is estimated to be similar in pixel value change to the first region including the target pixel. In the second region in the image, the corresponding position of the decimal point accuracy of the pixel estimated to have the same pixel value as the target pixel is calculated (step S1204).

  Next, in addition to the range calculation process in which the range calculation unit 1103 sets the pixel position of the edge pixel determined to belong to the edge region by the determination unit 101, the corresponding position calculated by the position calculation unit 1106 is also sequentially processed. The limit range at the time of pixel value correction is calculated and set from the distribution of pixel values around the position in the temporarily sharpened image corresponding to the corresponding position (step S1205).

Next, the conversion unit 1104 converts the pixel value of the temporarily sharpened image (step S1206).
Next, the conversion unit 1104 determines whether the conversion process has been performed for a predetermined number of repetitions (step S1207).
According to the image processing apparatus of the present embodiment, the edge region ringing and jaggies are reduced by calculating the corresponding position with decimal precision by the self-congruity in the edge region pixels, setting the limit range, and performing the sharpening process. An image obtained by sharply enlarging a blurred image can be output while suppressing it.

[Sixth Embodiment]
FIG. 13 is a diagram illustrating the image display apparatus according to the present embodiment. The image display device includes an image sharpening unit 1301 and an image display unit 1302.

  The image sharpening unit 1301 has the same configuration as that shown in FIG.

  The image display unit 1302 displays the sharpened image obtained by the image sharpening unit 1301.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  In addition, the image sharpening device of each of the above embodiments can be realized, for example, by using a general-purpose computer device as basic hardware. The program to be executed has a module configuration including each function described above. The program is an installable or executable file that is recorded on a computer-readable recording medium such as a CD-ROM, CD-R, DVD, etc. Also good.

1301 ... Image sharpening unit, 101 ... Determination unit, 102, 902, 1102 ... Storage unit, 103, 403, 603, 1103 ... Range calculation unit, 104, 604, 1104 ... Conversion 905, 1105 ... Temporary enlargement unit, 605, 1106 ... Position calculation unit, 1302 ... Image display unit

Claims (8)

A storage unit for storing an input image and a temporary sharpened image having a pixel value of the input image as an initial value;
For each pixel of the input image, a determination unit that determines whether the pixel belongs to an edge region;
Distribution of pixel values of each pixel of the input image around the edge pixel determined to belong to the edge region, or pixel values of each pixel around the corresponding pixel of the temporarily sharpened image corresponding to the edge pixel A range calculation unit for calculating an upper limit value and a lower limit value of a pixel value in a limited range from the distribution;
(A) calculating a pixel value of a sum obtained by multiplying each pixel value of each pixel around the corresponding pixel by a weighting factor corresponding to a distance from the corresponding pixel;
(B1) Trial calculation obtained when the pixel value of the edge pixel and the pixel value after conversion of each pixel around the corresponding pixel are calculated based on the difference between the pixel value of the edge pixel and the trial calculation pixel value Find the converted pixel value of each pixel around the corresponding pixel so that the difference amount with the pixel value is smaller,
(B2) a conversion unit that writes the pixel value corrected so that the calculated pixel value of each pixel is within the limit range to the pixel value of the temporarily sharpened image;
An image sharpening device comprising:
The range calculation unit includes a pixel value of each pixel of the input image around the edge pixel determined to belong to the edge region, or each pixel around the corresponding pixel of the temporarily sharpened image corresponding to the edge pixel. The image sharpening apparatus according to claim 1, wherein the lower limit value is calculated based on a minimum pixel value among the pixel values of the first pixel value and the upper limit value is calculated based on a maximum pixel value.
The image sharpening apparatus according to claim 2, wherein the range calculation unit sets the maximum pixel value as the upper limit value, and sets the minimum pixel value as the lower limit value.
The range calculation unit sets the Mth largest pixel value from the maximum pixel value among the pixel values of the surrounding pixels as the upper limit value, and sets the Nth smallest pixel value from the minimum pixel value as the lower limit value. The image sharpening device according to claim 2, wherein:
The determination unit also determines whether each pixel of the input image belongs to a texture region,
The conversion unit performs a trial calculation by multiplying a pixel value of a pixel around a corresponding pixel of the temporarily sharpened image corresponding to a texture pixel determined to belong to a texture region by a weighting factor corresponding to a distance from the corresponding pixel. Calculated pixel obtained by calculating a pixel value and calculating the converted pixel value of each pixel included in the peripheral area of the corresponding pixel, rather than the difference between the pixel value of the texture pixel and the estimated pixel value Converting the pixel value of the temporarily sharpened image into the pixel value after the conversion so that the difference amount with the value is smaller,
The image sharpening apparatus according to claim 3 or 4,
There is further provided a position calculation unit for obtaining, in the input image, a corresponding position of the decimal point accuracy of a pixel that is estimated to be similar to the change in the pixel value in the region including the edge pixel in the peripheral region. And
The range calculation unit calculates the lower limit value based on a minimum pixel value of a pixel in the input image around the corresponding position or a pixel in the vicinity of the temporarily sharpened image corresponding to the corresponding position. And calculating the upper limit value based on the maximum pixel value,
The converter is
(C) calculating a trial pixel value of a sum obtained by multiplying each pixel value of each pixel around the target position in the temporarily sharpened image corresponding to the corresponding position by a weighting coefficient corresponding to the distance from the target position. ,
(D1) Trial calculation obtained when the pixel value of the edge pixel and the pixel value after conversion of each pixel around the target position are calculated based on the difference amount between the pixel value of the edge pixel and the trial calculation pixel value Find the converted pixel value of each pixel around the corresponding pixel so that the difference amount with the pixel value is smaller,
(D2) The pixel value corrected so that the calculated pixel value of each pixel is within the limit range is written in the pixel value of the temporarily sharpened image. The image sharpening apparatus according to claim 1.
The input image further includes a temporary enlargement unit that calculates a pixel value of a high-resolution image having a larger number of pixels than the number of pixels of the input image,
The image sharpening apparatus according to claim 1, wherein the storage unit stores the temporarily sharpened image having a pixel value of the temporary high resolution image as an initial value.
Storing an input image and a temporarily sharpened image having a pixel value of the input image as an initial value in a storage unit;
A determination step of determining, for each pixel of the input image, whether the pixel belongs to an edge region;
Distribution of pixel values of each pixel of the input image around the edge pixel determined to belong to the edge region, or pixel values of each pixel around the corresponding pixel of the temporarily sharpened image corresponding to the edge pixel A range calculating step for calculating an upper limit value and a lower limit value of a pixel value of the limited range from the distribution;
(A) calculating a pixel value of a sum obtained by multiplying each pixel value of each pixel around the corresponding pixel by a weighting factor corresponding to a distance from the corresponding pixel;
(B1) Trial calculation obtained when the pixel value of the edge pixel and the pixel value after conversion of each pixel around the corresponding pixel are calculated based on the difference between the pixel value of the edge pixel and the trial calculation pixel value Find the converted pixel value of each pixel around the corresponding pixel so that the difference amount with the pixel value is smaller,
(B2) a conversion step of writing the pixel value corrected so that the calculated pixel value of each pixel is within the restriction range into the pixel value of the temporarily sharpened image;
An image sharpening device comprising:

Images