JP4133746B2 - Interpolated pixel value calculation method, apparatus, and program - Google Patents

Description

The present invention relates to an interpolation pixel value calculation method, apparatus and program therefor for interpolating an interpolation pixel between pixels constituting the digital image.

  Conventionally, a digital image obtained by photoelectrically reading a photographic image recorded on a photographic film such as a negative film or a color reversal film with a reading device such as a scanner, a digital still camera (DSC), a mobile phone with a camera, etc. 2. Description of the Related Art Image data obtained by imaging a subject with a digital imaging device is enlarged or reduced so as to match the size of a reproduction device such as a monitor that reproduces the image data. For example, when image data obtained by imaging a subject with a camera-equipped mobile phone is transmitted as an e-mail, the image data is enlarged or reduced according to the size of the liquid crystal monitor provided in the destination mobile phone. Service is provided.

  Such enlargement / reduction of the image data is performed by interpolating new pixels (hereinafter referred to as interpolation pixels) between the pixels constituting the image represented by the image data in accordance with the enlargement / reduction ratio. As such an interpolation method, various methods such as a linear interpolation method, nearest neighbor method, Gaussian filter, bilinear method, and bicubic method are known. In these methods, a pixel in the vicinity of the interpolation pixel is used as a reference pixel, and an interpolation calculation is performed using the pixel value of the reference pixel to calculate the pixel value of the interpolation pixel.

  However, when the image data is enlarged / reduced using only a single method, there is a problem that blur or shaggy occurs in an edge portion included in the image. For this reason, a method has been proposed in which an edge component included in an image is detected and an interpolation operation is performed by different processing between the edge portion and the non-edge portion (see Patent Document 1).

  In order to detect an edge from an image in the method described in Patent Document 1, a Sobel filter or a Laplacian filter is generally used. Such a Sobel filter or Laplacian filter has an odd number of taps with a minimum of 3 taps, and can detect an edge by determining whether or not the pixel of interest is an edge by filtering.

  On the other hand, when an image is enlarged or reduced, an interpolation pixel is interpolated between pixels, so it is necessary to determine whether or not an edge is determined between pixels instead of pixels included in the image. When an edge is detected using the above-described filter having an odd number of taps, it is only possible to determine whether or not the pixel of interest in the image is an edge, and whether or not there is an edge between pixels. Cannot judge.

  The simplest method for determining whether or not there is an edge between pixels is to perform filtering processing using, for example, a difference filter shown in FIG. A method of obtaining a difference and determining that an edge exists between two adjacent pixels when the absolute value of the difference is equal to or greater than a predetermined threshold value is conceivable. In the following description of the present invention, such an edge detection method is referred to as a first edge detection method.

  Whether or not an edge exists between two pixels located at the center of the four pixels is determined based on the pixel values of the four pixels adjacent in series as well as the pixel values of the two adjacent pixels. It is also possible to use a technique to Specifically, for example, in determining whether an edge exists between two pixels G2 and G3 located at the center of four pixels G1, G2, G3, and G4 adjacent in series shown in FIG. First, with respect to the pixels G1 to G4, a filtering process using a difference filter is performed on three pixel pairs (G1, G2), (G2, G3), (G3, G4) including two adjacent pixels. Differences between the paired pixels are calculated as primary differences d1, d2, and d3. Subsequently, filtering processing by the difference filter is similarly performed on two adjacent primary difference pairs (d1, d2) and (d2, d3) in the three primary differences d1, d2, and d3, and each 1 The difference of the next difference pair is calculated as secondary differences d4 and d5. There are 18 types of combinations of the positive and negative of the primary difference and the secondary difference calculated for four pixels adjacent in series in the image. FIGS. 6, 7 and 8 show these 18 types of combinations, The relationship with the profile shape of these four pixels is shown. Among them, there are two combinations of the edge 1 and the edge 2 shown in FIG. 6 that indicate that an edge exists between two adjacent pixels G2 and G3. Edge 1 is a right rising edge such that (d1, d2, d3, d4, d5) = (+, +, +, +, −) and (d1, d2, d3, d4, d5) = (−, −, There are two types of rising edges that are-,-, +), and edge 2 has a downwardly convex shape where (d1, d2, d3, d4, d5) = (+, +, +, +, +) Upwardly rising edge where (d1, d2, d3, d4, d5) = (+, +, +, −, −), (d1, d2, d3, d4, d5) = ( Downward convex left rising edge that becomes −, −, −, +, +), and upward convex leftward that becomes (d1, d2, d3, d4, d5) = (−, −, −, −, −) There are four types of rising edges. The primary differences d1, d2, d3 and the secondary differences d4, d5 are obtained for four pixels adjacent in series, and the positive / negative relationship between these differences is the relationship of edge 1 or edge 2 shown in FIG. In the case where it is determined that an edge exists between two adjacent pixels G2 and G3, and the positive / negative relationship between these differences is a mountain, valley, or other relationship as shown in FIGS. In addition, it is determined that no edge exists between two adjacent pixels G2 and G3. In the following description of the present invention, such an edge detection method is referred to as a second edge detection method.

  Furthermore, it is possible to detect whether or not an edge exists between two adjacent pixels with higher accuracy by using an edge detection method that is a combination of the first and second edge detection methods described above. Specifically, as in the second method, primary differences d1, d2, d3 and secondary differences d4, d5 are obtained for four pixels G1 to G4 adjacent in series, and the positive and negative of these differences are obtained. When the relationship is the relationship of edge 1 or edge 2 shown in FIG. 6, an edge exists between two adjacent pixels G2 and G3, and the positive / negative relationship of these differences is as shown in FIG. In the case of a relationship such as a mountain, a valley, or the like, the first determination is performed such that no edge exists between the two adjacent pixels G2 and G3. In the first determination, when it is determined that an edge exists between two adjacent pixels G2 and G3, whether or not the absolute value of the difference between the pixel values of the pixels G2 and G3 is equal to or greater than a predetermined threshold value. It is further determined (second determination), and it is determined that a true edge exists between the pixels G2 and G3 only when the second determination is affirmed. By doing so, when the difference between the pixel values of the pixels G2 and G3 is very small and can be regarded as noise, it is possible to prevent erroneous determination as an edge. In the following description of the present invention, such an edge detection method is referred to as a third edge detection method.

  Further, as an improvement method of the first edge detection method, when an absolute value of a difference between two adjacent pixels is equal to or greater than a predetermined threshold (first threshold), an edge exists between the two pixels. In addition, the absolute value of the difference between two adjacent pixels is smaller than the first threshold, but is greater than or equal to a threshold smaller than the first threshold (second threshold), and the two adjacent pixels Between the two pixels adjacent to the center when the absolute value of the absolute value of the difference of each pixel pair consisting of two adjacent pixels included in four or more pixels adjacent in series with the center as the center It is also possible to detect an edge by determining that there is an edge. By doing so, not only a sharp edge but also a relatively gentle edge can be detected. In the following description of the present invention, such an edge detection method is referred to as a fourth edge detection method.

  As described in these examples, various methods are conceivable for determining whether or not an edge exists between pixels constituting an image and detecting an edge between pixels of the image. When interpolating interpolated pixels between pixels constituting an image to scale the image, different reference pixels depend on whether or not an edge exists between pixels near the interpolated pixel and in which direction the edge extends Or selecting the pixel value of the interpolated pixel using a different interpolation calculation method can prevent the image quality of the enlarged / reduced image from being degraded. For example, when no edge is detected between pixels in the vicinity of the interpolation pixel, all pixels located in the vicinity of the interpolation pixel (for example, 4 × 4 16 pixels surrounding the interpolation pixel) are set as the reference pixels, and the pixels of the reference pixels While calculating the pixel value of the interpolated pixel by performing an interpolation operation on the value, if an edge is detected between the pixels in the vicinity of the interpolated pixel, it is determined that the interpolated pixel exists on either side of the edge In addition, among the pixels in the vicinity of the interpolation pixel, the pixel value of the interpolation pixel is obtained using only the pixel located on the same side as the interpolation pixel with respect to the edge as a reference pixel, thereby reducing the image quality of the enlarged / reduced image. Possible ways to prevent it. Of course, different interpolation calculation methods may be used depending on whether an edge is detected between pixels in the vicinity of the interpolation pixel as well as the reference pixel.

  By the way, in a digital image obtained by photoelectrically reading an image recorded on a photographic film or a digital image obtained by imaging a subject with a digital imaging device, a sampling process is performed to digitize an analog image signal. A pixel representing a portion including an edge (hereinafter referred to as an edge portion) in the subject is a pixel indicating an intermediate value of signal values of portions on both sides of the edge (hereinafter, such a pixel is referred to as an intermediate value pixel). For example, in the X direction of the digital image of the subject shown in FIG. 33A, the pixel configuration is as shown in FIG. 33B. Of these pixels, the edge L of the subject shown in FIG. The pixel Gb representing the portion including the pixel value indicates an intermediate signal value between the signal value on the left side of the edge L and the signal value on the right side of the edge L.

  Therefore, for example, as described above, it is detected whether or not there is an edge between pixels in the vicinity of the interpolation pixel. If there is, the detected edge among the pixels located in the vicinity of the interpolation pixel is detected. Thus, when only the pixel located on the same side as the interpolation pixel is used as the reference pixel and the pixel value of the interpolation pixel is obtained by performing the interpolation calculation using the pixel value of the reference pixel, if the intermediate value pixel is included among the reference pixels Since the pixel value of the interpolated pixel is affected by the pixel value of the intermediate value pixel, there is a problem in that blurring remains in the enlarged / reduced image, and in the worst case, the image becomes wider than the original image.

  Therefore, in order to improve the image quality of the enlarged / reduced image, when obtaining the pixel value of the interpolation pixel, the intermediate value pixel is not used, and only the reference pixel that is not the intermediate value pixel is used to obtain the pixel value of the interpolation pixel. It is possible.

Alternatively, the intermediate value pixel is used as the reference pixel, but the reference pixel that is the intermediate value pixel is enlarged / reduced even when the interpolation calculation is performed with a weight smaller than the reference pixel that is not the intermediate value. It is possible to reduce the residual, spread, etc. of the blur due to the intermediate value pixels in the image.
JP 2002-312020 A

  However, for example, when the subject in the example shown in FIG. 33A continuously extends in the direction orthogonal to the x direction (referred to as the y direction) in the drawing (see FIG. 35A). As shown in FIG. 35B, an edge A and an edge B extending in the y direction are detected between the pixel Ga1 and the pixel Gb1, and between the pixel Gb1 and the pixel Gb2, respectively. In this case, for example, when the interpolation pixel G (see FIG. 35C) is interpolated at a position near the pixel Gb2 between the pixel Gb2 between the pixels Gb3 and Gb1 and the pixel Gc2 between the pixels Gc3 and Gc1. When a pixel located on the same side as the interpolation pixel G with respect to the edge is used as the reference pixel, there are only three pixels Gb1, Gb2, and Gb3 sandwiched between the edge A and the edge B that can be the reference pixel. . By the way, since these three pixels are intermediate value pixels, if these three pixels are removed from the reference pixel, the reference pixel disappears and the pixel value of the interpolation pixel G cannot be obtained. In addition, when an intermediate value pixel is used as a reference pixel, a method of performing an interpolation operation with a weight smaller than a reference pixel that is not an intermediate value is applied to a reference pixel that is an intermediate value pixel. Since the pixel is an intermediate value pixel, the interpolation pixel G whose pixel value is obtained from the pixel values of these reference pixels is also an intermediate value pixel, and reduces blurring caused by the intermediate value pixel in the enlarged / reduced image. I can't.

The present invention has been made in view of the above circumstances, and adjust the detected edge between pixels of the digital image, by calculating the pixel value of the interpolation pixel by using the result of the adjustment, the digital image It is an object of the present invention to provide an interpolation pixel value calculation method, apparatus, and program that can reduce blur in an image obtained by enlarging or reducing.

The interpolation pixel value calculation method of the present invention includes:
An interpolation pixel value calculation method for calculating a pixel value of an interpolation pixel for enlarging or reducing a digital image obtained by imaging a subject based on a pixel value of a neighboring pixel located in the vicinity of the interpolation pixel,
Detect an edge located between the neighboring pixels by a predetermined edge detection process ,
When the edge is detected by the edge detection process, the pixel value of the interpolation pixel is calculated based on the pixel value of the neighboring pixel on the side where the interpolation pixel is located on both sides of the edge,
By the edge detection processing, when the edge at any between two adjacent pixels of the three pixels adjacent in series on said digital image is detected, the merges the two edges 3 one of the single merged edges located near the center pixel among the pixels, and calculates the pixel value of the interpolation pixel based on the pixel values of the neighboring pixels on the side where the interpolation pixel is located among the opposite sides of said combination if edge It is characterized by this.

The interpolation pixel value calculation method of the present invention compares the pixel values of the three pixels at the two edges, and the pixel values of the three pixels increase or decrease monotonously along the direction in which the three pixels are arranged. It is preferable to perform the merging on the two edges only in the case of doing so.

More preferably , the interpolation pixel value calculation method of the present invention performs the merging on the two edges only when the extending directions of the two edges are the same.

  Here, “digital image” means an image represented by digital data of an image obtained by capturing an image of a subject, and is not limited to one acquired by a digital camera, but a photographic film such as a negative film or a color reversal film. Also included are those obtained by photoelectrically reading a photographic image recorded on the printer, a printed photographic image, or the like with a reading device such as a scanner. In the following description, a digital photographic image is simply referred to as an image.

Further, the "center located near the pixel" means to position the position between the center point of the respective pixel pair consisting of mutually adjacent two pixels of the three pixels, positioned in the position of the center pixel it is intended to include that. For example, when the three pixels are G1, G2, and G3, the position near the central pixel G2 is a position between the central point between the pixels G1 and G2 and the central point between the pixels G2 and G3. .

Interpolation pixel value calculating device of the present invention,
An interpolation pixel value having an interpolation pixel value calculation means for calculating a pixel value of an interpolation pixel for enlarging / reducing a digital image obtained by imaging a subject based on a pixel value of a neighboring pixel located in the vicinity of the interpolation pixel A calculation device,
Edge detecting means for detecting an edge located between pixels of the neighboring pixels by a predetermined edge detection process ;
When the edge is detected in any of two adjacent pixels of the three pixels adjacent in series on the digital image by the edge detection means, the two edges are merged to and a edge merging means for a single merged edges located near the center pixel among the pixels,
The interpolation pixel value calculation means is
When the edge is detected by the edge detection means, the pixel value of the interpolation pixel is calculated based on the pixel value of the neighboring pixel on the side where the interpolation pixel is located on both sides of the edge;
When the two edges are merged by the edge merging means, the pixel value of the interpolation pixel is calculated based on the pixel value of the neighboring pixel on the side where the interpolation pixel is located on both sides of the merged edge It is characterized by being.

  Further, the edge merging means compares the pixel values of the three pixels of the two edges, and the pixel values of the three pixels monotonously increase or decrease monotonously along the direction in which the three pixels are arranged. It is preferable that the merging is performed only on the two edges.

  The edge detection means also detects the extending direction of the edge, and the edge merging means performs the merging with respect to the two edges only when the extending directions of the two edges are the same. More preferably.

In addition, you may provide the interpolation pixel value calculation method of this invention as a program for making a computer perform.

  According to the present invention, when an edge is detected between any two adjacent pixels of three pixels adjacent in series in the image, the two edges are the center of the three pixels. Merged at a position near the pixel.

  When an edge is detected between the pixel located at the center of three pixels adjacent in series on the image and any of the other two pixels (for convenience of explanation, in the following, When two edges are detected between two adjacent pixels of three adjacent pixels, these two edges are referred to as adjacent edges), and the pixel located in the center is determined to be an intermediate value pixel Can do. For example, as a method for detecting whether or not there is an edge between pixels in the image of the subject in the example shown in FIG. 33A, for example, the second edge detection method described above, that is, four adjacent in series A primary difference and a secondary difference are obtained for a pixel, and an edge is found between two adjacent pixels located at the center of the four pixels based on the positive / negative relationship between the obtained primary difference and the secondary difference. When the edge detection method is applied so as to determine whether or not there is, as shown in FIG. 33 (b), an edge A and an edge between pixels Ga and Gb and between pixels Gb and Gc, respectively. B is detected. Then, the pixel Gb between the edge A and the edge B, which are adjacent edges, can be regarded as an intermediate value pixel indicating an intermediate value of signal values on both sides of the boundary line L (edge in the subject) shown in FIG. it can.

  The present invention pays attention to the point that the pixels sandwiched between the adjacent edges can be regarded as intermediate value pixels, and merges adjacent edges. By doing so, first, when enlarging or reducing the image, a pixel that is not an intermediate value pixel among pixels located on the same side as the interpolation pixel with respect to the edge is set as a reference pixel, and an interpolation calculation is performed based on the pixel value of the reference pixel. When the pixel value of the interpolation pixel is obtained by performing the above, there is no reference pixel for obtaining the pixel value of the interpolation pixel located between adjacent edges, and the problem that the pixel value of the interpolation pixel cannot be obtained can be solved. .

  Further, for example, when the adjacent edge is merged with the position of the center pixel of the three pixels corresponding to the adjacent edge, that is, the pixel that can be determined as the intermediate value pixel, the position of the intermediate value pixel overlaps the edge position. Even if it is not determined whether or not the pixel is an intermediate value pixel, the intermediate value pixel is not used as a reference pixel, so that it is possible to prevent the blurring from remaining or spreading in an easily enlarged / reduced image.

  In addition, even when the position where adjacent edges are merged is not the same as the position of the central pixel, other reference pixels can be obtained in addition to the central pixel for the interpolated pixels positioned between adjacent edges. Therefore, it is possible to perform enlargement / reduction with good image quality by performing interpolation calculation based on the pixel value of the reference image so that the central pixel is not used as a reference pixel or a weight smaller than other reference pixels is given to the central pixel. Images can be obtained.

  On the other hand, for example, as shown in FIG. 34A, the pixel configuration is as shown in FIG. 34B in the X direction of the image of the subject where one thin line exists. In this case, the first edge detection method described above, that is, a difference between pixel values between two adjacent pixels is obtained, and when the absolute value of the difference is equal to or greater than a predetermined threshold value, the difference between the two pixels is determined. When the first edge detection method described above that detects an edge by determining that an edge is present is applied, between the pixel G′a and the pixel G′b and between the pixel G′b and the pixel G′c. In addition, edge A ′ and edge B ′ are detected, respectively. If a pixel sandwiched between adjacent edges is set as an intermediate value pixel, the pixel G′b located at the center is set as an intermediate value pixel. In this case, the candidate pixel G′b is a pixel indicating a thin line. It is not an intermediate value pixel. Even when the above-described fourth edge detection method is used, such an erroneous determination may occur. On the other hand, when the second or third edge detection method described above is applied as an edge detection method, no edge is detected between the pixels G′a and G′b and between the pixels G′b and G′c. Therefore, even if a pixel sandwiched between adjacent edges is determined as an intermediate element, the pixel G′b is not erroneously determined as an intermediate value pixel. That is, if a pixel sandwiched between adjacent edges is an intermediate value pixel, a pixel that is not an intermediate value pixel may be determined to be an intermediate value pixel depending on a method for detecting whether an edge exists between pixels on the image. There is. Therefore, in the example shown in FIG. 34, when the adjacent edges A ′ and B ′ are merged, the pixel G′b sandwiched between the two adjacent edges is not used as a reference pixel when the image is enlarged or reduced. Or, since there is a possibility of being used with a reduced weight, the line that actually exists in the subject (the line L ′ in FIG. 34A) disappears or becomes thin in the scaled image. Problems arise.

  On the other hand, in the present invention, if the adjacent edges are not merged, the corresponding three pixel values of the adjacent edges are compared, and the pixel values of the three pixels increase monotonously along the direction in which the pixels are arranged or Only when monotonously decreasing, that is, only when the pixel sandwiched between adjacent edges is a true intermediate value pixel, the above problem can be prevented by merging the adjacent edges. Blur can be reduced and the lines on the subject can be prevented from disappearing.

  Further, the present invention focuses on the fact that the certainty factor that is an intermediate value pixel of pixels sandwiched by adjacent edges extending in different directions is lower than the certainty factor that is an intermediate value pixel of pixels sandwiched by adjacent edges extending in the same direction. If only the adjacent edges extending in the same direction are merged, it is possible to more reliably prevent problems such as removal of pixels that are not intermediate value pixels from the reference pixels of the interpolation pixels.

The program of the present invention can cause a computer to execute the interpolation pixel value calculation method of the present invention.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an image enlargement / reduction apparatus according to the first embodiment of the present invention. As illustrated, the image enlargement / reduction apparatus according to the present embodiment includes an input unit 1 that receives input of image data S0 and an enlargement / reduction rate K of the image data S0, an edge detection unit 10, an edge merging unit 20, and an interpolation pixel. An interpolation calculation unit 30 for calculating the pixel value of the input unit 1, an edge detection unit 10, an edge merging unit 20, and a control unit 50 for controlling the operation of the interpolation calculation unit 30.

  Here, the image represented by the image data S0 is configured by two-dimensionally arranging pixels as shown in FIG. 2, and in the following description, as shown in FIG. The y direction shall be determined. In the following description, image data and an image represented by the image data are not particularly distinguished, and the same reference numerals (here, S0) are given.

  FIG. 3 is a block diagram showing the configuration of the edge detection unit 10 in the image enlargement / reduction apparatus of this embodiment. As shown in the figure, the edge detection unit 10 includes a filtering unit 12 and a determination unit 14 that determines whether an edge exists between two adjacent pixels.

  The filtering unit 12 performs a filtering process as follows. As shown in FIG. 4, the filtering unit 12 includes two adjacent pixels G2 and G3 and two adjacent pixels G2 and G3 for each column in the x direction and the y direction of the image S0. With respect to two pixels G1 to G4 adjacent in series consisting of pixels, a difference filter is used for three pixel pairs (G1, G2), (G2, G3), (G3, G4) consisting of two adjacent pixels. A filtering process is performed to calculate the differences between the pixel values of the pixel pairs (G1, G2), (G2, G3), (G3, G4) as primary differences d1, d2, d3.

  FIG. 5 is a diagram illustrating an example of the difference filter. As shown in FIG. 5, the differential filter used in the present embodiment is a 2-tap filter having a filter value of (−1, 1). Note that the difference filter is not limited to this, and a filter having a filter value for obtaining a weighted difference between two pixel values of a pixel pair, or a filter having an even number of taps of 2 taps or more may be used.

  Subsequently, the filtering unit 12 similarly applies the difference filter shown in FIG. 5 to two adjacent primary difference pairs (d1, d2) and (d3, d4) in the three primary differences d1, d2, and d3. A filtering process is performed to calculate the differences between the primary difference pairs (d1, d2) and (d3, d4) as secondary differences d4, d5.

  Further, the filtering unit 12 performs a filtering process on the pixel pair (G2, G3) including the center two pixels of the four pixels G1 to G4 adjacent in series, and the pixels of the pixel pair (G2, G3) A value difference d0 (= d2) is calculated. The primary difference d2 may be used as the difference d0 as it is.

  As a first determination, the determination unit 14 determines whether an edge exists between two adjacent pixels G2 and G3 based on the positive / negative relationship between the primary differences d1, d2, and d3 and the secondary differences d4 and d5. Determine whether or not.

  6 to 8 are tables showing the relationship between the positive / negative of the primary differences d1, d2, d3 and the secondary differences d4, d5 of four pixels adjacent in series and the profile shape of the four pixels adjacent in series. It is. There are a total of 18 kinds of combinations of positive and negative of the primary differences d1, d2, d3 and the secondary differences d4, d5 of four pixels adjacent in series. There are two combinations of edge 1 and edge 2 shown in FIG. 6 in which an edge exists between two adjacent pixels G2 and G3. Edge 1 is a right rising edge such that (d1, d2, d3, d4, d5) = (+, +, +, +, −) and (d1, d2, d3, d4, d5) = (−, −, There are two types of rising edges that are-,-, +), and edge 2 has a downwardly convex shape where (d1, d2, d3, d4, d5) = (+, +, +, +, +) Upwardly rising edge where (d1, d2, d3, d4, d5) = (+, +, +, −, −), (d1, d2, d3, d4, d5) = ( Downward convex left rising edge that becomes −, −, −, +, +), and upward convex leftward that becomes (d1, d2, d3, d4, d5) = (−, −, −, −, −) There are four types of rising edges.

  The determination unit 14 stores the tables shown in FIGS. 6 to 8, and the positive / negative relationship between the primary differences d1, d2, d3 and the secondary differences d4, d5 of four pixels adjacent in series is shown in FIG. 7 and 8, it is determined that an edge exists between two adjacent pixels G <b> 2 and G <b> 3, and the positive / negative relationship between these differences is as shown in FIGS. 7 and 8. When there is a relationship such as a mountain, a valley, or the like, it is determined that no edge exists between two adjacent pixels G2 and G3.

  Furthermore, when the determination unit 14 determines that an edge is present in the first determination as the second determination, the absolute value of the difference d0 between the pixel values of the pixels G2 and G3 is equal to or greater than a predetermined threshold Th1. If the second determination is affirmed, it is determined that there is a true edge. On the other hand, if the second determination is negative, it is determined that there is no true edge. In this way, the second determination is performed even if it is determined that there is an edge in the first determination, for example, as shown in FIG. 9, the difference between the pixel values of the pixels G2 and G3 is very slight and noise. In such a case, the interpolation calculation unit 30 performs an interpolation calculation suitable for the edge portion as will be described later, and prevents noise from being emphasized. When the determination is made between two adjacent pixels in this way, the edge detection unit 10 outputs the determination result to the edge merging unit 20.

  When the edge merging unit 20 detects an edge in each of the edges detected by the edge detection unit 10, an adjacent edge, that is, an edge between two adjacent pixels of three pixels adjacent in series. And detecting a pair of edges, and merging the detected adjacent edges at the pixel positions between the adjacent edges. For example, in the pixel row shown in FIG. 4, when edges are detected between the pixels G1 and G2 and between the pixels G2 and G3, the two edges are merged at the position of the pixel G2, and the pixel G1 Between the pixel G2 and the pixel G3, an edge exists only at the position of the pixel G2, and no edge exists between the pixels G2 and G3 and between the pixels G2 and G3.

  FIG. 10 is a block diagram illustrating a configuration of the interpolation calculation unit 30. As illustrated, the interpolation calculation unit 30 includes a boundary setting unit 32, a determination unit 34, and a calculation unit 36, and calculates the pixel value of the interpolation pixel P located between the pixels in the image S0. Details of the interpolation calculation unit 30 will be described below.

  When there is no edge between two adjacent pixels between which the interpolation pixel P is located and the position of the two adjacent pixels, the interpolation calculation unit 30 uses the bicubic method to calculate the pixel value of the interpolation pixel P. Is calculated.

  Here, the bicubic method is one method of cubic interpolation, and is a method of obtaining the pixel value of the interpolation pixel P using 16 pixels in the vicinity of the interpolation pixel P. Hereinafter, the bicubic method will be described.

FIG. 11 is a diagram for explaining the bicubic method. As shown in the figure, when the point P is set as the position of the interpolation pixel P, the pixel indicated by the black circle in FIG. 11 is referred to as the primary neighborhood, and the pixel indicated by the white circle is referred to as the secondary neighborhood. For each of the primary neighborhood and the secondary neighborhood, as shown in the following formula (1), the distances dx and dy (indicated by simply d in the formula (1)) are independently obtained in the x direction and the y direction, respectively. On the other hand, weights Wx and Wy are obtained, and finally a weight W = WxWy for the pixel is obtained.

For example, when the weights Wx, Wy, and W are obtained for the pixel (second order neighborhood) of (−1, −1) in FIG.

It becomes.

When W (i, j) is the weight of the pixel (i, j) and f (i, j) is the pixel value of the pixel (i, j), the pixel value f ′ (P) of the interpolation pixel P is

Can be calculated.

  In the present embodiment, the pixel value of the interpolation pixel P is calculated by applying the bicubic method only in the one-dimensional direction of the x direction or the y direction.

  On the other hand, for the portion determined to be an edge, the interpolation calculation unit 30 obtains the pixel value of the interpolation pixel P as follows. First, a case where this edge is located between pixels will be described. FIG. 12 is a diagram illustrating a profile of pixel values for a portion determined to have an edge between pixels. In FIG. 12, the horizontal direction is a direction in which pixels are arranged, and the vertical direction is a direction indicating the magnitude of the pixel value. When it is determined that an edge exists between two adjacent pixels G2 and G3, the pixels of four pixels G1 to G4 adjacent in series obtained by adding two adjacent pixels G1 and G4 to the two pixels, respectively. The value profile is as shown in FIG. 12 (a) or FIG. 12 (b).

  When the profile has the step edge shape shown in FIG. 12A, the boundary setting unit 32 uses the middle line M (dashed line) that bisects the distance between the pixels G2 and G3 in the direction in which the pixels are arranged as a boundary line. Set. Then, the determination unit 34 determines whether the interpolation pixel P exists on the right side or the left side of the boundary line M, and when the interpolation pixel (referred to as P1) exists on the right side of the boundary line M, the calculation unit 36 calculates the value on the extension of the straight line connecting the pixels G3 and G4 as the pixel value of the interpolation pixel P1. When the interpolation pixel (P2) is present on the left side of the boundary line M, the calculation unit 36 calculates the value on the extension line of the straight line connecting the pixels G1 and G2 as the pixel value of the interpolation pixel P2.

  When the profile has the edge shape shown in FIG. 12B, the boundary setting unit 32 uses the intersection C between the extension line of the straight line connecting the pixels G1 and G2 and the extension line of the straight line connecting the pixels G3 and G4 as a boundary point. Set. Then, the determination unit 34 determines whether the interpolation pixel P exists on the right side or the left side of the boundary point C. When the interpolation pixel P1 exists on the right side of the boundary point C, the calculation unit 36 calculates the pixel G3. , G4 is calculated as the pixel value of the interpolation pixel P1. When the interpolation pixel P2 exists on the left side of the boundary point C, the calculation unit 36 calculates a value on an extension line of a straight line connecting the pixels G1 and G2 as the pixel value of the interpolation pixel P2.

  Here, the pixel value of the interpolation pixel P is calculated using only the pixel values of two pixels, but three or more pixel values may be used. In this case, it may be difficult to connect the pixels with a straight line. Therefore, the pixels may be connected by a curve defined by an arbitrary function such as a spline curve, and the value on the extension line of the curve may be set as the pixel value of the interpolation pixel P.

  Next, when the edge exists at the pixel position, that is, when the edge is obtained by merging the adjacent edge by the edge merging unit 20 at the pixel position sandwiched between the two adjacent edges, the interpolation calculation unit The calculation of the pixel value of the interpolation pixel by 30 will be described.

  First, in this case, the pixel values of the three pixels that are pixels that sandwich the adjacent edge and the two pixels that are adjacent to each of the two pixels that are located outside the adjacent edge among the three pixels. The profile will be described. In FIG. 13A, a pair of adjacent edges belong to the type (d1, d2, d3, d4, d5) = (+, +, +, +, −) at edge 1 shown in FIG. The figure which shows the example of the profile mentioned above when it is an edge and the edge which belongs to the kind of (d1, d2, d3, d4, d5) = (+, +, +,-,-) in the edge 2 shown in FIG. It is. In the case of the example illustrated in FIG. 13A, the edge detection unit 10 detects edges between the adjacent pixels G2 and G3 and between the adjacent pixels G3 and G4. The two edges are merged at the position of the pixel G3.

  First, as shown in FIG. 13B, the interpolation calculation unit 30 sets the edge position, that is, a vertical line (dotted line in the drawing) M passing through the pixel G3 as the boundary line by the boundary setting unit 32. Then, the determination unit 34 determines whether the interpolation pixel exists on the right side or the left side of the boundary line M, and when the interpolation pixel (P1) exists on the right side of the boundary line M, the calculation unit 36 determines whether the pixel is a pixel. The value on the extended line of the straight line connecting G4 and G5 is calculated as the pixel value of the interpolation pixel. When the interpolation pixel (P2) exists on the left side of the boundary line M, the calculation unit 36 calculates the value on the extension line of the straight line connecting the pixels G1 and G2 as the pixel value of the interpolation pixel.

  Here, the pixel value of the interpolation pixel is calculated using only the pixel values of the two pixels, but three or more pixel values may be used. In this case, it may be difficult to connect the pixels with a straight line. Therefore, the pixels may be connected by a curve defined by an arbitrary function such as a spline curve, and the value on the extension line of the curve may be the pixel value of the interpolation pixel.

  Further, in this case, when the interpolation pixel (P1) exists on the right side of the boundary line M, the pixel value of the pixel G4 is set as the pixel value of the interpolation pixel, and the interpolation pixel (P2) exists on the left side of the boundary line M. In this case, the pixel value of the pixel G2 may be the pixel value of the interpolation pixel.

  In the following explanation, the calculation (calculation by bicubic method) for obtaining the pixel value of an interpolation pixel located at a part where no edge exists is the first interpolation calculation, and the pixel of the interpolation pixel located at a part where the edge exists between the pixels The calculation for obtaining a value is referred to as a second interpolation calculation, and the calculation for obtaining a pixel value of an interpolation pixel located at a portion where an edge exists at the pixel position is referred to as a third interpolation calculation.

  Next, processing performed in the present embodiment will be described. FIG. 14 is a flowchart showing processing performed in the present embodiment. In the present embodiment, it is assumed that the interpolation pixel P exists between the pixels of the image S0. First, the input unit 1 receives input of image data S0 to be enlarged / reduced and an enlargement / reduction ratio K of the image data (S1). The direction for interpolating the pixels is first set in the x direction (S2). Next, for the first interpolation pixel P corresponding to the enlargement / reduction ratio K (for example, the pixel located in the upper left on the enlarged image S1), the filtering unit 12 of the edge detection unit 10 has the interpolation pixel P positioned between them. A pixel column Φ composed of four pixels G1 to G4 adjacent in series is set as shown in FIG. 18 (S3). It is detected whether there is an edge between the pixels G2 and G3 using the pixel value of each pixel of the set pixel row Φ (S10). Here, the details of the edge detection process A performed in step S10 will be described later. If it is determined in step S10 that there is no edge between the pixels G2 and G3 (S20: No), the interpolation calculation unit 30 performs the first interpolation calculation B1, that is, the interpolation calculation by the bicubic method described above. The pixel value of the interpolation pixel P is calculated (S30).

  On the other hand, if it is determined in step S10 that there is an edge between the pixels G2 and G3 (S20: Yes), the filtering unit 12 of the edge detection unit 10 obtains the pixel row Φ by shifting the pixel row Φ by one pixel each on the left and right. The pixel column Φ1 (G0 to G3) and the pixel column Φ2 (G2 to G5) shown in FIG. 18 are set (S40). Edge detection processing A by the edge detection unit 10 is performed on the set two pixel rows Φ1 and Φ2, and it is detected whether or not there is an edge between the pixels G1 and G2 and between the pixels G3 and G4. (S42). A detection result indicating whether or not there is an edge between the pixels G2 and G3 and an edge exists between the pixels G1 and G2 and between the pixels G3 and G4 is output to the edge merging unit 20. When no edge is detected in any of the pixels G1 and G2 and between the pixels G3 and G4 (S44: No), the edge merging unit 20 directly uses the detection result of the edge detection unit 10 as an interpolation calculation unit 30. (S46). Further, when the edge merging unit 20 detects an edge only between one of the pixels G1 and G2 and between the pixels G3 and G4 (S44: Yes, S60: No), the edge merging unit 20 is between G2 and G3. The edge and the edge detected between G1 and G2 or between the pixels G3 and G4 are merged at the position of the pixel sandwiched between the two edges (G2 or G3 in the example of FIG. 18), Information representing the pixel where the merged edge is located is output to the interpolation calculation unit 30 (S67). The edge merging unit 20 determines the edge detection result of the edge unit 10 when an edge is detected between the pixels G1 and G2 and between the pixels G3 and G4 (S44: Yes, S60: Yes). The data is output to the interpolation calculation unit 30 as it is (S65).

  The interpolation calculation unit 30 calculates the pixel value of the pixel P by performing an interpolation calculation according to the information output from the edge merging unit 20 in step S46, step S67, and step S65. Specifically, when an edge exists between the pixels G2 and G3 (S20: Yes, S44: No), the interpolation calculation unit 30 performs the second interpolation calculation B2 (details will be described later) to perform interpolation. The pixel value of the pixel P is calculated (S50). When the edge is located in either one of the pixels G2 and G3 (S22: Yes, S44: Yes, S60: No, S67), the interpolation calculation unit 30 performs the third interpolation calculation B3 (details will be described later). ) To calculate the pixel value of the interpolation pixel P (S70). In addition, when an edge exists in any of two adjacent pixels G1, G2, G3, and G4 (S22: Yes, S44: Yes, S60: Yes, S65), the interpolation calculation unit 30 performs the first operation. The pixel value of the interpolated pixel P is calculated by performing the interpolation calculation B1 of the above, that is, the interpolation calculation by the bicubic method (S30). Here, if there is an edge between any two adjacent pixels G1, G2, G3, G4, that is, if three adjacent edges continue, each edge is not due to an edge in the subject. This is because there is a high possibility of gradation, and it can be determined that no edge exists between the pixels G2 and G3.

  FIG. 15 is a flowchart showing edge detection processing A by the edge detection unit 10. As shown in the figure, the filtering unit 12 of the edge detection unit 10 applies the four pixels included in the pixel column to the set pixel column (the pixel column Φ or the pixel column Φ1 or the pixel column Φ2 shown in FIG. 18). On the other hand, primary differences d1, d2, d3 and secondary differences d4, d5 are calculated (S11). In addition, the filtering unit 12 performs a filtering process on the two pixels located in the center of the pixel row to calculate a difference d0 (= d2) (S12). Then, the determination unit 14 determines whether or not an edge exists between two pixels located at the center based on the positive / negative combination of the primary differences d1, d2, d3 and the secondary differences d4, d5 ( First determination, S13). If the first determination in step S13 is affirmed (S13: Yes), the determination unit 14 determines whether or not the absolute value of the difference d0 is equal to or greater than a predetermined threshold Th1 (S14, second determination). . If the second determination in step S14 is affirmed (S14: Yes), the determination unit 14 determines that an edge exists between the two central pixels (S15). On the other hand, if the first determination in step S13 is negative (S13: No) and the second determination in step S14 is negative (S14: No), the determination unit 14 determines whether the center two pixels Is determined to have no edge (S16).

  FIG. 16 is a flowchart showing the processing of the second interpolation calculation B2. As illustrated, the boundary setting unit 32 of the interpolation calculation unit 30 sets a boundary line or boundary point as a boundary between two adjacent pixels with the interpolation pixel P existing therebetween (S52). The determination unit 34 determines whether the interpolation pixel P is on either side of the boundary (S54), and the calculation unit 36 uses only pixels on the same side as the side where the interpolation pixel P exists with respect to the boundary. Then, the interpolation calculation is performed to obtain the pixel value of the interpolation pixel P (S56).

  FIG. 17 is a flowchart showing the process of the third interpolation calculation B3. As illustrated, the boundary setting unit 32 of the interpolation calculation unit 30 sets a vertical line passing through the pixel where the edge is located as a boundary (S72). The determination unit 34 determines whether the interpolation pixel P is on either side of the boundary (S74), and the calculation unit 36 uses only pixels on the same side as the side where the interpolation pixel P exists with respect to the boundary. Then, the interpolation calculation is performed to obtain the pixel value of the interpolation pixel P (S76).

  Returning to FIG. 14, the control unit 50 determines whether or not pixel values have been calculated for all the interpolation pixels P in the set interpolation direction (S <b> 84), and when step S <b> 84 is negative, calculates a pixel value. The interpolation pixel P is set to the next interpolation pixel P (S86), and the process returns to step S3.

  On the other hand, when step S84 is affirmed, the control unit 50 determines whether the interpolation direction in which the calculation of the pixel values of all the interpolation pixels is completed is the x direction or the y direction (S90). When the interpolation direction for which the calculation of the pixel values of all the interpolation pixels has been completed is the x direction (S90: No), the interpolation direction is set to the y direction (S92), and the process returns to step S3. On the other hand, when the interpolation direction in which the calculation of the pixel values of all the interpolation pixels has been completed is the Y direction (S90: Yes), the enlarged / reduced image data S1 composed of the interpolation pixels P is output (S94), and the processing ends. To do.

  FIG. 19 is a block diagram showing a configuration of an image enlargement / reduction apparatus according to the second embodiment of the present invention. As illustrated, the image enlargement / reduction apparatus according to the present embodiment includes an input unit 101 that receives input of image data S0 and an enlargement / reduction rate K of the image data S0, an edge detection unit 110 that detects an edge between pixels in the image, The edge merging unit 120, the interpolation calculation unit 130 that calculates the pixel value of the interpolation pixel, and the control unit 150 that controls operations of the input unit 101, the edge detection unit 110, the edge merging unit 120, and the interpolation calculation unit 130 are provided. Details of each component will be described below.

  As illustrated in FIG. 20, the edge detection unit 110 includes a filtering unit 112, a determination unit 114, and an edge pattern classification unit 116. The filtering unit 112 first has 4 × 4 16 pixels (16 pixels P (i, j) (i, j = −1 to 2 shown in FIG. 21)) located in the vicinity of the interpolation pixel P on the image S0. The pixel pair consisting of two adjacent pixels is subjected to a filtering process using the difference filter shown in FIG. 5 to each pixel pair (hereinafter referred to as the difference between adjacent pixel pairs). Is referred to as d). Here, the two pixels adjacent to each other are two pixels of P (-1, 0) and P (0, 0), or P (-1, -1) and P (-1, 0). Not only two pixels adjacent to each other in the x direction or y direction of the pixel arrangement direction, such as two pixels, but two pixels of P (-1, -1) and P (0, 0), and P (0 , 1) and P (1, 0) also include two pixels adjacent in the diagonal direction of 2 × 2 pixels.

  The determination unit 114 determines whether or not the absolute value of the difference d of each pixel pair is greater than or equal to a predetermined threshold Th, and when this determination is affirmative, the determination unit 114 determines whether the two pixels of the pixel pair corresponding to the difference correspond to each other. It is determined that there is an edge between them, and the determination result is output to the edge pattern classification unit 116.

  Based on the determination result output from the determination unit 114, the edge pattern classification unit 116 classifies the edge patterns in the block composed of four 2 × 2 pixels. Here, a block composed of four pixels P (0, 0), P (1, 0), P (1, 1), and P (0, 1) will be described first. As shown in FIG. 22, the edge pattern classification unit 116 classifies the edge pattern according to a straight line connecting the midpoints between pixels in which edges exist. 23 to 25 are diagrams showing edge patterns according to positions where edges exist. As shown in FIGS. 23 to 25, the edge patterns are classified into nine types of patterns 0 to 8 according to the position where the edge exists.

  When edges exist between the spaces e1, e2, e3, e4, and when edges exist between the spaces e1, e2, e3, e4, e5, e6, the edge pattern is the pattern 7 or the pattern 8 I do n’t know. For this reason, the edge pattern classification unit 116 further increases the pixel P when there are edges in the intervals e1, e2, e3, e4 and when there are edges in the intervals e1, e2, e3, e4, e5, e6. The absolute value of the difference between the pixel values of (0, 0) and the pixel P (1, 1) (referred to as | d11 |) and the absolute value of the difference between the pixel values of the pixel (0, 1) and the pixel (1, 0) (If | d11 | <| d12 |), the edge pattern is classified into the pattern 7, and if | d11 | ≧ | d12 |, the edge pattern is classified into the pattern 8. .

  The edge pattern classification unit 116 performs processing for classifying such edge patterns on each 2 × 2 pixel block, and outputs the result of the classification processing to the edge merging unit 120.

  FIGS. 26 and 27 show examples of edge patterns in 4 × 4 16 pixels obtained by the edge pattern classification unit 116. In the example of FIG. 26, the edge pattern in the four pixels adjacent to the interpolation pixel P is pattern 4, four pixels P (-1, -1), P (0, -1), P (0, 0), Edge pattern in P (-1, 0) is pattern 0, edge in four pixels P (0, -1), P (1, -1), P (1,0), P (0, 0) The pattern is pattern 5, the four pixels P (1, -1), P (2, -1), P (2,0), and the edge pattern in P (1,0) are pattern 0, four pixels P ( -1,0), P (0,0), P (0,1), and P (-1,1) have an edge pattern of pattern 2, four pixels P (1,0), P (2,0 ), P (2,1), P (1,1) has an edge pattern of pattern 0, four pixels P (-1,1), P (0,1), P (0,2), P ( -1, 2) The edge pattern in Turn 4, an edge pattern in four pixels P (0,1), P (1,1), P (1,2), P (0,2) is pattern 0, and four pixels P (1,1 ), P (2,1), P (2,2), and the edge pattern in P (1,2) is pattern 0. In addition, as shown in FIG. 26, the area | region in 16 pixels of the interpolation pixel P is divided | segmented into two area | region A1, A2 by the edge (dotted line in a figure). Here, the region A2 is indicated by oblique lines.

  In the example of FIG. 27, two edges (edges I1 and I2 indicated by a dotted line in the figure) in which edge patterns are continuous in 16 pixels of the interpolation pixel P are shown.

  Based on the result output from the edge pattern classifying unit 116, the edge merging unit 120 first detects this when there is an edge between any two adjacent pixels of three pixels adjacent in series. The pixel values of the three pixels are compared, and when the pixel value monotonously increases or decreases monotonously in the direction in which the three pixels are arranged and the two edges are edges extending in the same direction, the two edges are Merge in position. FIG. 28 is a diagram for explaining a process of merging adjacent edges by the edge merging unit 120. In FIG. 28, it is assumed that the pixel values of three pixels corresponding to each adjacent edge satisfy the relationship of monotonically increasing or monotonously decreasing along the direction in which the pixels are arranged. As shown in FIG. 28 (a), edges are detected between the pixels G1a and G2a, between the pixels G2a and G3a, and the directions in which the respective edges extend are the same. Merged. Further, since the edge between the pixels G4a and G5a and the edge between the pixels G4a and G5a also extend in the same direction, as shown in the right part of FIG. 28A, two edges I1a and I2b ( A dotted line in the figure is merged with one edge Ja that overlaps a line (thick line in the figure) connecting pixels G2a and G5a. Similarly, as shown in FIG. 28B, the edges I1b and I2b are merged into one edge Jb. Further, as shown in FIG. 28C, in two adjacent pixel blocks with the diagonal pixel serving as a contact, the diagonal pixel G2c serving as the contact and each pixel (G1c, G3c, G4c, G5c) also have edges, and each edge extends in the same direction, so the edge between G5c and G2c and the edge between pixels G2c and G3c are merged at the position of pixel G2c Since the edge between the pixels G1c and G2c and the edge between the pixels G2c and G4c are merged at the position of the pixel G2c, the two edges I1c and I2c on the left in FIG. As shown in the figure, it becomes one edge point Jc (position of the pixel G2c). As shown in FIG. 28 (d), the edge between the pixels G1d and G2d and the edge between the pixels G2d and G3d are at the position of the pixel G2c, and the edge between the pixels G3d and G4d and between the pixels G4d and G5d By merging the edges with the position of the pixel G4d, the two edges I1d and I2d extending in the same direction in the pixel G2d, G3d, G4d, and G6d pixel blocks are shown in the right part of FIG. As shown in FIG. 4, one edge Jd is formed by connecting pixels G2d and G4d located at diagonal points of the block.

  The edge merging unit 120 performs the merging process shown in FIG. 28 on the corresponding adjacent edge, and outputs the result to the interpolation calculation unit 130. In the example of the edge pattern shown in FIG. 27, two edges I1 and I2 are merged into one edge I indicated by a thick line in the figure by the edge merging unit 120. 27, the area within 16 pixels of the interpolation pixel P is divided into two areas by the edge I (thick line in the figure) as shown in FIG. 27. Here, the areas on both sides of the edge I are also areas A1. , A2 and the area A2 is indicated by hatching.

  In the case of the edge pattern example shown in FIG. 26, there is no edge to be merged, and the edge merging unit 120 outputs the output result from the edge pattern classification unit 116 to the interpolation calculation unit 130 as it is. In the case of the example of FIG. 27, the result obtained by merging the edges in 16 pixels of the interpolation pixel P is output to the interpolation calculation unit 130.

  The interpolation calculation unit 130 calculates the pixel value of the interpolation pixel P based on the result indicating the edges in 16 pixels of the interpolation pixel P output from the edge merging unit 120. Specifically, when there is no edge in 16 pixels surrounding the interpolation pixel P, the pixel value of the interpolation pixel P is calculated using the bicubic method. Interpolation calculation by the bicubic method is the same as the first interpolation calculation performed by the interpolation calculation unit 30 of the embodiment shown in FIG. 1, and thus detailed description thereof is omitted here. In the following description of the present embodiment, when there is no edge in 16 pixels of the interpolation pixel P, the calculation for calculating the pixel value of the interpolation pixel P using the bicubic method is performed in the first embodiment. This is called interpolation calculation.

  On the other hand, when an edge exists in 16 pixels of the interpolation pixel P, the interpolation calculation unit 130 calculates the pixel value of the interpolation pixel P by the second interpolation calculation. Here, the second interpolation calculation of the interpolation calculation unit will be specifically described with reference to the example of FIG. For ease of explanation, the pixels P (-1, -1), P (-1, 0), P (-1, 1), P (0, 0), P (0, -1) in FIG. ) Will be described with reference to P14, P11, P13, P12, and P15, respectively. As shown in FIG. 26, the area within 16 pixels surrounding the interpolation pixel P is divided into two areas A1 and A2 by edges. Region A2 is indicated by hatching. The interpolation calculation unit 130 first determines a pixel to be used for the interpolation calculation depending on which side the interpolation pixel P is located across the edge based on the edge pattern in the 16 pixels. For example, as shown in FIG. 26, when the interpolation pixel P exists on the region A1 side, the pixels P11, P12, P13, P14, and P15 on the region A1 side are determined as pixels used for the interpolation calculation. Conversely, when an interpolation pixel exists on the region A2 side, the pixel on the region A2 side is determined as a pixel used for the interpolation calculation.

Next, the interpolation calculation unit 130 performs an interpolation calculation to obtain a pixel value of the interpolation pixel P by increasing the weight of the determined pixel closer to the interpolation pixel P. Specifically, the weights W11 to W15 for the pixels P11 to P15 are set so as to maximize the weight W12 of the pixel P12, and the following formula (6) is set for the pixel values Pt11 to Pt15 of the pixels P11 to P15. The pixel value of the interpolation pixel P (here, Pt) is calculated by performing the calculation shown in FIG.

  Note that when calculating the pixel value of the interpolation pixel P, it is not necessary to use all the determined pixels, and the calculation method is not limited to this. For example, the pixel value of the interpolation pixel P is obtained by applying the second interpolation calculation (see FIG. 12A) by the interpolation calculation unit 30 in the embodiment shown in FIG. Also good. Specifically, as shown in FIG. 29, the pixels P11, P12, and P13 (see FIG. 28) on the region A1 side in the three-dimensional space in which the pixel position is the x coordinate and the y coordinate and the pixel value is the z coordinate. A plane A10 passing through the values Pt11, Pt12, Pt13 is set. In the plane A10, the sides A12 and A13 correspond to the positions of the edges. Then, the value of the z coordinate corresponding to the x coordinate and the y coordinate of the interpolation pixel P on the plane A10 may be obtained as the pixel value of the interpolation pixel P.

  Similarly, in the case of the example shown in FIG. 27, the pixels P (−1, −1), P (0, −1), P (0, −1), P (1, -1), P (2, -1), P (-1, 0), P (-1, 1), P (-1, 2), P (0, 0) are interpolated pixels P The pixel value is determined as the pixel used for the interpolation calculation for obtaining the pixel value, and the interpolation calculation based on these pixel values is performed to obtain the pixel value of the interpolation pixel P.

  As can be seen from the description here, when the merging process by the edge merging unit 120 is not performed, when the pixel value of the interpolated pixel P sandwiched between the two edges I1 and I2 is obtained, Similarly, there are only pixels P (2,0), P (1,0), P (0,1), and P (0,2) sandwiched between the edges I1 and I2, and these pixels are intermediate value pixels. Therefore, if it is removed from the pixels used for the interpolation calculation, the pixel value of the interpolation pixel P cannot be obtained. The merging process by the edge merging unit 120 can avoid using the intermediate value pixels P (2,0), P (1,0), P (0,1), P (0,2), and The pixel value of the interpolation pixel P can be obtained.

  The edge merging unit 120 compares the pixel values of the three pixels when there is an edge between any two adjacent pixels of the three pixels adjacent in series, and the three pixels are arranged. Merging is performed only when the pixel value monotonously increases or decreases monotonously in the direction and the two edges are edges extending in the same direction. For example, in the example shown in FIG. 27, when the pixel values of each of the three pixels are not in a monotonically increasing or monotonic decreasing relationship along the pixel arrangement direction, the edge merging unit 120 performs the edge I1 without performing the merging process. , I2 is output to the interpolation calculation unit 130. The interpolation calculation unit 130 converts the pixel value of the interpolation pixel P located between the two edges into pixels P (2,0), P (1,0), P, which are also sandwiched between the edges I1 and I2. It can be obtained by performing an interpolation operation with pixel values of (0, 1) and P (0, 2). In this case, there is a possibility that the pixels P (2, 0), P (1, 0), P (0, 1), and P (0, 2) are not intermediate value pixels but pixels indicating a thin line in the subject. Therefore, by obtaining the pixel value of the interpolation pixel P using the pixel values of these pixels, it is possible to prevent the problem that the thin line that matches the subject disappears in the enlarged / reduced image.

  In the following description, when an edge exists in 16 pixels of the interpolation pixel P, the interpolation calculation performed by the interpolation calculation unit 130 is set as the second interpolation calculation in the present embodiment.

  FIG. 30 is a flowchart showing processing performed in the image enlargement / reduction apparatus of this embodiment. In the present embodiment, it is assumed that the interpolation pixel P exists between the pixels of the image S0. First, the input unit 101 receives input of the image data S0 and the enlargement / reduction ratio K of the image data S0 (S101). Then, the filtering unit 112 is a 4 × 4 pixel located in the vicinity of the first interpolation pixel P (for example, the pixel located at the upper left on the image represented by the enlarged / reduced image data S1) according to the enlargement / reduction ratio K. With respect to 16 pixels (16 pixels of the interpolation pixel P), a filtering process using a difference filter is performed on a pixel pair including two adjacent pixels to calculate a difference between the respective pixel pairs (S110).

  Next, the determination unit 114 determines whether or not an edge exists between two pixels of each pixel pair based on whether or not the absolute value of the difference between each pixel pair is equal to or greater than a predetermined threshold Th. (S115). Specifically, if the absolute value of the pixel pair difference is equal to or greater than the threshold Th, it is determined that an edge exists between the two pixels of the pixel pair, while the absolute value of the pixel pair difference is smaller than the threshold Th. For example, it is determined that no edge exists between the two pixels of the pixel pair.

  The edge pattern classification unit 116 classifies the edge pattern in each 2 × 2 pixel based on the determination result from the determination unit 114, and outputs the result to the edge merging unit 120 (S135). The edge merging unit 120 performs a merging process (edge merging process C, the details of which will be described later) on the edges within 16 pixels of the interpolation pixel P, and outputs the result to the interpolation calculation unit 130 (S140). ). Based on the result output from the edge merging unit 120, the interpolation calculation unit 130 uses the first interpolation calculation D1, that is, the bicubic interpolation method when there is no edge in these 16 pixels (S165: No). Interpolation calculation is performed to determine the pixel value of the interpolation pixel P (S168). On the other hand, when there are edges in 16 pixels of the interpolation pixel P (S165: Yes), the second interpolation calculation D2 is performed to obtain the pixel value of the interpolation pixel P (S170).

  The control unit 150 determines whether or not pixel values have been calculated for all the interpolation pixels P with respect to the image S0 (S175), and when step S175 is negative, the interpolation pixel P for calculating the pixel value is determined as the next interpolation pixel P. The interpolation pixel P is set (S180), and the process returns to step S110. On the other hand, when step S180 is affirmed, the control unit 150 outputs the enlarged / reduced image data S1 including the interpolation pixel P (S190), and ends the process.

  FIG. 31 is a flowchart showing details of the edge merging process C performed by the edge merging unit 120 performed in step S140. As shown in the figure, the edge merging unit 120 includes three pixels adjacent in series within the 4 × 4 16 pixels surrounding the interpolation pixel P, and an edge exists between any two adjacent pixels. If there is such a pixel set (S141: Yes), two edges (adjacent edges) in the pixel set extend in the same direction, and there are three in the pixel set. When the pixel value of the pixel monotonously increases or decreases monotonously along the pixel arrangement direction (S142: Yes, S143, S144: Yes), the two edges are merged at the position of the center pixel of the corresponding pixel set ( On the other hand, in the other cases (S142: No or S142: Yes, S143, S144: No), these two edges are not merged (S146).

  The edge merging unit 120 performs the processing from step S142 to S145 on all the adjacent edges in the 16 pixels of the interpolation pixel P (S147: Yes, S142 to S145, S147: No), and the interpolation pixel P obtained. Information indicating edges within 16 pixels is output to the interpolation calculation unit 130 (S150), and the process is terminated.

  On the other hand, when it is determined in step S141 that there are no adjacent edges in 16 interpolation pixels P (S141: No), the edge merging unit 120 directly interpolates the edge pattern output from the edge pattern classification unit 116. It outputs to the calculating part 116 (S150), and complete | finishes a process.

  FIG. 32 is a flowchart showing details of the second interpolation calculation D2 performed by the interpolation calculation unit 130 when there are edges in 16 pixels of the interpolation pixel P. As shown in the figure, the interpolation calculation unit 130 determines on which side the interpolation pixel P is present across the edge in the 16-pixel region (S171). Then, a pixel located on the same side as the interpolation pixel P with respect to the edge is selected as a reference pixel used for the interpolation calculation (S172). Next, the interpolation calculation unit 130 performs an interpolation calculation based on the pixel value of the reference pixel to obtain the pixel value of the interpolation pixel P (S173).

Having described preferred embodiments of the present invention, a program of the interpolated pixel value calculation method Oyo BiSo location and therefore the present invention is not be limited to the embodiments described above, without departing from the gist of the present invention As long as the configurations of the above-described embodiments can be combined, increase / decrease, and change can be added.

  For example, in the above-described embodiment, the edges (adjacent edges) detected between two adjacent pixels of three pixels adjacent in series are merged at the position of the center pixel of the three pixels. However, the position is not limited to the position of the center pixel of the three pixels, and may be any position near the center pixel. In this case, the position where the edges are merged may be arbitrarily determined in the range near the center pixel. For example, the position where the edge is merged is smaller than a pixel within the range near the center pixel and having a smaller pixel value difference from the center pixel. Alternatively, a position closer to the pixel having the larger pixel value difference from the central pixel may be used.

  For example, the method for detecting whether or not there is an edge between pixels is not limited to the method used in the above-described embodiment, and the method exemplified in the description of the prior art may be used.

  Further, the interpolation calculation method is not limited to the bicubic interpolation calculation used in the above-described embodiment and the weighted addition according to the distance from the interpolation pixel, and other interpolation calculation methods such as a Gaussian filter are used. May be.

1 is a block diagram showing a configuration of an image enlargement / reduction apparatus according to a first embodiment of the present invention. The figure which shows the pixel arrangement | sequence of the image represented by image data The block diagram which shows the structure of the edge detection part 10 of the image expansion / contraction apparatus shown in FIG. The figure for demonstrating an example of the method of detecting whether an edge exists between adjacent pixels. The figure which shows the example of difference filter Table showing the relationship between the positive and negative of primary differences d1, d2, d3 and secondary differences d4, d5 and the profile shapes of four pixels adjacent in series (part 1) Table showing the relationship between the positive and negative of primary differences d1, d2, d3 and secondary differences d4, d5 and the profile shapes of four pixels adjacent in series (part 2) Table showing the relationship between the positive and negative of primary differences d1, d2, d3 and secondary differences d4, d5 and the profile shapes of four pixels adjacent in series (part 3) The figure which shows the example of the profile shape determined as an edge, although the difference of the pixel value of two adjacent pixels is very slight The block diagram which shows the structure of the interpolation calculating part 30 of the image expansion / contraction apparatus shown in FIG. Illustration for explaining the bicubic method The figure for demonstrating calculation of the pixel value of the interpolation pixel about the part determined to be an edge (2nd interpolation calculation B2) The figure for demonstrating calculation of the pixel value of the interpolation pixel about the part determined to be an edge (3rd interpolation calculation B3) A flowchart showing processing performed in the present embodiment Flow chart of edge detection processing A Flow chart of second interpolation calculation B2 Flow chart of third interpolation calculation B3 The figure which shows the pixel row | line set in order to detect an adjacent edge The block diagram which shows the structure of the image expansion / contraction apparatus by the 2nd Embodiment of this invention. The block diagram which shows the structure of the edge detection part 110 of the image expansion / contraction apparatus shown in FIG. The figure for demonstrating the process performed in the filtering part 112 The figure for demonstrating operation | movement of the edge pattern classification | category part 116. The figure which shows the edge pattern according to the position where an edge exists (the 1) The figure which shows the edge pattern according to the position where an edge exists (the 2) The figure which shows the edge pattern according to the position where an edge exists (the 3) The figure which shows an example of the result of the classification | category by the edge pattern classification | category part 116 (the 1) The figure which shows an example of the result of the classification | category by the edge pattern classification | category part 116 (the 2) The figure which shows the process performed in the edge merging part 120 The figure for demonstrating the interpolation calculation (2nd interpolation calculation D2) performed when an edge exists in 16 pixels surrounding the interpolation pixel P A flowchart showing processing performed in the present embodiment Flowchart of edge merging process C by edge merging unit 120 Flow chart of second interpolation calculation D2 The figure which shows the example of the relationship between the edge in a to-be-photographed object, and the edge detected in the digital image The figure which shows the example of the relationship between the thin line in a to-be-photographed object, and the edge detected in the digital image The figure which shows the example of the relationship between the edge in a to-be-photographed object, and the edge detected in the digital image

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Input part 10 Edge detection part 12 Filtering part 14 Judgment part 16 Edge pattern classification | category part 20 Edge merge part 30 Interpolation operation part 32 Determination part 34 Reference image selection part 36 Operation part 50 Control part 101 Input part 110 Edge detection part 112 Filtering part 114 Judgment Unit 116 Edge Pattern Classification Unit 120 Edge Merging Unit 130 Interpolation Operation Unit 150 Control Unit K Scale Ratio S0 Image Data S1 Scaled Image Data

Claims (9)

An interpolation pixel value calculation method for calculating a pixel value of an interpolation pixel for enlarging or reducing a digital image obtained by imaging a subject based on a pixel value of a neighboring pixel located in the vicinity of the interpolation pixel,
Detect an edge located between the neighboring pixels by a predetermined edge detection process ,
When the edge is detected by the edge detection process, the pixel value of the interpolation pixel is calculated based on the pixel value of the neighboring pixel on the side where the interpolation pixel is located on both sides of the edge,
By the edge detection processing, when the edge at any between two adjacent pixels of the three pixels adjacent in series on said digital image is detected, the merges the two edges 3 one of the single merged edges located near the center pixel among the pixels, and calculates the pixel value of the interpolation pixel based on the pixel values of the neighboring pixels on the side where the interpolation pixel is located among the opposite sides of said combination if edge An interpolation pixel value calculation method characterized by the above. Comparing pixel values of the three pixels of the two edges;
2. The interpolation according to claim 1, wherein the merging is performed on the two edges only when pixel values of the three pixels monotonously increase or monotonously decrease along a direction in which the three pixels are arranged. pixel value calculation method. 3. The interpolation pixel value calculation method according to claim 1, wherein the merging is performed on the two edges only when the extending directions of the two edges are the same. An interpolation pixel value having an interpolation pixel value calculation means for calculating a pixel value of an interpolation pixel for enlarging / reducing a digital image obtained by imaging a subject based on a pixel value of a neighboring pixel located in the vicinity of the interpolation pixel A calculation device,
Edge detection means for detecting an edge located between pixels of the neighboring pixels by a predetermined edge detection process ;
When the edge is detected in any of two adjacent pixels of the three pixels adjacent in series on the digital image by the edge detection means, the two edges are merged to and a edge merging means for a single merged edges located near the center pixel among the pixels,
The interpolation pixel value calculation means is
When the edge is detected by the edge detection means, the pixel value of the interpolation pixel is calculated based on the pixel value of the neighboring pixel on the side where the interpolation pixel is located on both sides of the edge;
When the two edges are merged by the edge merging means, the pixel value of the interpolation pixel is calculated based on the pixel value of the neighboring pixel on the side where the interpolation pixel is located on both sides of the merged edge An interpolated pixel value calculating apparatus characterized in that: The edge merging means compares the pixel values of the three pixels of the two edges, and only when the pixel values of the three pixels monotonously increase or decrease monotonously along the direction in which the three pixels are arranged. The interpolated pixel value calculation apparatus according to claim 4, wherein the merging is performed on the two edges. The edge detection means also detects the extending direction of the edge;
6. The interpolated pixel value calculation according to claim 4, wherein the edge merging unit performs the merging on the two edges only when the extending directions of the two edges are the same. apparatus. Causes a computer to execute an interpolation pixel value calculation process for calculating a pixel value of an interpolation pixel for enlarging / reducing a digital image obtained by imaging a subject based on a pixel value of a neighboring pixel located in the vicinity of the interpolation pixel A program,
An edge detection process for detecting an edge located between the neighboring pixels by a predetermined edge detection process;
If the edge is detected in any of two adjacent pixels of the three pixels adjacent in series on the digital image by the edge detection process , the two edges are merged to edge merging to one merged edges located near the center pixel among the pixels,
When the edge is detected by the edge detection process, the pixel value of the interpolation pixel is calculated based on the pixel value of the neighboring pixel on the side where the interpolation pixel is located on both sides of the edge,
Interpolation for calculating the pixel value of the interpolation pixel based on the pixel value of the neighboring pixel on the side where the interpolation pixel is located on both sides of the merged edge when the two edges are merged by the edge merge processing A program that causes a computer to execute pixel value calculation processing . The edge merging process compares the pixel values of the three pixels of the two edges, and only when the pixel values of the three pixels monotonically increase or decrease monotonously along the direction in which the three pixels are arranged. according to claim 7, wherein a program, characterized in that it performs the merge to the two edges. 9. The program according to claim 7 or 8, wherein the edge merging process performs the merging on the two edges only when the extending directions of the two edges are the same.

Images