JP6469397B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to a technique for changing the size of an image.

  Conventionally, when the image size is changed (enlarged or reduced), an interpolation process for deriving the value of each pixel of the image after the change by interpolation using the pixel of the image before the change has been performed. As such interpolation processing methods, for example, methods such as bilinear and bicubic are conventionally known.

  Note that there is Patent Document 1 as a prior art document disclosing a technique related to the present invention.

JP2013-126134A

  In the case where the image size is changed by performing the interpolation process using the conventional interpolation method as described above, in the contour (edge) of the image in the image after the change, The phenomenon called occurs. Jaggy generally occurs where the contour line is inclined with respect to the horizontal direction.

  Such jaggy damages the outline of the image in the image and impairs the beauty of the image. Therefore, a technique for reducing jaggies that occur when changing the size of an image has been desired.

  This invention is made | formed in view of the said subject, and it aims at providing the technique which can reduce jaggy.

In order to solve the above-mentioned problem, the invention of claim 1 is an image processing apparatus for generating an output image by changing the size of an input image, wherein one of a horizontal direction and a vertical direction of the input image is defined as an array direction. First interpolation means for performing interpolation processing using values of a plurality of input pixels and deriving values of a plurality of points of interest on an inclined interpolation line that is inclined with respect to a horizontal direction through the position of the target pixel of the output image Second interpolation means for deriving the value of the target pixel by performing interpolation processing using the values of the plurality of points of interest on the inclined interpolation line, and edge information for deriving information on the target edge related to the target pixel Deriving means, wherein the first interpolation means performs interpolation processing according to an angle of the target edge with respect to a horizontal direction, and the edge information deriving means is a reference area of the input image corresponding to the position of the target pixel. Special Then, the center of the identified reference region is determined as the peripheral position of the target pixel, and information on the target edge is derived based on the peripheral edges at a plurality of peripheral positions and the local position edge at the position of the target pixel. .

Further, the invention of claim 2, the image processing apparatus according to claim 1, wherein the first interpolation means, in accordance with the angle with respect to the horizontal direction of the target edge, the plurality of input pixels to be used for the interpolation processing Select either the horizontal direction or the vertical direction as the arrangement direction.

According to a third aspect of the present invention, in the image processing apparatus according to the second aspect, when the angle of the target edge with respect to the horizontal direction is θ, the first interpolation means is 45 ° <θ <135 °. The vertical direction is the arrangement direction of the plurality of input pixels used for the interpolation processing, and when 0 ° <θ <45 ° or 135 ° <θ <180 °, the plurality of inputs used for the interpolation processing The pixel arrangement direction is defined as the horizontal direction.

Further, the invention of claim 4, the image processing apparatus according to claim 1, wherein the peripheral position is the center of the four input pixels that form the minimum rectangular.

In addition, according to a fifth aspect of the present invention, in the image processing apparatus according to any one of the first to fourth aspects, the edge information deriving means includes edges of the plurality of peripheral positions and edges of the target pixel. Of these, the edge having the maximum strength is selected as the target edge.

The invention of claim 6 is the image processing apparatus according to claim 5 , wherein when the extending direction of the edge at any of the plurality of peripheral positions is substantially along either the horizontal direction or the vertical direction, A weakening means for weakening the strength of the edge is further provided.

According to a seventh aspect of the present invention, in the image processing device according to the fifth aspect , the intensity of the edge at each of the plurality of peripheral positions is determined according to the distance from the position of the target pixel at each of the plurality of peripheral positions. It further has a weakening means for weakening.

The invention according to claim 8 is the image processing apparatus according to any one of claims 5 to 7 , wherein the extending direction of the edge at the position of the target pixel is substantially along either the horizontal direction or the vertical direction. Further includes weakening means for weakening the strength of the edge.

The invention according to claim 9 is the image processing apparatus according to any one of claims 1 to 7 , wherein the extending direction of the edge at the position of the target pixel is substantially along either the horizontal direction or the vertical direction. Is further provided with third interpolation means for deriving the value of the target pixel by performing interpolation processing along the horizontal direction and the vertical direction instead of the first and second interpolation means.

The invention of claim 10 is an image processing method for generating an output image by changing the size of an input image, wherein (a) a plurality of input images are arranged in a horizontal direction or a vertical direction. (B) deriving values of a plurality of points of interest on an inclined interpolation line that is inclined with respect to the horizontal direction through the position of the target pixel of the output image, A step of performing an interpolation process using values of the plurality of points of interest on the inclined interpolation line, and deriving a value of the target pixel; and (c) deriving information of a target edge related to the target pixel. The step (a) performs an interpolation process according to the angle of the target edge with respect to the horizontal direction, and the step (c) specifies a reference region of the input image corresponding to the position of the target pixel. The center of the identified reference area The peripheral position of the target pixel is determined, and information on the target edge is derived based on the peripheral edges at a plurality of peripheral positions and the own position edge at the position of the target pixel .

The invention of claim 11 is a program that can be executed by a computer included in an image processing apparatus that generates an output image by changing the size of the input image. Interpolation processing using values of a plurality of input pixels having either the direction or the vertical direction as an array direction, and a plurality of attentions on an inclined interpolation line that is inclined with respect to the horizontal direction through the position of the target pixel of the output image (B) deriving the value of the target pixel by performing interpolation processing using the values of the plurality of points of interest on the inclined interpolation line; and (c) the target pixel. A step of deriving information on the target edge according to the method, wherein the step (a) performs an interpolation process according to an angle of the target edge with respect to a horizontal direction, and the step (c) Picture A reference area of the input image corresponding to the prime position is identified, the center of the identified reference area is determined as a peripheral position of the target pixel, a peripheral edge at a plurality of peripheral positions, and a self-position edge at the position of the target pixel Based on the above, information on the target edge is derived .

According to the first to eleventh aspects of the present invention, since the value of the target pixel of the output image is derived by the interpolation process using the values of the plurality of points of interest on the slope interpolation line, jaggy in the output image can be reduced.

Further, according to the first aspect of the invention, since the interpolation process is performed according to the angle of the target edge, jaggies in the output image can be reduced.

In particular, according to the second and third aspects of the invention, since the arrangement direction of the plurality of input pixels used for the interpolation processing is selected according to the angle of the target edge, jaggy in the output image can be effectively reduced.

In particular, according to the first aspect of the invention, not only the edge at the position of the target pixel but also the edges at a plurality of peripheral positions around the target pixel are taken into account, and information on the target edge related to the target pixel is appropriately derived. it can.

In particular, according to the invention of claim 4 , the edge at the peripheral position can be easily detected.

In particular, according to the invention of claim 5, the edge that most affects the target pixel can be set as the target edge related to the target pixel.

In particular, according to the invention of claim 6 , by reducing the strength of the edge where jaggies are hardly generated, the edge related to jaggy can be appropriately selected as the target edge.

In particular, according to the seventh aspect of the invention, since the edge strength is adjusted in consideration of the reduction in the influence due to the distance, the edge that affects the target pixel can be appropriately selected as the target edge.

In particular, according to the invention of claim 8 , it is possible to appropriately select the edge related to jaggy as the target edge by weakening the strength of the edge where jaggy is unlikely to occur.

In particular, according to the ninth aspect of the invention, when the extending direction of the edge at the position of the target pixel is substantially along either the horizontal direction or the vertical direction, jaggies are hardly generated. The generation of noise can be suppressed by performing the interpolation process along.

FIG. 1 is a diagram illustrating a schematic configuration of an image processing apparatus. FIG. 2 is a diagram illustrating the pixel position of the input image and the pixel position of the output image. FIG. 3 is a diagram illustrating an example of an output image generated by a conventional interpolation method. FIG. 4 is a diagram illustrating an example of an output image generated by the gradient interpolation method. FIG. 5 is a diagram illustrating a functional configuration of the size changing unit. FIG. 6 is a diagram showing an overall flow of processing related to the gradient interpolation method. FIG. 7 is a diagram illustrating a relationship between an edge in an image and a vector. FIG. 8 is a diagram for explaining a pattern for classifying edges. FIG. 9 is a diagram illustrating an example of an edge angle. FIG. 10 is a diagram illustrating an example of an edge angle. FIG. 11 is a diagram illustrating an interpolation process in the case of the pattern A. FIG. 12 is a diagram illustrating an interpolation process in the case of the pattern B. FIG. 13 is a diagram illustrating an interpolation process in the case of the pattern C. FIG. 14 is a diagram illustrating an interpolation process in the case of the pattern D. FIG. 15 is a diagram illustrating a configuration of the edge information deriving unit. FIG. 16 is a diagram illustrating a plurality of peripheral positions around the target pixel. FIG. 17 is a diagram showing a detailed flow of edge information acquisition processing. FIG. 18 is a diagram illustrating input pixels around the target pixel. FIG. 19 is a diagram illustrating a method for deriving a peripheral edge vector. FIG. 20 is a diagram illustrating an example of an edge whose extending direction is substantially along the vertical direction. FIG. 21 is a diagram showing the coefficient J. FIG. 22 is a diagram for explaining a method of deriving a vector of the own position edge. FIG. 23 is a diagram illustrating a configuration of the pixel interpolation unit. FIG. 24 is a diagram showing a detailed flow of pixel interpolation processing. FIG. 25 is a diagram illustrating neighboring pixels in the vicinity of the target pixel. FIG. 26 is a diagram for classifying neighboring pixels. FIG. 27 is a diagram illustrating an example of the angle of the slope interpolation line in the case of pattern A. In FIG. FIG. 28 is a diagram illustrating an interpolation process in the case of pattern A. FIG. 29 is a diagram illustrating an example of the angle of the slope interpolation line in the case of the pattern B. FIG. 30 is a diagram illustrating an interpolation process in the case of the pattern B. FIG. 31 is a diagram illustrating an example of the angle of the slope interpolation line in the case of the pattern C. FIG. 32 is a diagram illustrating an interpolation process in the case of the pattern C. FIG. 33 is a diagram illustrating an example of the angle of the inclined interpolation line in the case of the pattern D. In FIG. FIG. 34 is a diagram showing an interpolation process in the case of the pattern D. FIG. 35 is a diagram for explaining overshoot and undershoot. FIG. 36 is a diagram for describing an input pixel for obtaining an input pixel range. FIG. 37 is a diagram for explaining overshoot and undershoot. FIG. 38 is a diagram illustrating another example of the configuration of the pixel interpolation unit.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<1. Overview of Image Processing Device>
FIG. 1 is a diagram showing a schematic configuration of an image processing apparatus 1 according to the present embodiment. The image processing device 1 is, for example, an in-vehicle device mounted on a vehicle and has a function of displaying various images.

  The image processing apparatus 1 includes a camera 11, a video acquisition unit 12, and a display device 14. The camera 11 is mounted toward the outside of the vehicle and acquires an image around the vehicle. The video acquisition unit 12 acquires an image from a recording medium such as a DVD or a broadcasting medium such as a digital television broadcast. The display device 14 is disposed in the passenger compartment and displays various images. Therefore, the image processing apparatus 1 can display the image obtained by the camera 11 or the image obtained by the video obtaining unit 12 on the display device 14.

  The image processing apparatus 1 further includes a control unit 10 and an image processing unit 13.

  The control unit 10 comprehensively controls the entire image processing apparatus 1. The control unit 10 includes a computer including a CPU, RAM, ROM, and the like. Various functions as the control unit 10 are realized by the CPU performing arithmetic processing according to a program stored in a ROM or the like. Such a program is provided via a recording medium or a network.

  The image processing unit 13 is, for example, a hardware circuit (integrated circuit) having a function of performing various image processing. The image processing unit 13 performs image processing on the image obtained by the camera 11 or the image obtained by the video obtaining unit 12 so as to be suitable for display on the display device 14.

  The image processing unit 13 includes a size changing unit 2 as one of functions for performing such image processing. The size changing unit 2 changes the size of a source image such as an image obtained by the camera 11 or an image obtained by the video obtaining unit 12 and generates an image having a size suitable for display on the display device 14. .

  Hereinafter, the original image before the size change by the size change unit 2 is referred to as an “input image”, and the image after the size change generated by the size change unit 2 is referred to as an “output image”. In the following description, it is assumed that the size changing unit 2 generates the output image by increasing the size of the input image. However, the technique described below is applied to the case of generating the output image by reducing the size of the input image. May be.

  FIG. 2 is a diagram illustrating the pixel position of the input image and the pixel position of the output image. In the drawing, the position of the pixel Ps of the input image is represented by a circular symbol, and the position of the pixel Pg of the output image is represented by a square symbol (the same applies to the following drawings).

  The position of each pixel in the image and the interval between adjacent pixels are defined based on the size of the image (the number of pixels in the horizontal direction and the number of pixels in the vertical direction). Since the input image and the output image are different in size, as shown in FIG. 2, the position of each pixel Ps in the input image is different from the position of each pixel Pg in the output image. For this reason, the value (luminance value and color difference value) of each pixel Pg of the output image is derived by interpolation processing using the values of the plurality of pixels Pg of the input image around the position of the pixel Pg.

  In general, interpolation methods such as bilinear and bicubic are used for such interpolation processing. Bilinear is a technique for obtaining a pixel value by linearly interpolating 2 × 2 pixels (4 pixels) around a target pixel. Bicubic is a technique for obtaining pixel values by interpolating with 4 × 4 pixels (16 pixels) around a target pixel by a predetermined arithmetic expression. All of these conventional interpolation methods perform interpolation along the vertical and horizontal directions of the image. More specifically, in the conventional interpolation method, after performing interpolation along one of the vertical direction and the horizontal direction, interpolation is performed along the other of the vertical direction and the horizontal direction.

  On the other hand, unlike the conventional interpolation method, the image processing apparatus 1 according to the present embodiment interpolates along a line inclined with respect to the horizontal direction (hereinafter referred to as “gradient interpolation method”). )).

  3 and 4 are diagrams illustrating examples of output images generated by performing interpolation processing on the same input image Gs. FIG. 3 shows an example of an output image Ggc generated by employing bilinear which is a conventional interpolation method. On the other hand, FIG. 4 shows an example of an output image Gg generated by employing the gradient interpolation method. In these figures, the luminance value of each pixel is represented by hatching, and the darker the hatching, the lower the luminance value.

  In the input image Gs shown in FIG. 3 and FIG. 4, an edge (a part where the brightness changes discontinuously) serving as the contour of the image is inclined with respect to the horizontal direction. As shown in the lower part of FIG. 3, in the output image Ggc obtained by enlarging the input image Gs by a conventional interpolation method, a noticeable jaggy having a stepped contour line occurs. On the other hand, as shown in the lower part of FIG. 4, in the output image Gg obtained by enlarging the input image Gs by the gradient interpolation method, jaggy is greatly suppressed as compared with the conventional interpolation method. Hereinafter, such an inclined interpolation method will be described in detail.

<2. Overview of the gradient interpolation method>
In the following, first, an overall outline of the gradient interpolation method will be described. Thereafter, two processes included in the gradient interpolation method (edge information deriving process and pixel interpolation process) will be described in detail.

  FIG. 5 is a diagram illustrating a functional configuration of the size changing unit 2 that executes the gradient interpolation method. The size changing unit 2 includes an image memory 21, an edge information deriving unit 3, and a pixel interpolation unit 4.

  The image memory 21 stores an input image to be processed. The edge information deriving unit 3 and the pixel interpolation unit 4 perform various types of image processing using the input image stored in the image memory 21.

  The edge information deriving unit 3 performs edge information deriving processing for deriving edge information related to the pixels of the output image based on the input image. In addition, the pixel interpolation unit 4 performs a pixel interpolation process that performs an interpolation process based on the input image to derive a pixel value of the output image. The pixel interpolation unit 4 performs an interpolation process according to the edge information derived by the edge information deriving unit 3.

  FIG. 6 is a diagram illustrating an overall flow of processing related to the gradient interpolation method executed by the size changing unit 2.

  First, the edge information deriving unit 3 executes edge information deriving processing for each pixel of the output image. The edge information deriving unit 3 selects one pixel of the output image as a “target pixel” to be processed (step S1), and executes edge information deriving processing for the target pixel (step S2). The edge information deriving unit 3 repeats the edge information deriving process while sequentially selecting each of the pixels included in the output image as a target pixel in accordance with the arrangement order (No in step S3). Thereby, the edge information deriving unit 3 executes the edge information deriving process for all the pixels of the output image.

  In the edge information derivation process (step S2), the edge information derivation unit 3 derives edge information related to the target pixel. The edge information deriving unit 3 handles the edges in the image as vectors in the calculation. FIG. 7 is a diagram illustrating a relationship between an edge E in the image and a vector V representing the edge E. As shown in the figure, the direction of the vector V is orthogonal to the extending direction of the edge E. The magnitude of the vector V corresponds to the intensity of the edge E.

  The horizontal direction of the image is defined as the X-axis direction, and the vertical direction is defined as the Y-axis direction. The edge information deriving unit 3 specifies the vector V using the component Vx in the X-axis direction and the component Yy in the Y-axis direction. The edge information deriving unit 3 derives the two components Vx and Vy of the vector V based on pixel values (luminance values) of the input image around the target pixel (details will be described later).

  Then, the edge information deriving unit 3 classifies the edge related to the target pixel into one of four patterns based on the angle with respect to the horizontal direction. The classified pattern becomes a part of edge information related to the target pixel. Hereinafter, the symbol θ is used for the angle of the edge with respect to the horizontal direction. The angle θ of the edge is derived by a trigonometric function using the two components Vx and Vy of the vector V, and is set to a value in the range of 0 ° <θ <180 ° using the frequency method.

  FIG. 8 is a diagram for explaining a pattern for classifying edges. As shown in the figure, the edge information deriving unit 3 performs pattern A when 0 ° <θ <45 °, pattern B when 45 ° <θ <90 °, and 90 ° <θ <135 °. Classify the edges into pattern C and pattern D when 135 ° <θ <180 °.

  For example, as shown in FIG. 9, when the edge angle θ is 30 °, the edge information deriving unit 3 classifies the edge into the pattern A. Also, as shown in FIG. 10, when the edge angle θ is 120 °, the edge information deriving unit 3 classifies the edge into the pattern C.

  The edge information deriving unit 3 performs this edge information deriving process on all the pixels of the output image (Yes in step S3 in FIG. 6), and thereby the edge information (pattern and the like) related to all the pixels of the output image. Is derived. The edge information deriving unit 3 delivers the derived edge information to the pixel interpolation unit 4 (step S4).

  Next, the pixel interpolation unit 4 performs pixel interpolation processing on each pixel of the output image. The pixel interpolation unit 4 selects one pixel of the output image as a “target pixel” to be processed (step S5), and performs pixel interpolation processing on the target pixel (step S6). The pixel interpolation unit 4 repeats the pixel interpolation process while sequentially selecting each of the pixels included in the output image as a target pixel according to the arrangement order (No in step S7). Thereby, the pixel interpolation unit 4 performs pixel interpolation processing on all the pixels of the output image.

  In the pixel interpolation process (step S6), the pixel interpolation unit 4 performs the interpolation process according to the edge pattern related to the target pixel to derive the value (luminance value and color difference value) of the target pixel. FIG. 11 to FIG. 14 are diagrams for explaining the interpolation processing executed by the pixel interpolation unit 4. 11 shows the interpolation process in the case of pattern A, FIG. 12 shows the case of pattern B, FIG. 13 shows the case of pattern C, and FIG. In either case, the pixel interpolation unit 4 performs two-stage interpolation processing (first interpolation processing and second interpolation processing). Hereinafter, the pixel Ps of the input image is referred to as “input pixel”.

  First, the pixel interpolation unit 4 performs a first interpolation process using values of a plurality of input pixels Ps whose arrangement direction is either the horizontal direction or the vertical direction of the input image. Thereby, the pixel interpolation unit 4 derives values of a plurality of attention points N1 to N4 on the inclined interpolation line La that passes through the position of the target pixel Pgt and is inclined with respect to the horizontal direction.

  Next, the pixel interpolation unit 4 performs a second interpolation process using the values of the plurality of attention points N1 to N4 on the gradient interpolation line La derived in the first interpolation process, and derives the value of the target pixel Pgt. In both the first interpolation process and the second interpolation process, the same calculation expression as that of bicubic is employed as the calculation expression for interpolation.

  The arrangement direction of the plurality of input pixels Ps used in the first interpolation process and the angle of the inclined interpolation line La with respect to the horizontal direction vary depending on the edge pattern of the target pixel. Hereinafter, the symbol α is used as the angle of the inclined interpolation line La with respect to the horizontal direction. The angle α of the inclined interpolation line La is set to a value in the range of 0 ° <α <180 ° using the power method.

  As shown in FIG. 11, in the case of the pattern A (0 ° <θ <45 °), the arrangement direction of the plurality of input pixels Ps used for the first interpolation processing is “horizontal direction”. Further, the angle α of the slope interpolation line La is set in a range of 0 ° <α <90 ° so that the slope interpolation line La becomes “upwardly rightward”.

  As the first interpolation process, the pixel interpolation unit 4 performs an interpolation process using four input pixels Ps arranged in the “horizontal direction” in each of the four regions R1, R2, R3, and R4 shown in FIG. Therefore, 4 × 4 16 input pixels Ps are used in the first interpolation process. Thereby, the pixel interpolation unit 4 derives the value of the attention point N1 in the region R1, the value of the attention point N2 in the region R2, the value of the attention point N3 in the region R3, and the value of the attention point N4 in the region R4. These attention points N1 to N4 are four points on the inclined interpolation line La that pass through the position of the target pixel Pgt and become “rising upward”. Then, the pixel interpolation unit 4 performs the second interpolation process using the values of the four attention points N1 to N4 arranged on the inclined interpolation line La as the second interpolation process, and derives the value of the target pixel Pgt.

  As shown in FIG. 12, in the case of the pattern B (45 ° <θ <90 °), the arrangement direction of the plurality of input pixels Ps used for the first interpolation process is the “vertical direction”. Further, the angle α of the slope interpolation line La is set in a range of 0 ° <α <90 ° so that the slope interpolation line La becomes “upwardly rightward”.

  The pixel interpolation unit 4 performs an interpolation process using four input pixels Ps arranged in the “vertical direction” in each of the four regions R1, R2, R3, and R4 shown in FIG. The values of the attention points N1 to N4 are derived. These attention points N1 to N4 are four points on the inclined interpolation line La that pass through the position of the target pixel Pgt and become “rising upward”. Then, the pixel interpolation unit 4 performs the second interpolation process using the values of the four attention points N1 to N4 arranged on the inclined interpolation line La as the second interpolation process, and derives the value of the target pixel Pgt.

  As shown in FIG. 13, in the case of the pattern C (90 ° <θ <135 °), the arrangement direction of the plurality of input pixels Ps used for the first interpolation processing is “vertical direction”. Further, the angle α of the slope interpolation line La is set in a range of 90 ° <α <180 ° so that the slope interpolation line La becomes “downwardly right”.

  As the first interpolation process, the pixel interpolation unit 4 performs an interpolation process using the four input pixels Ps arranged in the “vertical direction” in each of the four regions R1, R2, R3, and R4 illustrated in FIG. The values of the attention points N1 to N4 are derived. These attention points N1 to N4 are the four points on the slope interpolation line La that pass through the position of the target pixel Pgt and become “downward to the right”. Then, the pixel interpolation unit 4 performs the second interpolation process using the values of the four attention points N1 to N4 arranged on the inclined interpolation line La as the second interpolation process, and derives the value of the target pixel Pgt.

  Further, as shown in FIG. 14, in the case of the pattern D (135 ° <θ <180 °), the arrangement direction of the plurality of input pixels Ps used for the first interpolation processing is “horizontal direction”. Further, the angle α of the slope interpolation line La is set in a range of 90 ° <α <180 ° so that the slope interpolation line La becomes “downwardly right”.

  As the first interpolation process, the pixel interpolation unit 4 performs an interpolation process using four input pixels Ps arranged in the “horizontal direction” in each of the four regions R1, R2, R3, and R4 illustrated in FIG. The values of the attention points N1 to N4 are derived. These attention points N1 to N4 are the four points on the slope interpolation line La that pass through the position of the target pixel Pgt and become “downward to the right”. Then, the pixel interpolation unit 4 performs the second interpolation process using the values of the four attention points N1 to N4 arranged on the inclined interpolation line La as the second interpolation process, and derives the value of the target pixel Pgt.

  The pixel interpolation unit 4 performs such pixel interpolation processing on all the pixels of the output image (Yes in step S7 in FIG. 6), thereby deriving the values of all the pixels of the output image. Thereby, the size changing unit 2 generates an output image in which the size of the input image is changed.

  As described above, in the image processing apparatus 1 according to the present embodiment, the pixel interpolation unit 4 uses the values of the plurality of input pixels Ps whose arrangement direction is either the horizontal direction or the vertical direction of the input image. Process. Thereby, the pixel interpolation unit 4 derives values of a plurality of attention points N1 to N4 on the inclined interpolation line La that passes through the position of the target pixel Pgt of the output image and is inclined with respect to the horizontal direction. Further, the pixel interpolation unit 4 performs the second interpolation process using the values of the plurality of attention points N1 to N4 on the inclined interpolation line La, and derives the value of the target pixel Pgt.

  Whether the arrangement direction of the input pixels Ps used for the first interpolation processing is the “horizontal direction” or the “vertical direction” is selected according to the edge pattern of the target pixel (that is, the edge angle θ). In the case of the pattern A or the pattern D (when 0 ° <θ <45 ° or 135 ° <θ <180 °), the arrangement direction of the input pixels Ps used for the first interpolation processing is “horizontal direction”. . On the other hand, in the case of the pattern B or the pattern C (when 45 ° <θ <135 °), the arrangement direction of the plurality of input pixels Ps used for the first interpolation processing is “vertical direction”.

  That is, when the absolute value of the inclination of the edge of the target pixel with respect to the horizontal direction is relatively small (when the inclination is gentle), the arrangement direction of the input pixels Ps used for the first interpolation processing is set to the “horizontal direction”. On the other hand, when the absolute value of the inclination of the edge of the target pixel with respect to the horizontal direction is relatively large (when the inclination is steep), the arrangement direction of the input pixels Ps used for the first interpolation processing is set to “vertical direction”. Accordingly, since the first interpolation process is performed along a direction that approximates the extending direction of the edge, jaggies can be effectively reduced.

  Further, the angle α with respect to the horizontal direction of the inclined interpolation line La is set according to the edge pattern of the target pixel (that is, the edge angle θ). In the case of the pattern A or the pattern B (when 0 ° <θ <90 °), the angle α of the inclined interpolation line La is set in the range of 0 ° <α <90 °. On the other hand, in the case of pattern C or pattern D (when 90 ° <θ <180 °), the angle α of the inclined interpolation line La is set in the range of 90 ° <α <180 °. That is, when the edge of the target pixel is “rising upward”, the slope interpolation line La is also set to “rising upward”. On the other hand, when the edge of the target pixel is “lower right shoulder”, the inclined interpolation line La is also set to “lower right shoulder”. As a result, since the second interpolation process is performed along a direction that approximates the extending direction of the edge, jaggies can be effectively reduced.

<3. Edge information derivation process>
Next, the edge information derivation process (step S2 in FIG. 6) will be described in more detail. FIG. 15 is a diagram illustrating a detailed configuration of the edge information deriving unit 3 (see FIG. 5) that executes the edge information deriving process.

  The edge information deriving unit 3 includes a first edge processing unit 31, a second edge processing unit 32, an edge selection unit 33, and an edge classification unit 34.

  As shown in FIG. 16, the first edge processing unit 31 detects edges at a plurality of peripheral positions Cs around the target pixel Pgt (hereinafter referred to as “peripheral edges”), and detects the plurality of peripheral edges. Perform the targeted processing. The first edge processing unit 31 includes an edge detection unit 31a, a first weakening unit 31b, and a second weakening unit 31c. Details of these processes will be described later. In the drawing, the peripheral position Cs is represented by hatched diamond symbols (the same applies to the following drawings).

  The second edge processing unit 32 detects an edge at the position of the target pixel Pgt (hereinafter referred to as “local position edge”), and performs processing for the local position edge. The second edge processing unit 31 includes an interpolation unit 32a, a third weakening unit 32b, and a strengthening unit 32c. Details of these processes will be described later.

  The edge selection unit 33 selects an edge (hereinafter referred to as an edge) related to a target pixel used for the subsequent processing from the peripheral edge processed by the first edge processing unit 31 and the local position edge processed by the second edge processing unit 32. , Referred to as “target edge”). The edge selection unit 33 selects an edge that has the most influence on the target pixel Pgt as the target edge. More specifically, the edge selection unit 33 selects, as a target edge, an edge having the maximum intensity among a plurality of peripheral edges and the own position edge.

  Further, the edge classification unit 34 classifies the target edge related to the target pixel Pgt into one of the four patterns shown in FIG. The pattern of the target edge is referred to in the pixel interpolation process (step S6 in FIG. 6) executed by the pixel interpolation unit 4.

  FIG. 17 is a diagram showing a detailed flow of the edge information acquisition process (step S2 in FIG. 6). As described above, this edge information acquisition processing is executed for all the pixels of the output image.

  First, the edge detection unit 31a of the first edge processing unit 31 detects peripheral edges at a plurality of peripheral positions Cs around the target pixel Pgt based on the input pixels Ps around the target pixel Pgt (step S10). .

  FIG. 18 is a diagram illustrating the input pixels Ps around the target pixel Pgt. The edge detection unit 31a refers to the 4 × 4 input pixels Ps around the target pixel Pgt. The edge detection unit 31a sets the centers of the four input pixels Ps forming the smallest rectangle (square) in the 16 input pixels Ps as the peripheral positions Cs to be noted. Then, the edge detection unit 31a detects an edge at each of the 3 × 3 peripheral positions Cs as a “peripheral edge”.

  The edge detection unit 31a derives a peripheral edge vector V based on the difference in luminance value between the four input pixels Ps surrounding the peripheral position Cs in the X-axis direction and the Y-axis direction. By deriving this vector V, the edge detector 31a substantially detects the peripheral edge. Assuming that the luminance values of the four input pixels Ps surrounding the peripheral position Cs are values in parentheses shown in FIG. 19, the edge detection unit 31a has the X-axis direction component Vx and the Y-axis direction component Yy of the peripheral edge vector V. Are derived by the following equations (1) and (2), respectively.

Vx = (d12 + d22) − (d11 + d21) (1)
Vy = (d11 + d12) − (d21 + d22) (2)
When the edge detection unit 31a detects the peripheral edge, the first edge processing unit 31 and the second edge processing unit 32 perform processing in parallel thereafter. 17, steps S11 to S16 are processes by the first edge processing unit 31, and steps S21 to S25 are processes by the second edge processing unit 32.

  In the first edge processing unit 31, the first weakening unit 31b and the second weakening unit 31c adjust the strength of each of the plurality of peripheral edges. On the other hand, in the second edge processing unit 32, the interpolation unit 32a derives the local position edge, and the third weakening unit 32b and the strengthening unit 32c adjust the strength of the local position edge. Such edge strength adjustment is performed toward the selection of the target edge by the edge selection unit 33 in the subsequent step S31.

  The first edge processing unit 31 first selects one peripheral edge from among a plurality of peripheral edges as a “target peripheral edge” to be processed (step S11). Then, the first weakening unit 31b derives the strength of the target peripheral edge (step S12).

  For example, the first weakening unit 31b uses the two components Vx and Vy of the target peripheral edge vector V to derive the intensity Se of the target peripheral edge by the following equation (3). In formula (3), I is a predetermined constant.

Se = (Vx ^ 2 + Vy ^ 2) / I (3)
Next, the first weakening unit 31b determines whether or not the extending direction of the target peripheral edge is substantially along either the horizontal direction or the vertical direction (step S13). As shown in FIG. 20, when the extending direction of the target peripheral edge E is substantially along either the horizontal direction or the vertical direction, one of the two components Vx and Vy of the vector V of the target peripheral edge is very high. Corresponds to the small case. For this reason, the first weakening unit 31b uses a smaller one of the two components Vx and Vy as a reference value, and when the reference value is smaller than a predetermined threshold, the extension direction of the target peripheral edge is the horizontal direction and the vertical direction. It is judged that it is substantially along one of the following.

  When the extending direction of the target peripheral edge is substantially along either the horizontal direction or the vertical direction, the first weakening unit 31b weakens the strength of the target peripheral edge according to the following equation (4) (step S14). In Equation (4), Se is the edge strength, and J is a coefficient (0 <J <1).

Se = Se · J (4)
The coefficient J in equation (4) is given by the following equation (5), where Vs is a reference value (the smaller of the two components Vx and Vy) and Th is a threshold value to be compared with the reference value Vs. . That is, the coefficient J takes the value shown in the graph of FIG.

J = Vs / Th (5)
Thus, the 1st weakening part 31b weakens the intensity | strength Se of the object periphery edge whose extending | stretching direction is substantially along either the horizontal direction or the vertical direction. Since jaggies are unlikely to occur with respect to such target peripheral edges, it is possible to avoid the edge selecting unit 33 from selecting an edge with which jaggies are not likely to be generated as the target edge by reducing the intensity Se of the target peripheral edge.

  Next, the second weakening unit 31b weakens the intensity Se of the target peripheral edge according to the distance from the position of the target pixel Pgt of the peripheral position Cs related to the target peripheral edge (step S15). The second weakening unit 31b weakens the intensity Se of the target peripheral edge according to the following equation (6). In Expression (6), K is a predetermined constant, and L is the distance from the position of the target pixel Pgt at the peripheral position.

Se = Se · (KL) / K (6)
Since the peripheral position Cs related to the target peripheral edge is away from the position of the target pixel Pgt, the extent to which the target peripheral edge affects the target pixel Pgt is reduced according to the distance. Since the second weakening unit 31b weakens the intensity Se of the target peripheral edge as the distance from the position of the target pixel Pgt of the peripheral position Cs increases, the second peripheral weakening unit 31b adjusts the intensity Se of the target peripheral edge in consideration of a decrease in the influence due to the distance. it can. As a result, the edge selection unit 33 can appropriately select an edge that affects the target pixel Pgt as the target edge.

  The first edge processing unit 31 executes the processes in steps S11 to S15 for each of the plurality of peripheral edges (No in step S16). For this reason, the strength Se of each peripheral edge is weakened according to the extending direction and distance. When first edge processing unit 31 completes the processing for all the peripheral edges (Yes in step S16), the process proceeds to step S31.

  On the other hand, in the second edge processing unit 32, first, the interpolation unit 32a detects the own position edge (step S21). The interpolation unit 32a substantially detects the self-position edge by deriving the vector V of the self-position edge. As illustrated in FIG. 22, the interpolation unit 32 a derives a vector V of the local position edge by linearly interpolating the peripheral edge vectors V of the four peripheral positions Cs closest to the position of the target pixel Pgt. Then, the interpolation unit 32a derives the strength Se of the local position edge using the above-described equation (3) (step S22).

  Next, the 3rd weakening part 32b determines whether the extending | stretching direction of an own position edge is substantially along either the horizontal direction or the vertical direction (step S23). And the 3rd weakening part 32b weakens the intensity | strength Se of an own position edge, when the extending | stretching direction of an own position edge is substantially along either the horizontal direction or the vertical direction (step S24). As a specific process performed by the third weakening unit 32b, the same process as that of the first weakening unit 31b can be employed. Note that the threshold Th used in the third weakening unit 32b and the first weakening unit 31b may be changed.

  Thus, the 3rd weakening part 32b weakens the intensity | strength Se of the self-position edge where the extending | stretching direction is substantially along either the horizontal direction or the vertical direction. For this reason, it is possible to weaken the strength Se of the local position edge where jaggies are unlikely to occur. As a result, it is possible to avoid the edge selection unit 33 from selecting an edge where jaggies are unlikely to occur as a target edge.

  Next, the strengthening unit 32c increases the strength Se of the local position edge (step S25). The strengthening part 32c weakens the strength Se of the own position edge by the following equation (7). In Expression (7), M is a coefficient (1 <M).

Se = Se · M (7)
Since the own position edge is an edge at the position of the target pixel, the degree of influence on the target pixel is larger than that of the peripheral edge. By strengthening the strength Se of the local position edge by the strengthening unit 32c, the strength Se of the local position edge can be appropriately adjusted according to the degree of the influence. As a result, the edge selection unit 33 can appropriately select the edge that most affects the target pixel as the target edge. When the process of the strengthening unit 32c is completed, the process proceeds to step S31.

  When the processing of both the first edge processing unit 31 and the second edge processing unit 32 is completed, the edge selection unit 33 selects the target edge related to the target pixel (step S31). The edge selection unit 33 selects a target edge from among a plurality of peripheral edges whose strength Se has been adjusted by the first edge processing unit 31 and the local position edge whose strength Se has been adjusted by the second edge processing unit 32. As described above, the edge selection unit 33 selects an edge having the maximum strength Se as the target edge.

  Thus, the edge selection unit 33 selects the target edge in consideration of not only the local position edge at the position of the target pixel but also the peripheral edges at a plurality of peripheral positions around the target pixel. For this reason, it is possible to appropriately select the edge that most affects the target pixel as the target edge related to the target pixel.

  Next, the edge classification unit 34 classifies the target edge related to the target pixel into one of the four patterns shown in FIG. 8 (step S32). As described with reference to FIG. 8, the edge classification unit 34 classifies the target edge based on the angle θ with respect to the horizontal direction of the target edge.

  The pattern classified by the edge classification unit 34 becomes a part of edge information passed to the pixel interpolation unit 4. Note that the edge information passed to the pixel interpolation unit 4 may include the angle θ, the intensity Se, and the components Vx and Vy of the vector V that are derived in the edge information derivation process.

  If the angle θ of the target edge is 45 ° or 135 ° and the target edge cannot be classified into any of the four patterns, the result of classifying the local position edge related to the target pixel instead of the target edge is set as the target pixel. What is necessary is just to use as a pattern of such an edge.

<4. Pixel interpolation processing>
Next, the pixel interpolation process (step S6 in FIG. 6) will be described in more detail. FIG. 23 is a diagram illustrating a detailed configuration of the pixel interpolation unit 4 (see FIG. 5) that executes the pixel interpolation processing.

  The pixel interpolation unit 4 includes an inclination angle setting unit 41, a first interpolation processing unit 42, a second interpolation processing unit 43, and a pixel value correction unit 44.

  The inclination angle setting unit 41 sets an angle α with respect to the horizontal direction of the inclination interpolation line La. In the tilt interpolation method, the effect of the interpolation process is the greatest when the angle α of the tilt interpolation line La is 45 ° or 135 °. As the angle α of the inclined interpolation line La approaches either the horizontal direction or the vertical direction (the further from 45 ° or 135 °), the effect of the interpolation process becomes smaller. In addition to the effect of suppressing jaggies, the effect of this interpolation process also has the effect of amplifying noise. For this reason, the inclination angle setting unit 41 uses 45 ° or 135 ° as a reference angle, and changes the angle α of the inclination interpolation line La from the reference angle under a specific condition, thereby reducing the effect of the interpolation processing. 11 to 14 described above all represent the first interpolation process and the second interpolation process when the angle α of the tilt interpolation line La is the reference angle (45 ° or 135 °).

  The first interpolation processing unit 42 performs a first interpolation process using values of a plurality of input pixels whose arrangement direction is either the horizontal direction or the vertical direction of the input image, and a plurality of points of interest on the inclined interpolation line La. Are derived respectively. The second interpolation processing unit 43 performs a second interpolation process using values of a plurality of points of interest on the inclined interpolation line La, and derives the value of the target pixel. In addition, the pixel value correcting unit 44 corrects the value of the target pixel when overshoot or undershoot occurs in the derived target pixel value.

  FIG. 24 is a diagram showing a detailed flow of the pixel interpolation process (step S6 in FIG. 6). As described above, this pixel interpolation processing is executed for all the pixels of the output image. Further, at the start of this pixel interpolation processing, edge information (patterns, etc.) relating to all the pixels of the output image is transferred from the edge information deriving unit 3 to the pixel interpolation unit 4. Hereinafter, an edge related to a target pixel to be subjected to pixel interpolation processing is referred to as a “target edge”.

  In the pixel interpolation process, first, the tilt angle setting unit 41 sets the angle α of the tilt interpolation line La related to the target pixel according to the pattern of the target edge related to the target pixel (steps S40 to S43). Then, the first interpolation processing unit 42 performs the first interpolation processing according to the set angle α of the inclined interpolation line La (step S44), and further, the second interpolation processing unit 42 performs the second interpolation processing, and the target A pixel value is derived (step S45).

  First, the inclination angle setting unit 41 sets the angle α of the inclination interpolation line La related to the target pixel as a reference angle as an initial value (step S40).

  As described above, the inclination angle setting unit 41 sets the angle α of the inclination interpolation line La in the range of 0 ° <α <90 ° in the case of the pattern A or the pattern B (when 0 ° <θ <90 °). In the case of the pattern C or D (90 ° <θ <180 °), the angle α of the inclined interpolation line La is set in the range of 90 ° <α <180 °. Therefore, in the case of pattern A or pattern B (0 ° <θ <90 °), the reference angle is 45 °, and in the case of pattern C or pattern D (90 ° <θ <180 °), the reference angle. Is 135 °.

  Next, the inclination angle setting unit 41 pays attention to a plurality of pixels in the output image in the vicinity of the target pixel (hereinafter referred to as “neighboring pixels”), and the pattern of the target edge related to the target pixel and the edge related to the neighboring pixel. The pattern is compared (step S41).

  FIG. 25 is a diagram illustrating the target pixel Pgt in the output image and a plurality of neighboring pixels Pg in the vicinity of the target pixel Pgt. As shown in the figure, the tilt angle setting unit 41 focuses on neighboring pixels Pg included in a 5 × 5 pixel centered on the target pixel Pgt in the output image. Then, the inclination angle setting unit 41 compares the target edge pattern related to the target pixel Pgt with the edge pattern related to the neighboring pixel Pg.

  In FIG. 25, the pattern of the edge relating to each of the target pixel Pgt and the neighboring pixel Pg is indicated by a symbol (A or D). In the example of FIG. 25, the target edge related to the target pixel Pgt is the pattern A. On the other hand, the edge of most neighboring pixels Pg is pattern A, but the edge of one neighboring pixel Pgx is pattern D.

  Usually, the pattern of the target edge related to the target pixel Pgt matches the pattern of the edge related to the neighboring pixel Pg. However, when there is a neighboring pixel (hereinafter referred to as “atypical pixel”) Pgx having an edge pattern different from that of the target pixel Pgt, there is a high possibility that noise exists in the vicinity of the target pixel Pgt. Such noise may be amplified in the interpolation process.

  For this reason, in order to prevent amplification of such noise, when the atypical pixel Pgx exists (Yes in step S42 in FIG. 24), the inclination angle setting unit 41 sets the angle α of the inclination interpolation line La. The reference angle is changed (step S43). The inclination angle setting unit 41 increases the degree of changing the angle α of the inclination interpolation line La from the reference angle as the distance between the target pixel Pgt and the atypical pixel Pgx is shorter.

  FIG. 26 is a diagram in which the neighboring pixels Pg are classified based on the distance relationship with the target pixel Pgt. In the figure, reference signs D1 to D5 attached to each neighboring pixel Pg represent the degree of distance from the target pixel Pgt. The neighboring pixel Pg with the reference sign D1 is closest to the target pixel Pgt, and the neighboring pixel Pg with the reference sign D5 is farthest from the target pixel Pgt. Therefore, when the neighboring pixel Pg with the reference sign D1 is the atypical pixel Pgx, the inclination angle setting unit 41 changes the angle of the inclination interpolation line La relatively large from the reference angle. On the other hand, when the neighboring pixel Pg denoted by reference sign D5 is the atypical pixel Pgx, the inclination angle setting unit 41 changes the angle α of the inclination interpolation line La from the reference angle to be relatively small.

  For example, the tilt angle setting unit 41 is 30 ° for the neighboring pixel Pg with the reference D1, 25 ° for the neighboring pixel Pg with the reference D2, 20 ° for the neighboring pixel Pg with the reference D3, and the reference D4. The angle α of the inclined interpolation line La is changed from the reference angle by 15 ° with respect to the neighboring pixel Pg marked with and 10 ° with respect to the neighboring pixel Pg marked with D5. In the case of FIG. 25, since the neighboring pixel Pg to which the reference sign D2 is attached becomes the atypical pixel Pgx, the inclination angle setting unit 41 changes the angle α of the inclination interpolation line La by 25 ° from the reference angle.

  If there are a plurality of atypical pixels Pgx in the vicinity of the target pixel Pgt, the tilt angle setting unit 41 changes the angle α of the tilt interpolation line La from the reference angle based on the atypical pixel Pgx closest to the target pixel Pgt. To do.

  The direction in which the tilt angle setting unit 41 changes the angle α of the tilt interpolation line La differs depending on the pattern of the target edge related to the target pixel.

  As shown in FIG. 27, in the case of pattern A (0 ° <θ <45 °), the inclination angle setting unit 41 changes the angle α of the inclination interpolation line La to a direction increasing from the reference angle of 45 °. Therefore, the inclination angle setting unit 41 sets the angle α of the inclination interpolation line La in a range of 45 ° ≦ α <90 °.

  FIG. 28 shows the first interpolation process and the second interpolation process when the angle α of the inclined interpolation line La is changed from the reference angle in the case of the pattern A. As can be seen by comparing FIG. 11 and FIG. 28, when the angle α of the tilt interpolation line La is changed from the reference angle, the input pixel Ps closer to the target pixel Pgt is compared to the case where the angle α is the reference angle. Used for interpolation processing. For this reason, the effect (the effect which reduces jaggy and the effect | action which amplifies noise) by an interpolation process can be relieved.

  As shown in FIG. 29, in the case of the pattern B (45 ° <θ <90 °), the inclination angle setting unit 41 changes the angle α of the inclination interpolation line La to a direction decreasing from the reference angle of 45 °. Therefore, the inclination angle setting unit 41 sets the angle α of the inclination interpolation line La in a range of 0 ° <α ≦ 45 °.

  FIG. 30 shows the first interpolation process and the second interpolation process when the angle α of the inclined interpolation line La is changed from the reference angle in the case of the pattern B. As can be seen by comparing FIG. 12 and FIG. 30, when the angle α of the inclined interpolation line La is changed from the reference angle, the input pixel Ps closer to the target pixel Pgt is compared to the case where the angle α is the reference angle. Used for interpolation processing. For this reason, the effect (the effect which reduces jaggy and the effect | action which amplifies noise) by an interpolation process can be relieved.

  As shown in FIG. 31, in the case of pattern C (90 ° <θ <135 °), the inclination angle setting unit 41 changes the angle α of the inclination interpolation line La to a direction increasing from the reference angle of 135 °. Therefore, the inclination angle setting unit 41 sets the angle α of the inclination interpolation line La in a range of 135 ° ≦ α <180 °.

  FIG. 32 shows the first interpolation process and the second interpolation process when the angle α of the inclined interpolation line La is changed from the reference angle in the case of the pattern C. As can be seen by comparing FIG. 13 and FIG. 32, when the angle α of the tilt interpolation line La is changed from the reference angle, the input pixel Ps closer to the target pixel Pgt is compared to the case where the angle α is the reference angle. Used for interpolation processing. For this reason, the effect (the effect which reduces jaggy and the effect | action which amplifies noise) by an interpolation process can be relieved.

  As shown in FIG. 33, in the case of the pattern D (135 ° <θ <180 °), the inclination angle setting unit 41 changes the angle α of the inclination interpolation line La to a direction decreasing from the reference angle of 135 °. Therefore, the inclination angle setting unit 41 sets the angle α of the inclination interpolation line La in a range of 90 ° <α ≦ 135 °.

  FIG. 34 shows the first interpolation process and the second interpolation process when the angle α of the inclined interpolation line La is changed from the reference angle in the case of the pattern D. As can be seen by comparing FIG. 14 and FIG. 34, when the angle α of the tilt interpolation line La is changed from the reference angle, the input pixel Ps closer to the target pixel Pgt is compared with the case where the angle α is the reference angle. Used for interpolation processing. For this reason, the effect (the effect which reduces jaggy and the effect | action which amplifies noise) by an interpolation process can be relieved.

  When the first and second interpolation processes (steps S44 and S45) are completed and the value of the target pixel is derived, the pixel value correcting unit 44 then overshoots or undershoots the derived target pixel value. If this occurs, the value of the target pixel is corrected (steps S46 to S48).

  As described above, in the first interpolation process and the second interpolation process, the same calculation expression as that of bicubic is employed as the calculation expression for interpolation. For this reason, as shown in FIG. 35, at the location where the pixel value B1 of the input image changes abruptly, the minimum value B1 of the pixel of the input image from which the pixel value B2 of the image after interpolation processing is the interpolation source. A phenomenon (overshoot or undershoot) that deviates from the range from the value Bmin to the maximum value Bmax may occur. When this phenomenon occurs, if the image after the interpolation processing is used as an output image as it is, the user who visually recognizes the output image may feel uncomfortable. Therefore, when this phenomenon occurs, the pixel value correction unit 44 corrects the pixel value B2 so as to approach the range (Bmin to Bmax) of the pixel value B1 of the input image, as shown in the lower part of FIG. The result is defined as a pixel value B3 of the output image.

  The pixel value correcting unit 44 first acquires a pixel value range (hereinafter referred to as “input pixel range”) as an interpolation source (step S46). The pixel value correcting unit 44 selects four input pixels Ps for obtaining an input pixel range from the 16 input pixels Ps used in the first interpolation process.

  As illustrated in FIG. 36, the pixel value correcting unit 44 selects four input pixels Pst that are adjacent to the inclined interpolation line La and closest to the target pixel Pgt. In other words, the pixel value correcting unit 44 selects the four input pixels Pst at the center among the 16 input pixels Ps used in the first interpolation process. Then, the pixel value correcting unit 44 acquires the maximum value Bmax and the minimum value Bmin of the four selected input pixels Pst, and acquires the range from the minimum value Bmin to the maximum value Bmax as the input pixel range.

  Next, the pixel value correcting unit 44 determines whether or not the value of the target pixel derived by the interpolation process is out of the input pixel range (step S47). If the value of the target pixel is out of the input pixel range, the pixel value correcting unit 44 corrects the value of the target pixel so as to approach the input pixel range (step S48).

  The pixel value correcting unit 44 corrects the target pixel value Bg using the following formula (8) in the case of overshoot, and corrects the target pixel value using the following formula (9) in the case of undershoot. To do. In these formulas (8) and (9), Bg is the value of the target pixel, and N is a coefficient (0 <N <1).

Bg = Bg− (Bg−Bmax) · N (8)
Bg = Bg + (Bmin−Bg) · N (9)
In this way, the interpolation processing using the gradient interpolation line La is performed in order to correct the value of the target pixel based on the input pixel range defined by the maximum value and the minimum value of the four input pixels Pst in consideration of the gradient interpolation line La. The overshoot and undershoot caused by the above can be corrected appropriately.

  In this embodiment, the pixel value correcting unit 44 corrects the value of the target pixel so as to approach the input pixel range. However, as shown in FIG. 37, the value of the target pixel is limited within the input pixel range. May be. In this case, the coefficient N in the above equations (8) and (9) may be set to 1.

<5. Modification>
Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications are possible. Below, such a modification is demonstrated. All the forms including the above-described embodiment and the form described below can be appropriately combined.

  In the above-described embodiment, in step S23 of the edge information acquisition process (see FIG. 17), if the extension direction of the own position edge is substantially along either the horizontal direction or the vertical direction, the strength Se of the own position edge. I was trying to weaken. On the other hand, when the extending direction of the own position edge is substantially along either the horizontal direction or the vertical direction, the horizontal direction and the vertical direction are used instead of the inclined interpolation method (first and second interpolation processes). The value of the target pixel may be derived by performing interpolation processing of a conventional interpolation method such as bilinear along In this case, as shown in FIG. 38, the pixel interpolation unit 4 includes a third interpolation processing unit 45 that performs an interpolation process using a conventional interpolation method. When the extending direction of the own position edge is substantially along either the horizontal direction or the vertical direction, jaggies are hardly generated at the position of the target pixel. Therefore, the first and second interpolations are performed by performing the conventional interpolation process. Generation of noise due to processing can be suppressed.

  In the above embodiment, in step S31 of the edge information acquisition process (see FIG. 17), the target edge is derived by selecting the edge having the maximum strength from the plurality of peripheral edges and the own position edge. It was. On the other hand, a weight corresponding to the distance from the position of the target pixel may be adopted to derive the target edge by a weighted average of a plurality of peripheral edges and the own position edge.

  Further, any peripheral edge may be selected as the peripheral edge to be referred to when the target edge is derived in step S31 of the edge information acquisition process.

  In the first interpolation process and the second interpolation process of the above-described embodiment, the same arithmetic expression as that of bicubic is used. However, other well-known arithmetic expressions such as bilinear and Lanczos 3 may be employed. The number of input pixels Ps used for the interpolation process is also changed according to the arithmetic expression employed.

  In the pixel interpolation process of the above embodiment, the inclination angle setting unit 41 changes the angle α of the inclination interpolation line La from the reference angle when the atypical pixel Pgx exists. Instead of this, or in addition to this, the inclination angle setting unit 41 may change the angle α of the inclination interpolation line La with respect to the horizontal direction from the reference angle according to the strength of the target edge. When the intensity of the target edge is relatively small, noise may be amplified if interpolation processing with a large effect is performed. For this reason, it is desirable to increase the degree of changing the angle α of the inclined interpolation line La from the reference angle as the intensity of the target edge is smaller. According to this, the effect (the effect of reducing jaggies and the effect of amplifying noise) by the interpolation process can be adjusted according to the intensity of the target edge.

  Moreover, the constants and coefficients used in the above-described formulas may be changed as appropriate.

  In the above embodiment, all or part of the functions of the size changing unit 2 described as functions by the hardware circuit may be realized by software. For example, the same function as that of the size changing unit 2 may be realized by the CPU of the control unit 10 performing arithmetic processing according to a program.

  Further, although the image processing apparatus 1 of the above-described embodiment is an in-vehicle apparatus, by executing a program on a general-purpose computer such as a personal computer or a tablet, the computer has the same function as the above. It may function as a device.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 2 Size change part 3 Edge information derivation part 4 Pixel interpolation part 33 Edge selection part 34 Edge classification part 41 Inclination angle setting part 42 1st interpolation process part 43 2nd interpolation process part Pgt Target pixel Ps Input pixel

Claims (11)

An image processing apparatus that generates an output image by changing the size of an input image,
Inclination interpolation that performs interpolation processing using values of a plurality of input pixels whose arrangement direction is either the horizontal direction or the vertical direction of the input image, and is inclined with respect to the horizontal direction through the position of the target pixel of the output image First interpolation means for deriving values of a plurality of points of interest on the line,
A second interpolation unit that performs an interpolation process using the values of the plurality of points of interest on the inclined interpolation line, and derives the value of the target pixel;
Edge information deriving means for deriving information of the target edge related to the target pixel,
The first interpolation means performs an interpolation process according to an angle of the target edge with respect to a horizontal direction;
The edge information deriving means includes
A reference area of the input image corresponding to the position of the target pixel is specified, a center of the specified reference area is determined as a peripheral position of the target pixel, and a peripheral edge at a plurality of peripheral positions and an automatic position at the position of the target pixel are determined. An image processing apparatus , wherein information on the target edge is derived based on a position edge . The image processing apparatus according to claim 1 .
The first interpolation means selects whether the array direction of the plurality of input pixels used for the interpolation processing is a horizontal direction or a vertical direction according to an angle of the target edge with respect to a horizontal direction. An image processing apparatus. The image processing apparatus according to claim 2 ,
When the angle of the target edge with respect to the horizontal direction is θ,
The first interpolation means includes
When 45 ° <θ <135 °, the arrangement direction of the plurality of input pixels used for the interpolation processing is a vertical direction,
When 0 ° <θ <45 ° or 135 ° <θ <180 °, the image processing apparatus is characterized in that the arrangement direction of the plurality of input pixels used for the interpolation processing is a horizontal direction. The image processing apparatus according to claim 1 .
The image processing apparatus according to claim 1, wherein the peripheral position is a center of four input pixels forming a minimum rectangle. The image processing apparatus according to any one of claims 1 to 4 ,
The image processing apparatus according to claim 1, wherein the edge information deriving unit selects, as the target edge, an edge having the maximum intensity among the edges at the plurality of peripheral positions and the edge at the position of the target pixel. The image processing apparatus according to claim 5 .
When the extending direction of the edge at any of the plurality of peripheral positions is substantially along either the horizontal direction or the vertical direction, weakening means for weakening the strength of the edge,
An image processing apparatus further comprising: The image processing apparatus according to claim 5 .
A weakening means for weakening the strength of the edge in each of the plurality of peripheral positions according to the distance from the position of the target pixel in each of the plurality of peripheral positions;
An image processing apparatus further comprising: The image processing apparatus according to any one of claims 5 to 7 ,
Weakening means for weakening the strength of the edge when the extending direction of the edge at the position of the target pixel is substantially along either the horizontal direction or the vertical direction;
An image processing apparatus further comprising: The image processing apparatus according to any one of claims 1 to 7 ,
When the extending direction of the edge at the position of the target pixel is substantially along either the horizontal direction or the vertical direction, interpolation processing along the horizontal direction and the vertical direction is performed instead of the first and second interpolation means. Third interpolation means for deriving the value of the target pixel;
An image processing apparatus further comprising: An image processing method for generating an output image by changing the size of an input image,
(A) performing an interpolation process using values of a plurality of input pixels whose arrangement direction is either the horizontal direction or the vertical direction of the input image, and passing through the position of the target pixel of the output image and tilting with respect to the horizontal direction Deriving values of a plurality of points of interest on the slope interpolation line
(B) performing an interpolation process using values of the plurality of points of interest on the inclined interpolation line, and deriving the value of the target pixel;
(C) deriving information of a target edge related to the target pixel;
Equipped with a,
The step (a)
Perform interpolation processing according to the angle of the target edge with respect to the horizontal direction,
The step (c)
A reference area of the input image corresponding to the position of the target pixel is specified, a center of the specified reference area is determined as a peripheral position of the target pixel, and a peripheral edge at a plurality of peripheral positions and an automatic position at the position of the target pixel are determined. An image processing method , wherein information on the target edge is derived based on a position edge . A program that can be executed by a computer included in an image processing apparatus that changes the size of an input image to generate an output image,
(A) performing an interpolation process using values of a plurality of input pixels whose arrangement direction is either the horizontal direction or the vertical direction of the input image, and passing through the position of the target pixel of the output image and tilting with respect to the horizontal direction Deriving values of a plurality of points of interest on the slope interpolation line
(B) performing an interpolation process using values of the plurality of points of interest on the inclined interpolation line, and deriving the value of the target pixel;
(C) deriving information of a target edge related to the target pixel;
Was executed,
The step (a)
Perform interpolation processing according to the angle of the target edge with respect to the horizontal direction,
The step (c)
A reference area of the input image corresponding to the position of the target pixel is specified, a center of the specified reference area is determined as a peripheral position of the target pixel, and a peripheral edge at a plurality of peripheral positions and an automatic position at the position of the target pixel are determined. A program for deriving information on the target edge based on a position edge .

Images