JP5781370B2 - Image processing apparatus, image processing method, image display apparatus including image processing apparatus, program, and recording medium - Google Patents

Description

  The present invention relates to an image processing device that performs high-resolution processing, an image processing method, an image display device including the image processing device, a program, and a recording medium.

  Upscaling processing is known as an important technique used in image processing and video processing. For example, with the widespread use of displays that support FHD (Full High Definition) resolution in recent years, upscaling processing is required to display video content with SD (Standard Definition) resolution on an FHD resolution display. In addition, a display corresponding to a QFHD (Quad Full High Definition) resolution exceeding the FHD resolution has been developed, and a display having a UHD (Ultra High Definition) resolution exceeding the QFHD is also being studied. Thus, as display resolution increases, the importance of high-performance upscaling processing is increasing.

  The upscaling process includes interpolation methods such as nearest neighbor interpolation (nearest neighbor), bilinear interpolation (bilinear interpolation), and bicubic interpolation (bicubic interpolation). Nearest-neighbor interpolation is performed by using the pixel value closest to the reference position. When the enlargement ratio is high, some of the same pixels continue, resulting in poor gradation reproduction, and jagged edges. Become. Bilinear interpolation is to obtain pixel values by linearly interpolating pixel values using 2 × 2 pixels (4 pixels) around the reference position, and linear interpolation is performed with higher accuracy than nearest neighbor interpolation. The pixel value is blurred because it is used. Bicubic interpolation is a method of obtaining pixel values by interpolating with a cubic equation using 4 × 4 pixels (16 pixels) around a reference position. Although the performance is better than nearest neighbor interpolation or bilinear interpolation, edge There is a problem that jaggy occurs in the part.

  Patent Document 1 describes a method for upscaling images and videos with high quality without requiring complicated processing. In the upscaling process described in Patent Document 1, first, a diagonal high resolution (HR) pixel is generated using low resolution input pixels adjacent in the diagonal direction. Next, a horizontal high resolution (HR) pixel is generated using a low resolution input pixel adjacent in the horizontal direction and an oblique high resolution (HR) pixel adjacent in the vertical direction. Finally, a high resolution (HR) image is generated by generating vertical high resolution (HR) pixels using low resolution input pixels adjacent in the vertical direction and diagonal high resolution (HR) pixels adjacent in the horizontal direction. Generate. Further, when generating each high resolution (HR) pixel, control is performed such that the direction close to the edge direction is heavily weighted, and the direction away from the edge direction is small. This suppresses the jaggy art factor without causing overshoot.

JP 2010-67272 A (published March 25, 2010)

  However, the upscaling process described in Patent Document 1 is for generating an interpolation pixel located in the middle of input low resolution pixels. The lower right pixel value of the low resolution input pixel is calculated, and then the right pixel value and the lower pixel value of the low resolution input pixel are calculated using the calculation result of the lower right pixel value. Since three new output pixels are generated from low-resolution input pixels, it is difficult to realize an arbitrary magnification enlargement process other than double.

  In addition, it is possible to perform arbitrary magnification enlargement processing by using bilinear interpolation or bicubic interpolation as a conventional technique. However, this results in the problems of these interpolation methods described above. End up.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to realize an arbitrary magnification enlargement process and an image processing apparatus and an image process capable of performing image processing with suppressed deterioration in image quality A method, an image display device including an image processing device, a program, and a recording medium are provided.

  In order to solve the above-described problem, the image processing apparatus of the present invention enlarges or reduces the image data by providing an interpolation pixel between input pixels included in the input image data according to a set enlargement / reduction ratio. In the image processing apparatus, for each input pixel, an edge direction indicating an edge direction or an edge direction estimation processing unit that estimates an edge gradient shifted by a predetermined angle from the edge direction, and an input existing in the vicinity of the interpolation pixel When the direction of the edge in the interpolation pixel specified using the edge direction or edge gradient of the pixel is the interpolation pixel edge direction, it is an area along the interpolation pixel edge direction and an area surrounding the interpolation pixel. An interpolating area setting unit that sets an interpolating area, and a temporary pixel that is a pixel between input pixels is set in the interpolating area. A temporary pixel value calculation processing unit for calculating a value, characterized in that it comprises an interpolation processor for calculating the value of the interpolation pixel by using a value of the input pixel and the temporary pixel included in the interpolation region.

  According to the above configuration, since the interpolation pixels are provided between the input pixels included in the input image data according to the set enlargement / reduction ratio, the interpolation pixels can be provided regardless of the enlargement / reduction ratio. Thereby, an arbitrary magnification enlargement process can be realized. Further, using the edge direction or edge gradient of the input pixel existing in the vicinity of the interpolation pixel, the interpolation pixel edge direction indicating the direction of the edge in the interpolation pixel is specified, and in the region along the specified interpolation pixel edge direction And an interpolation area that is an area surrounding the interpolation pixel is set, and the value of the interpolation pixel is calculated using the values of the input pixel and the temporary pixel included in the interpolation area. The interpolation area can be set. Furthermore, since the value of the interpolation pixel is calculated using the values of the input pixel and the temporary pixel included in the interpolation area set along the interpolation pixel edge direction, the interpolation area is set regardless of the edge direction. In comparison, jaggy caused by the interpolation process can be suppressed. In particular, when the interpolation pixel edge direction is an oblique direction, jaggy in the oblique direction can be suppressed. Furthermore, since the temporary pixel is calculated at a position where no input pixel exists in the interpolation area, the distance between the pixels in the interpolation area can be kept moderate. In particular, the distance from the interpolation pixel to be interpolated can be kept moderate as compared with the case where only the input pixel exists on the interpolation plane. Thereby, since the distance between pixels does not become long, it can prevent that a window size becomes large and memory consumption increases. In addition, it is possible to suppress deterioration in image quality. Therefore, it is possible to provide an image processing apparatus that can perform arbitrary magnification enlargement processing and can perform image processing in which deterioration in image quality is suppressed.

  Furthermore, in the image processing apparatus of the present invention, the interpolation area setting unit includes two sides parallel to the first axis along the interpolation pixel edge direction, and two sides parallel to the second axis in a direction different from the first axis. It is preferable to set a parallelogram area consisting of as an interpolation area.

  According to the above configuration, since the set interpolation region is a parallelogram region, the shape is suitable for being along the interpolation pixel edge direction.

  Furthermore, in the image processing apparatus of the present invention, the interpolation processing unit can calculate the value of the interpolation pixel using the values of a plurality of pixels arranged along the first axis and the second axis in the interpolation region. preferable.

  According to said structure, the interpolation calculation which considered the interpolation pixel edge direction can be performed easily.

  Furthermore, in the image processing apparatus of the present invention, the temporary pixel value calculation processing unit includes a temporary pixel, sets a temporary pixel value calculation area along the interpolation pixel edge direction, and sets the temporary pixel value calculation area in the temporary pixel value calculation area. It is preferable to calculate the value of the temporary pixel using the value of the included input pixel.

  Specifically, the temporary pixel value calculation processing unit uses the value of the input pixel adjacent to the temporary pixel in the interpolation pixel edge direction or a direction within a predetermined angle range from the interpolation pixel edge direction. May be calculated.

  According to the above configuration, the value of the temporary pixel can be brought close to the original value. Thereby, an image can be kept smooth.

  Alternatively, in the image processing apparatus of the present invention, the temporary pixel value calculation processing unit temporarily calculates the edge direction of the temporary pixel specified using the edge direction or edge gradient of the input pixel existing in the vicinity of the temporary pixel. When the pixel edge direction is set, a temporary pixel value calculation area including the temporary pixel and along the temporary pixel edge direction is set, and the temporary pixel value calculation area is set using the input pixel value included in the temporary pixel value calculation area. It is preferable to calculate the pixel value.

  Specifically, the temporary pixel value calculation processing unit sets the edge direction of the temporary pixel specified by using the edge direction or the edge gradient of the input pixel existing in the vicinity of the temporary pixel as the temporary pixel edge direction. At this time, the value of the temporary pixel may be calculated using the value of the input pixel adjacent in the temporary pixel edge direction or the direction within the predetermined angle range from the temporary pixel edge direction.

  The interpolated pixel edge direction may differ from the interim pixel edge direction, and the interpolated pixel edge direction is more accurate than the interpolated pixel edge direction specified by using an input pixel located closer to the interim pixel. is there. For this reason, according to said structure, the value of a temporary pixel can be made closer to an original value. Thereby, an image can be kept smooth.

  Furthermore, it is preferable that the image processing apparatus of the present invention further includes a correction processing unit that sharpens the image data after the interpolation calculation by the interpolation processing unit.

  According to said structure, the blurring which arises after interpolation calculation can be suppressed.

  Furthermore, in the image processing apparatus of the present invention, the correction processing unit is configured such that the maximum pixel value and the minimum pixel value in pixels in a predetermined range including the target pixel are calculated from the image data after the interpolation calculation by the interpolation processing unit. A maximum value / minimum value calculation processing unit, a filter processing unit for performing sharpening processing, a maximum pixel value and a minimum pixel value calculated by the maximum value / minimum value calculation processing unit, and the above It is preferable to include a saturation processing unit that controls the degree of sharpening based on the filter processing result calculated by the filter processing unit and the pixels of the image data after the interpolation calculation by the interpolation processing unit.

  According to the above configuration, since the degree of sharpening is controlled, it is possible to suppress the occurrence of undershoot and overshoot that are annoying for human eyes. As a result, it is possible to correct the blur caused by the interpolation calculation.

  In order to solve the above-described problem, the image processing method of the present invention enlarges or reduces the image data by providing an interpolation pixel between input pixels included in the input image data according to a set enlargement / reduction ratio. An image processing method, for each input pixel, estimating an edge direction indicating an edge direction or an edge gradient shifted by a predetermined angle from the edge direction, and an input pixel existing in the vicinity of the interpolation pixel When the edge direction of the interpolation pixel specified using the edge direction or the edge gradient is the interpolation pixel edge direction, the interpolation is an area along the interpolation pixel edge direction and an area surrounding the interpolation pixel. A step of setting a region, and a step of setting a temporary pixel that is a pixel between input pixels in the interpolation region and calculating a value of the temporary pixel. And flop, characterized in that it comprises the steps of calculating the value of the interpolation pixel by using a value of the input pixel and the temporary pixel included in the interpolation region.

  According to the above configuration, it is possible to provide an image processing method capable of realizing an arbitrary magnification enlargement process and performing image processing in which deterioration in image quality is suppressed.

  The image processing apparatus of the present invention may be realized by a computer. In this case, a program for realizing the image processing apparatus by the computer by operating the computer as each of the units, and the program are recorded. Computer-readable recording media are also within the scope of the present invention.

  The present invention provides, for each input pixel, an edge direction indicating an edge direction, or an edge direction estimation processing unit that estimates an edge gradient shifted from the edge direction by a predetermined angle, and an input pixel existing in the vicinity of the interpolation pixel. When the edge direction of the interpolation pixel specified using the edge direction or the edge gradient is the interpolation pixel edge direction, it is an area along the interpolation pixel edge direction and an area surrounding the interpolation pixel. Included in the interpolation area setting section that sets an interpolation area, a temporary pixel value calculation processing section that sets a temporary pixel that is a pixel between input pixels in the interpolation area, and that calculates a value of the temporary pixel. An interpolation processing unit that calculates the value of the interpolated pixel using the value of the pixel to be processed. As a result, it is possible to realize an arbitrary magnification enlargement process and to perform an image process in which a reduction in image quality is suppressed.

It is a block diagram which shows the outline of a structure of the television receiver in one of this Embodiment. It is a block diagram which shows the structure of a video signal processing circuit. It is a block diagram which shows the structure of a scaler process part. It is a figure which shows a Sobel filter. It is a flowchart which shows the flow of an edge direction estimation process. It is a figure which shows an example of the interpolation surface set in the 4x4 block containing an interpolation pixel. It is a 1st figure which shows an example of the temporary pixel calculated by the temporary pixel calculation process. It is a 2nd figure which shows an example of the temporary pixel calculated by the temporary pixel calculation process. It is a 3rd figure which shows an example of the temporary pixel calculated by the temporary pixel calculation process. It is a figure which shows the parameter used for an interpolation calculation with a 1st interpolation surface. It is a figure which shows the parameter used for an interpolation calculation with a 2nd interpolation surface. It is a figure which shows the parameter used for an interpolation calculation with a 3rd interpolation surface. It is a figure which shows the parameter used for an interpolation calculation with a 4th interpolation surface. It is a figure which shows the parameter used for an interpolation calculation with a 5th interpolation surface. It is a figure which shows the parameter used for an interpolation calculation with a 6th interpolation surface. It is a figure which shows an example of distribution of a weighting coefficient. It is a figure which shows an example of the interpolation surface set in the 6x6 block containing an interpolation pixel. It is a figure which shows the parameter used for an interpolation calculation with a 7th interpolation surface. FIG. 7 is a diagram illustrating an example of four interpolation planes set in a 4 × 4 block instead of FIG. 6. It is a flowchart which shows the flow of the process performed by a video signal processing circuit. It is a block diagram which shows the structure of the video signal processing circuit in a modification. It is a figure which shows the structure of a correction process part. It is a figure which shows an example of the suppression effect of an overshoot and an undershoot. It is a flowchart which shows the flow of the process performed by the video signal processing circuit in a modification. It is a flowchart which shows the flow of a saturation process. It is a figure which shows an example of the multi display comprised with the television receiver in this Embodiment. It is a figure which shows an example of a Prewitt filter. It is a figure which shows a Scherr filter. It is a figure which shows an example of a template. It is a figure which shows the relationship between the pixel value and position of an interpolation pixel. It is a figure which shows the relationship between an edge direction and an edge gradient.

  Embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

  FIG. 1 is a block diagram showing an outline of the configuration of a television receiver according to one embodiment. As shown in FIG. 1, a television receiver 1 is an integrated circuit (LSI) including a central processing unit (CPU) 11 that controls the whole, a video signal processing circuit 21, an audio signal processing circuit 23, and a panel controller 25. 3, a power supply unit 5, a tuner 7, an interface 9, a display unit 13, an operation unit 15, and a speaker 17.

  The interface 9 includes a TV antenna 41, a digital visual interface (DVI) terminal 43 and a high-definition multimedia interface (HDMI) and a high-definition multimedia interface (TMD) terminal 43 for serial communication using a TMDS (Transition Minimized Differential Signaling) method. And a LAN terminal 47 for communicating with a communication protocol such as UDP (User Datagram Protocol). The interface 9 transmits / receives data to / from an external device connected to the DVI terminal 43, the HDMI terminal 45, or the LAN terminal 47 in accordance with an instruction from the CPU 11.

  The operation unit 15 includes at least a power switch and a changeover switch. The power switch is a switch for inputting an operation instruction for instructing to turn on or off the power of the television receiver 1. The change-over switch is a switch for inputting an operation instruction for designating a broadcast channel to be received by the television receiver 1. The operation unit 15 outputs an operation instruction corresponding to each switch to the CPU 11 in response to pressing of the power switch and the changeover switch.

  Here, the case where the operation unit 15 included in the television receiver 1 is operated has been described as an example, but the operation unit 15 is provided in a remote controller capable of wirelessly communicating with the television receiver 1. Thus, an operation instruction corresponding to each switch may be transmitted to the television receiver 1. In this case, the communication medium with which the remote controller communicates with the television receiver 1 may be infrared light or electromagnetic waves.

  The display unit 13 is, for example, a liquid crystal display (LCD), a plasma display panel, or the like, and can display a television program.

  The power supply unit 5 controls electric power supplied from the outside. The CPU 11 supplies power to the power supply unit 5 or cuts off supply of power in accordance with an operation instruction input from the power switch. When the operation instruction input from the power switch is an operation instruction to switch on the power, when power is supplied to the entire television receiver 1 and the operation instruction input from the power switch is an operation instruction to switch off the power The electric power supplied to the television receiver 1 is cut off.

  The tuner 7 is, for example, a terrestrial digital tuner or a BS / CS digital tuner, is connected to the TV antenna 41, and receives a broadcast signal received by the TV antenna 41. The broadcast signal is a high frequency digital modulation signal including video data and audio data. The tuner 7 includes a deinterleave circuit and an error correction circuit, and demodulates the extracted high frequency digital modulation signal of a specific frequency to generate a variable length code. The tuner 7 selects a broadcast signal of a channel designated by the CPU 11 from the broadcast signals received by the TV antenna 41, and uses the video signal processing circuit 21 and the audio signal for the variable length code generated based on the broadcast signal. Output to the processing circuit 23.

  The video signal processing circuit 21 performs various processes on the variable length code input from the tuner 7 to generate a video signal.

  The panel controller 25 controls the display unit 13 to display the video of the video signal output from the video signal processing circuit 21 on the display unit 13. Specifically, the panel controller 25 includes a panel drive circuit and a backlight control circuit, controls the drive of the panel drive circuit in accordance with an instruction from the CPU 11, and turns on the backlight disposed on the back surface of the display unit 13. Controls turning off.

  The audio signal processing circuit 23 includes an audio decoder, decodes the variable length code input from the tuner 7 to generate an audio signal, performs D / A (digital / analog) conversion on the audio signal, and outputs the speaker 17. Output to.

  Here, the case where the tuner 7 generates the video signal based on the broadcast signal received by the TV antenna 41 has been described as an example. However, the external device includes the DVI terminal 43 and the HDMI terminal. The video signal processing circuit 21 may generate the video signal based on the video data transmitted via the 45 or LAN terminal 47. The tuner 7 may include both a terrestrial digital tuner and a BS / CS digital tuner.

  FIG. 2 is a block diagram showing the configuration of the video signal processing circuit. As shown in FIG. 2, the video signal processing circuit 21 includes a video decoder 51, an IP conversion processing unit 53, a noise reduction processing unit 55, a sharpness processing unit 57, a color adjustment processing unit 59, and a scaler processing unit. 61.

  The video decoder 51 decodes the variable length code input from the tuner 7 to generate a video signal, performs D / A conversion on the video signal, and outputs the video signal to the IP conversion processing unit 53. The IP conversion processing unit 53 converts the video signal input from the video decoder 51 from the interlace method to the progressive method as necessary. The noise reduction processing unit 55 executes noise reduction processing for reducing noise components included in the video signal output from the IP conversion processing unit 53. The sharpness processing unit 57 executes sharpness processing that sharpens the video of the video signal output from the noise reduction processing unit 55. The color adjustment processing unit 59 performs color processing for adjusting contrast, saturation, and the like on the video signal output from the noise reduction processing unit 55. The scaler processing unit 61 performs a scaling process on the video signal output from the color adjustment processing unit 59 according to the number of pixels of the display unit 13. Note that the CPU 11 appropriately stores a video signal when various processing is executed by the video signal processing circuit 21 in a frame memory (not shown) in units of frames.

  The scaling process enlarges (or reduces) the frame-unit image data included in the video data to a magnification according to the number of pixels of the display unit 13, and calculates pixels that need interpolation by enlargement (or reduction) as interpolation pixels. Including at least processing to perform. The position of the interpolation pixel is determined by the magnification at which the image data is enlarged (or reduced). Specifically, it is the position between the input pixel and the input pixel in the vertical direction, the horizontal direction, and the diagonal direction in the image data. For example, when the magnification is 2, the interpolation pixel is set at a position that equally divides the input pixel between the input pixel and the input pixel in the vertical direction, the horizontal direction, and the diagonal direction. In general, when image data is enlarged n times, interpolation pixels are set at positions that equally divide four input pixels adjacent in the vertical and horizontal directions by n × n.

  FIG. 3 is a block diagram illustrating a configuration of the scaler processing unit. As shown in FIG. 3, the scaler processing unit 61 includes an edge direction estimation processing unit 71, an interpolation plane setting unit 73, a provisional pixel value calculation processing unit 75, and an interpolation processing unit 77.

  Based on the video signal input from the color adjustment processing unit 59, the edge direction estimation processing unit 71 estimates the edge direction of the input pixel in the image data in units of frames or an edge gradient shifted by a predetermined angle from the edge direction. To do. Here, the input pixel is a pixel existing in the input image data. The edge direction, that is, the direction of the edge line segment is orthogonal to the direction of the edge gradient. In the present embodiment, the edge direction estimation processing unit 71 estimates the edge direction by determining an angle range that includes the direction of the edge gradient (that is, the direction shifted by 90 degrees from the edge direction). The direction of the edge gradient can be calculated using a known technique. Here, a case will be described as an example in which the direction θ of the edge gradient in the horizontal direction and the vertical direction is calculated using the Sobel filter. Note that the direction θ of the edge gradient is represented by an angle with respect to the X axis, which is the horizontal direction.

  FIG. 4 is a diagram illustrating the Sobel filter. FIG. 4A is a diagram illustrating a Sobel filter corresponding to the horizontal direction. FIG. 4B is a diagram illustrating a Sobel filter corresponding to the vertical direction. The edge direction estimation processing unit 71 uses the first-order differentiation in the horizontal direction (X-axis direction) and the vertical direction (Y-axis direction) calculated using the Sobel filter shown in FIGS. The direction θ of the edge gradient is calculated. The first-order differential gradX in the horizontal direction (detecting the edge in the vertical direction) is calculated according to the equation (1) using the Sobel filter shown in FIG.

gradX = IN (x + 1, y−1) + IN (x + 1, y) × 2
+ IN (x + 1, y + 1) -IN (x-1, y-1)
−IN (x−1, y) × 2−IN (x−1, y + 1) (1)
However, IN (x, y) represents the pixel density of the input pixel at the coordinates (x, y).

  The first-order differential gradY in the vertical direction (detecting an edge in the horizontal direction) is calculated according to the equation (2) using the Sobel filter shown in FIG.

gradY = IN (x-1, y + 1) + IN (x, y + 1) × 2
+ IN (x + 1, y + 1) -IN (x-1, y-1)
−IN (x, y−1) × 2-IN (x + 1, y−1) (2)
However, IN (x, y) represents the pixel density of the input pixel at the coordinates (x, y).

  Here, an edge direction estimation process for determining an angle range including the direction of the edge gradient using the horizontal first-order differential gradX and the vertical first-order differential gradY will be described. FIG. 5 is a flowchart showing the flow of the edge direction estimation process. The edge direction estimation process is a process executed by executing an edge direction estimation program stored in a ROM or HDD. As shown in FIG. 5, it is determined whether or not the edge strength calculated by the following equation 1 is smaller than a predetermined threshold T (for example, 16) (step S01). If the edge strength is smaller than predetermined threshold value T, the process proceeds to step S02; otherwise, the process proceeds to step S03.

  In step S02, it is determined that the direction of the edge gradient is indefinite, that is, it is impossible to determine the direction θ of the edge gradient.

  In step S03, it is determined whether or not a condition of the following formula (4) is satisfied. In other words, it is determined whether the code of gradX is different from the code of gradY. If the code of gradX and the code of gradY are different, the process proceeds to step S13. If not, the process proceeds to step S04.

gradX × gradY <0 Equation (4)
In step S04, it is determined whether or not a condition of the following equation (5) is satisfied. If the condition of equation (5) is satisfied, the process proceeds to step S05; otherwise, the process proceeds to step S06.

| GradY | ≦ t ₁ × | gradX |
However, _{t 1} is the value of the tangent at π / 16 (0.1989 if expressed in four decimal places).

  In step S05, it is determined that the direction of the edge gradient is in the range of −π / 16 ≦ θ ≦ π / 16 or 15π / 16 ≦ θ ≦ 17π / 16.

  In step S06, it is determined whether or not the following equation (6) is satisfied. If the condition of equation (6) is satisfied, the process proceeds to step S07; otherwise, the process proceeds to step S08.

| GradY | ≦ t ₂ × | gradX |
However, _{t 2} is the value of the tangent at 3π / 16 (0.6682 if expressed in four decimal places).

  In step S07, it is determined that the direction of the edge gradient is in the range of π / 16 <θ ≦ 3π / 16 or -15π / 16 <θ ≦ -13π / 16.

  In step S08, it is determined whether or not the following equation (7) is satisfied. If the condition of expression (7) is satisfied, the process proceeds to step S09; otherwise, the process proceeds to step S10.

| GradY | ≦ t ₃ × | gradX | Expression (7)
However, _{t 3} is the value of loss tangent at 5π / 16 (1.4966 if expressed in four decimal places).

  In step S09, it is determined that the direction of the edge gradient is in the range of 3π / 16 <θ ≦ 5π / 16 or −13π / 16 <θ ≦ -11π / 16.

  In step S10, it is determined whether or not the following equation (8) is satisfied. If the condition of equation (8) is satisfied, the process proceeds to step S11; otherwise, the process proceeds to step S12.

| GradY | ≦ t ₄ × | gradX | Expression (8)
However, _{t 4} is the value of loss tangent at 7π / 16 (5.0273 if expressed in four decimal places).

  In step S11, it is determined that the direction of the edge gradient is in the range of 5π / 16 <θ ≦ 7π / 16 or -11π / 16 <θ ≦ -9π / 16.

  In step S12, it is determined that the edge gradient direction is in a range of 7π / 16 ≦ θ <9π / 16 or −9π / 16 <θ ≦ −7π / 16. Note that when the processes in steps S02 and S05 to S12 are finished, the edge direction estimation process is finished.

  Step S13 is a case where it is determined in step S12 that the code of gradX and the code of gradY are different. In step S13, it is determined whether or not the condition of equation (5) is satisfied. If the condition of equation (5) is satisfied, the process proceeds to step S14; otherwise, the process proceeds to step S15.

  In step S14, it is determined that the direction of the edge gradient is in the range of −π / 16 ≦ θ ≦ π / 16 or 15π / 16 ≦ θ ≦ 17π / 16.

  In step S15, it is determined whether or not the condition of equation (6) is satisfied. If the condition of equation (6) is satisfied, the process proceeds to step S16; otherwise, the process proceeds to step S17.

  In step S16, it is determined that the direction of the edge gradient is in the range of 13π / 16 ≦ θ <15π / 16 or −3π / 16 ≦ θ <−π / 16.

  In step S17, it is determined whether or not the condition of equation (7) is satisfied. If the condition of equation (7) is satisfied, the process proceeds to step S18; otherwise, the process proceeds to step S19.

  In step S18, it is determined that the edge gradient direction is in the range of 11π / 16 ≦ θ <13π / 16 or −5π / 16 ≦ θ <−3π / 16.

  In step S19, it is determined whether or not the condition of equation (8) is satisfied. If the condition of equation (8) is satisfied, the process proceeds to step S20; otherwise, the process proceeds to step S21.

  In step S20, it is determined that the direction of the edge gradient is in the range of 9π / 16 ≦ θ <11π / 16 or −7π / 16 <θ ≦ −5π / 16.

  In step S21, it is determined that the direction of the edge gradient is in the range of 7π / 16 <θ <9π / 16 or -9π / 16 <θ <-7π / 16. Note that when the processes in steps S14 to S21 are completed for all input pixels, the edge direction estimation process is terminated.

  Here, although the edge gradient direction is calculated using the pixel density of the input pixel, a luminance value generated from an RGB image may be used instead of the pixel density of the input pixel. In this case, for example, the RGB data may be converted into the luminance Y using a matrix operation shown in the following equation (9).

Y = 0.29891 × R + 0.58661 × G + 1.14848 × B (9)
Further, instead of using the luminance value, the first-order derivatives gradX _R and gradY _{R of R} , the first-order derivatives gradY _G and gradY _G of B, gradY _B and gradY _B of _B are obtained, and the following equation 2 is obtained. The first derivative of the channel with the maximum gradient calculated in step 1 may be set as a representative value, and gradX and gradY, which are representative values, may be set as the first-order differential in the horizontal direction and the first-order differential in the vertical direction.

  The interpolation plane setting unit 73 is configured to determine the edge of the interpolation pixel based on the frame-unit image data input from the edge direction estimation processing unit 71 and the direction of the edge gradient in each input pixel calculated by the edge direction estimation processing unit 71. An interpolation plane (interpolation region) that is a region along the direction (interpolation pixel edge direction) and that surrounds the interpolation pixel is set. Here, the edge direction is perpendicular to the direction of the edge gradient. For this reason, the interpolation plane setting unit 73 identifies the direction of the edge gradient in the interpolation pixel from the direction of the edge gradient of the input pixel existing in the vicinity of the interpolation pixel, and sets the direction perpendicular to the direction of the identified edge gradient to the interpolation pixel. The edge direction may be used. Then, the interpolation plane setting unit 73 sets the interpolation plane along the interpolation pixel edge direction that is perpendicular to the edge gradient direction of the interpolation pixel. The interpolation plane is set in a block including a quadrangle formed by 4 × 4 input pixels located in the vicinity of the interpolation pixel to be interpolated by the enlargement / reduction ratio of the video data according to the number of pixels of the display unit 13. And Specifically, the interpolation pixel is included in a quadrangle formed by 4 × 4 input pixels located in the vicinity thereof.

  The interpolation plane is an area determined based on the block size, the interpolation pixel edge direction, and the number of pixels in the area where the interpolation plane is set. Specifically, the interpolation plane has a size that fits in the block, and two sides parallel to the first axis along the interpolation pixel edge direction and a second direction different from the first axis. A parallelogram having two sides parallel to the axis. The parallelogram interpolation plane includes 4 × 4 pixels having the same number of pixels as the pixels in the block, which are lattice points arranged along the first axis and the second axis. In some cases, the plurality of pixels included in the interpolation plane include a provisional pixel having no pixel value in addition to the input pixel. The provisional pixel will be described later. The interpolation plane is set so that the interpolation pixel is included in a region surrounded by 2 × 2 pixels at the center of 4 × 4 pixels arranged along the first axis and the second axis.

  As the direction of the edge gradient in the interpolation pixel, the direction of the edge gradient calculated for the input pixel closest to the interpolation pixel may be used, or calculated for the input pixel located in the vicinity of the interpolation pixel. A representative value of the direction of the edge gradient may be used. In this case, the representative value may be, for example, an average value of a plurality of edge gradient directions calculated by the edge direction estimation processing unit 71 or a mode value among a plurality of edge gradient directions.

  FIG. 6 is a diagram illustrating an example of an interpolation plane set in a block including an interpolation pixel. As shown in FIG. 6, a circle 81, a circle 83, and a circle 85 indicate the positions of the interpolation pixel, the input pixel, and the temporary pixel, respectively. The interpolation pixel is a pixel to be subjected to interpolation processing. The input pixel is a pixel having a pixel value in the input image data. The temporary pixel is a pixel that does not have a pixel value in the input image data, and is a temporary pixel that compensates for each of the plurality of input pixels included in the interpolation plane. The temporary pixel is calculated by a temporary pixel value calculation processing unit 75 described later. The block size includes 4 × 4 input pixels.

  FIG. 6A is a first diagram illustrating an example of an interpolation plane set in a block including an interpolation pixel. Here, the direction θ of the edge gradient in the interpolation pixel is almost horizontal, and is included in the angle range of −π / 16 ≦ θ ≦ π / 16 or 15π / 16 ≦ θ ≦ 17π / 16. That is, the interpolation pixel edge direction θ ′, which is a direction perpendicular to the direction of the edge gradient in the interpolation pixel, is in an angle range of 7π / 16 ≦ θ ′ ≦ 9π / 16 or −9π / 16 ≦ θ ′ ≦ −7π / 16. included. As shown in FIG. 6A, the interpolation plane defined in this angular range is a square interpolation plane, and is set for 4 × 4 pixels located in the vicinity of the interpolation pixel. Here, since the direction of the edge gradient is almost horizontal, all 4 × 4 pixels located in the vicinity of the interpolation pixel are input pixels. For this reason, even if the conventionally used interpolation processing is performed, the interpolation processing can be performed without generating shaggy.

  FIG. 6B is a second diagram illustrating an example of an interpolation plane set in a block including an interpolation pixel. Here, the direction θ of the edge gradient in the interpolation pixel is an oblique direction, and specifically, is inclined about 26.5 degrees with respect to the horizontal direction. This inclination is included in the angle range of π / 16 <θ ≦ 3π / 16 or −15π / 16 <θ ≦ −13π / 16. That is, the interpolation pixel edge direction θ ′, which is a direction perpendicular to the direction of the edge gradient in the interpolation pixel, falls within an angle range of 9π / 16 <θ ′ ≦ 11π / 16 or −7π / 16 <θ ′ ≦ −5π / 16. included. The interpolation plane defined in this angle range is a parallelogram interpolation plane as shown in FIG. The interpolation plane is set along a direction in which two parallel long sides are perpendicular to the direction of the edge gradient (that is, the interpolation pixel edge direction). In other words, the interpolation plane is set along two parallel long sides. For this reason, the plurality of pixels included in the interpolation plane are 4 × 4 pixels including the input pixel and the temporary pixel.

  FIG. 6C is a third diagram illustrating an example of an interpolation plane set in a block including an interpolation pixel. Here, the direction θ of the edge gradient in the interpolation pixel is an oblique direction, specifically, it is inclined by about 45 degrees with respect to the horizontal direction. This inclination is included in an angle range of 3π / 16 <θ ≦ 5π / 16 or −13π / 16 <θ ≦ -11π / 16. That is, the interpolation pixel edge direction θ ′, which is a direction perpendicular to the direction of the edge gradient in the interpolation pixel, falls within an angle range of 11π / 16 <θ ′ ≦ 13π / 16 or −5π / 16 <θ ′ ≦ −3π / 16. included. The interpolation plane defined in this angular range is a square interpolation plane as shown in FIG. Further, the interpolation plane is set along a direction in which the two sides are perpendicular to the direction of the edge gradient (that is, the interpolation pixel edge direction). In other words, the two sides of the interpolation plane are set along the interpolation pixel edge direction. For this reason, the plurality of pixels included in the interpolation plane are 4 × 4 pixels including the input pixel and the temporary pixel.

  FIG. 6D is a fourth diagram illustrating an example of an interpolation plane set in a block including an interpolation pixel. Here, the direction θ of the edge gradient in the interpolation pixel is an oblique direction, specifically, it is inclined by about 63.5 degrees with respect to the horizontal direction. This inclination is included in the angle range of 5π / 16 <θ ≦ 7π / 16 or −11π <θ ≦ −9π / 16. That is, the interpolation pixel edge direction θ ′ that is a direction perpendicular to the edge gradient direction in the interpolation pixel is included in the angle range of 13π / 16 <θ ′ ≦ 15π / 16 or −3π / 16 <θ ′ ≦ −π / 16. It is. The interpolation plane defined in this angle range is a parallelogram interpolation plane as shown in FIG. In addition, the interpolation plane is set along a direction in which two parallel long sides are perpendicular to the direction of the edge gradient (that is, the interpolation pixel edge direction). In other words, the two long sides of the interpolation plane are set along the interpolation pixel edge direction. For this reason, the plurality of pixels included in the interpolation plane are 4 × 4 pixels including the input pixel and the temporary pixel.

  FIG. 6E is a fifth diagram illustrating an example of the interpolation plane set in the block including the interpolation pixel. Here, the direction θ of the edge gradient is almost vertical and is included in the angle range of 7π / 16 <θ <9π / 16 or −9π / 16 <θ <−7π / 16. That is, the interpolation pixel edge direction θ ′ that is a direction perpendicular to the direction of the edge gradient in the interpolation pixel is included in an angle range of −π / 16 <θ ′ <π / 16 or 15π / 16 <θ ′ <17π / 16. It is. The interpolation plane defined in this angle range is the same as the interpolation plane shown in FIG. 6A, as shown in FIG. 6E, and for the 4 × 4 pixels located in the vicinity of the interpolation pixel. Is set. Here, since the direction of the edge gradient is almost vertical, all 4 × 4 pixels located in the vicinity of the interpolation pixel are input pixels. For this reason, even if the conventionally used interpolation processing is performed, the interpolation processing can be performed without generating shaggy.

  FIG. 6F is a sixth diagram illustrating an example of the interpolation plane set in the block including the interpolation pixel. Here, the direction θ of the edge gradient is an oblique direction, specifically, it is inclined by about 116.5 degrees with respect to the horizontal direction. This inclination is included in the angle range of 9π / 16 ≦ θ <11π / 16 or −7π / 16 ≦ θ <−5π / 16. That is, the interpolation pixel edge direction θ ′, which is a direction perpendicular to the direction of the edge gradient in the interpolation pixel, falls within an angle range of π / 16 ≦ θ ′ <3π / 16 or −15π / 16 ≦ θ ′ <− 13π / 16. included. The interpolation plane defined in this angle range is a parallelogram interpolation plane as shown in FIG. In addition, the interpolation plane is set along a direction in which two parallel long sides are perpendicular to the direction of the edge gradient (that is, the interpolation pixel edge direction). In other words, the two parallel long sides of the interpolation plane are set along the interpolation pixel edge direction. For this reason, the plurality of pixels included in the interpolation plane are 4 × 4 pixels including the input pixel and the temporary pixel.

  FIG. 6G is a seventh diagram illustrating an example of an interpolation plane set in a block including an interpolation pixel. Here, the direction θ of the edge gradient is an oblique direction, specifically, it is inclined 135 degrees with respect to the horizontal direction. This inclination is included in an angle range of 11π / 16 ≦ θ <13π / 16 or −5π / 16 ≦ θ <−3π / 16. That is, the interpolation pixel edge direction θ ′, which is a direction perpendicular to the direction of the edge gradient in the interpolation pixel, falls within an angle range of 3π / 16 ≦ θ ′ <5π / 16 or −11π / 16 ≦ θ ′ <− 11π / 16. included. The interpolation plane defined in this angle range is the same as the interpolation plane shown in FIG. 6C as shown in FIG. 6G, and the interpolation plane has two sides along the interpolation pixel edge direction. Is set.

  FIG. 6H is a seventh diagram illustrating an example of the interpolation plane set in the block including the interpolation pixel. Here, the direction θ of the edge gradient is an oblique direction, specifically, it is inclined about 153.4 degrees with respect to the horizontal direction. This inclination is included in an angle range of 13π / 16 ≦ θ <15π / 16 or −3π / 16 ≦ θ <−π / 16. That is, the interpolation pixel edge direction θ ′ that is a direction perpendicular to the direction of the edge gradient in the interpolation pixel is included in an angle range of 5π / 16 ≦ θ ′ <7π / 16 or −11π ≦ θ ′ <− 9π / 16. . The interpolation plane defined in this angular range is a parallelogram interpolation plane as shown in FIG. In addition, the interpolation plane is set along a direction in which two parallel long sides are perpendicular to the direction of the edge gradient (that is, the interpolation pixel edge direction). In other words, the two parallel long sides of the interpolation plane are set along the interpolation pixel edge direction. For this reason, the plurality of pixels included in the interpolation plane are 4 × 4 pixels including the input pixel and the temporary pixel.

  Here, the relationship between the interpolation pixel edge direction and the edge gradient will be specifically described. FIG. 31 is a diagram illustrating the relationship between the interpolation pixel edge direction and the edge gradient. Here, the relationship between the interpolation pixel edge direction and edge gradient corresponding to FIGS. 6B and 6D will be described. FIG. 31A is a diagram illustrating the relationship between the interpolation pixel edge direction and the edge gradient corresponding to FIG. FIG. 31 (b) is a diagram showing the relationship between the interpolation pixel edge direction and the edge gradient corresponding to FIG. 6 (d). As shown in FIGS. 31A and 31B, an arrow indicates an edge gradient, and a thick line perpendicular to the arrow indicates an edge line segment (that is, an interpolation pixel edge direction). The direction θ of the edge gradient indicates an angle with respect to the horizontal direction. The longest side of the interpolation plane shown in FIG. 6B is set along a thick line perpendicular to the arrow shown in FIG. 31A, and the long side of the interpolation plane shown in FIG. It is set along a thick line perpendicular to the arrow shown in 31 (b). Thereby, an interpolation plane is set along the interpolation pixel edge direction.

  As described above, the interpolation pixel edge direction can be specified by estimating the direction of the edge gradient in the interpolation pixel, so that the interpolation plane can be set along the interpolation pixel edge direction. As a result, the jaggy can be suppressed by forming the interpolation pixel at the position where the interpolation is required due to the enlargement of the image data. Here, the interpolation plane can be set using only 4 × 4 input pixels located in the vicinity of the interpolation pixel. However, since the distance between the pixels becomes very long, the correlation with the interpolation pixel becomes low.

  Therefore, here, a temporary pixel is specified at a position where no input pixel exists, and an interpolation plane of 4 × 4 pixels including the temporary pixel is set. As a result, the distance between the pixels can be kept moderate as compared to the case of forming a 4 × 4 pixel interpolation plane using only the input pixels. Therefore, an interpolation process using a pixel having a high correlation with the interpolation pixel is possible. In addition, since the interpolation plane along the interpolation pixel edge direction is set in the block including a plurality of pixels located in the vicinity of the interpolation pixel, the interpolation process is performed compared to the case where the interpolation plane is set regardless of the edge direction. The jaggy which arises can be controlled.

  When the direction of the edge gradient is indefinite, an interpolation plane corresponding to FIG. 6A or FIG. 6C may be set uniquely, or an input pixel located in the vicinity of the interpolation pixel may be set. An interpolation plane corresponding to any one of FIGS. 6A to 6D may be set with reference to the direction of the edge gradient.

  The temporary pixel value calculation processing unit 75 calculates a pixel value (a value indicating a pixel density) of the temporary pixel specified in the interpolation plane set by the interpolation plane setting unit 73 based on an input pixel located in the vicinity thereof. The temporary pixel calculation process is executed. Specifically, the provisional pixel calculation process is a process of calculating the pixel value of the provisional pixel according to a calculation method defined for each direction of the edge gradient estimated for the input pixel located in the vicinity of the provisional pixel.

  That is, the provisional pixel value calculation processing unit 75 specifies a provisional pixel edge direction indicating the edge direction of the provisional pixel from the direction of the edge gradient of the input pixel existing in the vicinity of the provisional pixel. Then, the temporary pixel value calculation processing unit 75 includes a temporary pixel, sets a temporary pixel value calculation area along the temporary pixel edge direction, and uses the value of the input pixel included in the temporary pixel value calculation area. The pixel value of the pixel is calculated. More specifically, the temporary pixel value calculation processing unit 75 calculates the pixel value of the temporary pixel using the value of the input pixel adjacent in the temporary pixel edge direction or the direction within the predetermined angle range from the temporary pixel edge direction. .

  Here, the input pixel located in the vicinity of the temporary pixel may be, for example, two input pixels located in the vicinity of the temporary pixel (in this case, an area including the two input pixels is used for calculating the temporary pixel). The input pixels may be four input pixels located in the vicinity of the temporary pixel (in this case, the region including the four input pixels is the temporary pixel calculation region). When the pixel located near the temporary pixel is two input pixels or four input pixels located near the temporary pixel, the pixel value of the temporary pixel is obtained by multiplying the pixel value of each of the plurality of input pixels by a weighting factor. It is the added value. The weighting factor may be a predetermined value or a value corresponding to the distance between pixels. Hereinafter, the method for calculating the pixel value of the temporary pixel is referred to as a temporary pixel value calculation method.

  FIG. 7 is a first diagram illustrating an example of a provisional pixel value calculation method. Here, the provisional pixel value calculation method for provisional pixels included in the interpolation plane shown in FIGS. 6B and 6H is shown for each pattern. Here, the edge gradient of the input pixel located near the temporary pixel may be different from the edge gradient of the interpolation plane. For this reason, it is preferable to calculate the temporary pixel according to a pattern according to the direction of the edge gradient of the input pixel located in the vicinity of the temporary pixel. A case where calculation is performed according to a corresponding pattern will be described as an example. Here, the direction of the edge gradient (the direction of the edge gradient in the temporary pixel) θ used when calculating the temporary pixel is a representative value (for example, the mode value) of the edge gradient direction θ in the input pixels in the vicinity of the temporary pixel. Or average value). Note that the provisional pixels for which the provisional pixel calculation process is executed here are provisional pixels located between the input pixels arranged in the horizontal direction.

  FIG. 7A is a diagram illustrating a temporary pixel value calculation method for the first pattern. In the first pattern, the direction of the edge gradient in the temporary pixel estimated in the input pixel located in the vicinity of the temporary pixel is π / 16 <θ ≦ 3π / 16 or −15π / 16 <θ ≦ −13π / 16. It is a provisional pixel value calculation method when included in the angle range. In this case, the temporary pixel edge direction θ ′ is included in an angle range of 9π / 16 <θ ′ ≦ 11π / 16 or −7π / 16 <θ ′ ≦ −5π / 16. Therefore, as shown in FIG. 7A, in the first pattern, the input pixels 1 and 2 located in the vicinity of the temporary pixel and adjacent to the temporary pixel in the temporary pixel edge direction are It is set as a pixel for calculating a pixel value. That is, a region (indicated by a dotted line in the figure) including the input pixels 1 and 2 adjacent along the temporary pixel edge direction is set as the temporary pixel calculation region. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 and 2. Specifically, the pixel value of the input pixel 1 and the pixel value of the input pixel 2 are each multiplied by a weighting factor, and the added value is calculated as the pixel value of the temporary pixel. Here, the weighting factor is set to 0.5 for the input pixel 1 and the input pixel 2.

  FIG. 7B is a diagram illustrating a temporary pixel value calculation method of the second pattern. In the second pattern, the direction of the edge gradient at the temporary pixel estimated at the input pixel located in the vicinity of the temporary pixel is 3π / 16 <θ ≦ 5π / 16 or −13π / 16 <θ ≦ −11π / 16. It is a provisional pixel value calculation method when included in the angle range. In this case, the temporary pixel edge direction θ ′ is included in an angle range of 11π / 16 <θ ′ ≦ 13π / 16 or −5π / 16 <θ ′ ≦ −3π / 16. Therefore, as shown in FIG. 7B, in the second pattern, the input pixels 1 to 4 that are located in the vicinity of the temporary pixel and are adjacent in the direction within 45 ° from the temporary pixel edge direction are temporary pixels. Is set as a pixel for calculating the pixel value. That is, the dotted line area including the input pixels 1 to 4 is set as the temporary pixel calculation area. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 to 4. Specifically, each of the pixel values of the input pixels 1 to 4 is multiplied by a weighting coefficient, and the added value is calculated as the pixel value of the temporary pixel. Note that the weighting factor may be set to 0.25 for the input pixels 1 to 4, and the distance between the temporary pixel and the input pixels 2 and 4 is closer than the distance between the temporary pixel and the input pixels 1 and 3. In consideration, the weighting factor may be set to 0.3 for the input pixels 2 and 4 and the weighting factor may be set to 0.2 for the input pixels 1 and 3. Whether the weighting factors set for the input pixels 1 to 4 are all set to the same value or set to a value corresponding to the distance from the temporary pixel is verified using an image and a preferred method is selected. That's fine. Specifically, a test image for evaluating the performance of resolution generated by a method in which all the weighting factors set for the input pixels 1 to 4 are set to the same value and the weighting factor set for the input pixels 1 to 4 Among the test images for evaluating the performance of the resolution generated by the method of setting to a value corresponding to the distance from the tentative pixel, a method having a preferable image quality is selected.

  FIG. 7C is a diagram illustrating a provisional pixel value calculation method of the third pattern. In the third pattern, the direction of the edge gradient in the temporary pixel estimated in the input pixel located in the vicinity of the temporary pixel is 5π / 16 <θ ≦ 7π / 16 or −11π / 16 <θ ≦ −9π / 16. This is a provisional pixel calculation method when included. In this case, the temporary pixel edge direction θ ′ is included in an angle range of 13π / 16 <θ ′ ≦ 15π / 16 or −3π / 16 <θ ′ ≦ −π / 16. Therefore, as shown in FIG. 7C, in the third pattern, the input pixels 1 to 4 that are located in the vicinity of the temporary pixel and are adjacent in the direction within 45 ° from the temporary pixel edge direction are temporary pixels. Is set as a pixel for calculating the pixel value. That is, the dotted line area including the input pixels 1 to 4 is set as the temporary pixel calculation area. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 to 4. Specifically, each of the pixel values of the input pixels 1 to 4 is multiplied by a weighting coefficient, and the added value is calculated as the pixel value of the temporary pixel. Note that the weighting factor may be set to 0.25 for the input pixels 1 to 4, and the distance between the temporary pixel and the input pixels 1 and 4 is closer than the distance between the temporary pixel and the input pixels 2 and 3. In consideration, the weighting factor may be set to 0.1 for the input pixels 2 and 4 and the weighting factor may be set to 0.4 for the input pixels 1 and 3.

  FIG. 7D is a diagram illustrating a provisional pixel value calculation method of the fourth pattern. In the fourth pattern, the direction of the edge gradient in the temporary pixel estimated in the input pixel located in the vicinity of the temporary pixel is 9π / 16 ≦ θ <11π / 16 or −7π / 16 ≦ θ <−5π / 16. It is a provisional pixel calculation method when included in the angle range. In this case, the provisional pixel edge direction θ ′ is included in an angle range of −π / 16 ≦ θ ′ <π / 16 or 15π / 16 ≦ θ ′ <17π / 16. Therefore, as shown in FIG. 7D, in the fourth pattern, the input pixels 1 to 4 that are located in the vicinity of the temporary pixel and are adjacent in the direction within 45 ° from the temporary pixel edge direction are temporary pixels. Is set as a pixel for calculating the pixel value. That is, the dotted line area including the input pixels 1 to 4 is set as the temporary pixel calculation area. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 to 4. Specifically, each of the pixel values of the input pixels 1 to 4 is multiplied by a weighting coefficient, and the added value is calculated as the pixel value of the temporary pixel. Thereby, a temporary pixel is calculated. It should be noted that the weighting factor may be set to 0.25 for the input pixels 1 to 4 and that the distance between the temporary pixel and the input pixels 1 and 3 is closer than the distance between the temporary pixel and the input pixels 2 and 4. In consideration, the weighting factor may be set to 0.4 for the input pixels 1 and 3 and the weighting factor may be set to 0.1 for the input pixels 2 and 3.

  FIG. 7E is a diagram illustrating a provisional pixel value calculation method of the fifth pattern. In the fifth pattern, the direction of the edge gradient in the temporary pixel estimated in the input pixel located in the vicinity of the temporary pixel is 11π / 16 ≦ θ <13π / 16 or −5π / 16 ≦ θ <−3π / 16. It is a provisional pixel value calculation method when it is included in the angle range. In this case, the provisional pixel edge direction θ ′ is included in the angle range of π / 16 ≦ θ ′ <3π / 16 or −15π / 16 ≦ θ ′ <− 13π / 16. Therefore, as shown in FIG. 7E, in the fifth pattern, the input pixels 1 to 4 that are located in the vicinity of the temporary pixel and are adjacent in the direction within 45 ° from the temporary pixel edge direction are temporary pixels. Is set as a pixel for calculating the pixel value. That is, the dotted line area including the input pixels 1 to 4 is set as the temporary pixel calculation area. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 to 4. Specifically, each of the pixel values of the input pixels 1 to 4 is multiplied by a weighting coefficient, and the added value is calculated as the pixel value of the temporary pixel. It should be noted that the weighting factor may be set to 0.25 for the input pixels 1 to 4 and that the distance between the temporary pixel and the input pixels 1 and 3 is closer than the distance between the temporary pixel and the input pixels 2 and 4. In consideration, the weighting factor may be set to 0.3 for the input pixels 1 and 3 and the weighting factor may be set to 0.2 for the input pixels 2 and 4.

  FIG. 7F is a diagram illustrating a provisional pixel value calculation method of the sixth pattern. In the sixth pattern, the direction of the edge gradient in the temporary pixel estimated in the pixel located in the vicinity of the temporary pixel is 13π / 16 ≦ θ <15π / 16 or −3π / 16 ≦ θ <−π / 16. It is a provisional pixel calculation method when included in the angle range. In this case, the temporary pixel edge direction θ ′ is included in an angle range of 3π / 16 ≦ θ ′ <5π / 16 or −11π / 16 ≦ θ ′ <− 11π / 16. Therefore, as shown in FIG. 7F, in the sixth pattern, the input pixels 1 and 2 that are located in the vicinity of the temporary pixel and are adjacent to the temporary pixel in the temporary pixel edge direction are It is set as a pixel for calculating a pixel value. That is, the dotted line area including the input pixels 1 and 2 is set as the temporary pixel calculation area. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 and 2. Specifically, the pixel value of the input pixel 1 and the pixel value of the input pixel 2 are each multiplied by a weighting factor, and the added value is calculated as the pixel value of the temporary pixel. Here, the weighting factor is set to 0.5 for the input pixel 1 and the input pixel 2.

  FIG. 7G is a diagram illustrating a provisional pixel value calculation method of the seventh pattern. The seventh pattern is a tentative pixel calculation method when the direction of the edge gradient in the tentative pixel estimated at the input pixel located in the vicinity of the tentative pixel is indefinite. As shown in FIG. 7G, in the seventh pattern, the input pixels 1 and 2 adjacent in the horizontal direction of the temporary pixel are set as pixels for calculating the pixel value of the temporary pixel. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 and 2. Specifically, the pixel value of the input pixel 1 and the pixel value of the input pixel 2 are each multiplied by a weighting factor of 0.5, and the added value is calculated as the pixel value of the temporary pixel. Note that the pixel value of the temporary pixel may be calculated with reference to the edge direction of the input pixel located in the vicinity of the temporary pixel.

  FIG. 8 is a second diagram illustrating an example of a provisional pixel calculation method. Here, the temporary pixel calculation method of the pixel value of the temporary pixel included in the interpolation plane shown in FIGS. 6C and 6G is shown for each pattern. Here, the edge gradient of the input pixel located near the temporary pixel may be different from the edge gradient of the interpolation plane. For this reason, the temporary pixel is preferably calculated according to a pattern corresponding to the direction of the edge gradient of the input pixel located in the vicinity of the temporary pixel. Note that the provisional pixels for which the provisional pixel calculation process is executed here are provisional pixels located between input pixels arranged in an oblique direction.

  FIG. 8A is a diagram illustrating a temporary pixel value calculation method of the first pattern. In the first pattern, the direction of the edge gradient in the temporary pixel estimated in the input pixel located in the vicinity of the temporary pixel is π / 16 <θ ≦ 3π / 16 or −15π / 16 <θ ≦ −13π / 16. It is a provisional pixel value calculation method when included in the angle range. In this case, the temporary pixel edge direction θ ′ is included in an angle range of 9π / 16 <θ ′ ≦ 11π / 16 or −7π / 16 <θ ′ ≦ −5π / 16. Therefore, as shown in FIG. 8A, in the first pattern, the input pixels 1 to 4 that are located in the vicinity of the temporary pixel and are adjacent in the direction within 45 ° from the temporary pixel edge direction are temporary pixels. Is set as a pixel for calculating the pixel value. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 to 4. Specifically, each of the pixel values of the input pixels 1 to 4 is multiplied by a weighting coefficient, and the added value is calculated as the pixel value of the temporary pixel. Note that the weighting factor may be set to 0.25 for the input pixels 1 to 4, and the distance between the temporary pixel and the input pixels 1 and 4 is closer than the distance between the temporary pixel and the input pixels 2 and 3. In consideration, the weighting factor may be set to 0.3 for the input pixels 1 and 4 and the weighting factor may be set to 0.2 for the input pixels 2 and 3. Whether the weighting factors set for the input pixels 1 to 4 are all set to the same value or set to a value corresponding to the distance from the temporary pixel is verified using an image and a preferred method is selected. That's fine.

  FIG. 8B is a diagram illustrating a temporary pixel value calculation method of the second pattern. In the second pattern, the direction of the edge gradient at the temporary pixel estimated at the input pixel located in the vicinity of the temporary pixel is 3π / 16 <θ ≦ 5π / 16 or −13π / 16 <θ ≦ −11π / 16. It is a provisional pixel value calculation method when included in the angle range. In this case, the temporary pixel edge direction θ ′ is included in an angle range of 11π / 16 <θ ′ ≦ 13π / 16 or −5π / 16 <θ ′ ≦ −3π / 16. Therefore, as shown in FIG. 8B, in the second pattern, the input pixels 1 and 2 that are located in the vicinity of the temporary pixel and are adjacent to the temporary pixel in the temporary pixel edge direction are It is set as a pixel for calculating a pixel value. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 and 2. Specifically, the pixel value of the input pixel 1 and the pixel value of the input pixel 2 are each multiplied by a weighting factor, and the added value is calculated as the pixel value of the temporary pixel. Here, the weighting factor is set to 0.5 for the input pixel 1 and the input pixel 2.

  FIG. 8C is a diagram illustrating a temporary pixel value calculation method of the third pattern. In the third pattern, the direction of the edge gradient at the temporary pixel estimated at the input pixel located in the vicinity of the temporary pixel is 5π / 16 <θ ≦ 7π / 16 or −11π / 16 <θ ≦ −9π / 16. It is a provisional pixel value calculation method when included in the angle range. In this case, the temporary pixel edge direction θ ′ is included in an angle range of 13π / 16 <θ ′ ≦ 15π / 16 or −3π / 16 <θ ′ ≦ −π / 16. Therefore, as shown in FIG. 8C, in the third pattern, the input pixels 1 to 4 that are located in the vicinity of the temporary pixel and are adjacent in the direction within 45 ° from the temporary pixel edge direction are temporary pixels. Is set as a pixel for calculating the pixel value. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 to 4. Specifically, each of the pixel values of the input pixels 1 to 4 is multiplied by a weighting coefficient, and the added value is calculated as the pixel value of the temporary pixel. Note that the weighting coefficient may be set to 0.25 for the input pixels 1 to 4, and that the distance between the temporary pixel and the input pixels 2 and 3 is closer than the distance between the temporary pixel and the input pixels 1 and 4. In consideration, the weighting factor may be set to 0.3 for the input pixels 2 and 3 and the weighting factor may be set to 0.2 for the input pixels 1 and 4.

  FIG. 8D is a diagram showing a fourth pattern provisional pixel value calculation method. In the fourth pattern, the direction of the edge gradient in the temporary pixel estimated in the input pixel located in the vicinity of the temporary pixel is 9π / 16 ≦ θ <11π / 16 or −7π / 16 ≦ θ <−5π / 16. It is a provisional pixel value calculation method when included in the angle range. In this case, the provisional pixel edge direction θ ′ is included in an angle range of −π / 16 ≦ θ ′ <π / 16 or 15π / 16 ≦ θ ′ <17π / 16. Therefore, as shown in FIG. 8D, in the fourth pattern, the input pixels 1 to 4 that are located in the vicinity of the temporary pixel and are adjacent in the direction within 45 ° from the temporary pixel edge direction are temporary pixels. Is set as a pixel for calculating the pixel value. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 to 4. Specifically, the point where the line connecting the input pixel 1 and the input pixel 3 and the line connecting the input pixel 2 and the input pixel 4 intersect is calculated as the position of the temporary pixel, and the pixel values of the input pixels 1 to 4 are calculated. Each is multiplied by a weighting factor, and the added value is calculated as the pixel value of the temporary pixel. Note that the weighting factor may be set to 0.25 for the input pixels 1 to 4, and the distance between the temporary pixel and the input pixels 1 and 4 is closer than the distance between the temporary pixel and the input pixels 2 and 3. In consideration, the weighting factor may be set to 0.3 for the input pixels 1 and 4 and the weighting factor may be set to 0.2 for the input pixels 2 and 3.

  FIG. 8E is a diagram illustrating a provisional pixel value calculation method of the fifth pattern. In the fifth pattern, the direction of the edge gradient in the temporary pixel estimated at the input pixel located in the vicinity of the temporary pixel is 11π / 16 ≦ θ <13π / 16 or −5π / 16 ≦ θ <−3π / 16. It is a provisional pixel value calculation method when included in the angle range. In this case, the provisional pixel edge direction θ ′ is included in the angle range of π / 16 ≦ θ ′ <3π / 16 or −15π / 16 ≦ θ ′ <− 13π / 16. Therefore, as shown in FIG. 8E, in the fifth pattern, the input pixels 1 and 2 that are located in the vicinity of the temporary pixel and are adjacent to the temporary pixel in the temporary pixel edge direction are It is set as a pixel for calculating a pixel value. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 and 2. Specifically, the pixel value of the input pixel 1 and the pixel value of the input pixel 2 are each multiplied by a weighting factor, and the added value is calculated as the pixel value of the temporary pixel. Here, the weighting factor is set to 0.5 for the input pixel 1 and the input pixel 2.

  FIG. 8F is a diagram illustrating a provisional pixel value calculation method of the sixth pattern. In the sixth pattern, the direction of the edge gradient at the temporary pixel estimated at the input pixel located in the vicinity of the temporary pixel is 13π / 16 ≦ θ <15π / 16 or −3π / 16 ≦ θ <−π / 16. It is a provisional pixel value calculation method when included in the angle range. In this case, the temporary pixel edge direction θ ′ is included in an angle range of 3π / 16 ≦ θ ′ <5π / 16 or −11π / 16 ≦ θ ′ <− 11π / 16. Therefore, as shown in FIG. 8F, in the sixth pattern, the input pixels 1 to 4 that are located in the vicinity of the temporary pixel and are adjacent in the direction within 45 ° from the temporary pixel edge direction are temporary pixels. Is set as a pixel for calculating the pixel value. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 to 4. Specifically, the point where the line connecting the input pixel 1 and the input pixel 3 and the line connecting the input pixel 2 and the input pixel 4 intersect is calculated as the position of the temporary pixel, and the pixel values of the input pixels 1 to 4 are calculated. Each is multiplied by a weighting factor, and the added value is calculated as the pixel value of the temporary pixel. Note that the weighting factor may be set to 0.25 for the input pixels 1 to 4, and the distance between the temporary pixel and the input pixels 1 and 4 is closer than the distance between the temporary pixel and the input pixels 2 and 3. In consideration, the weighting factor may be set to 0.3 for the input pixels 1 and 4 and the weighting factor may be set to 0.2 for the input pixels 2 and 3.

  FIG. 8G is a diagram illustrating a provisional pixel value calculation method of the seventh pattern. The seventh pattern is a tentative pixel calculation method when the direction of the edge gradient in the tentative pixel is indeterminate estimated in the input pixel located in the vicinity of the tentative pixel. As shown in FIG. 8G, in the seventh pattern, the input pixels 1 to 4 adjacent in the oblique direction of the temporary pixel are set as pixels for calculating the pixel value of the temporary pixel. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 to 4. Specifically, each of the pixel values of the input pixels 1 to 4 is multiplied by a weighting coefficient, and the added value is calculated as the pixel value of the temporary pixel. Here, the weighting coefficient is set to 0.25 for the input pixels 1 to 4.

  FIG. 9 is a third diagram illustrating an example of a provisional pixel calculation method. Here, the calculation method of the pixel value of the temporary pixel included in the interpolation plane shown in FIGS. 6D and 6F is shown for each pattern. Here, the edge gradient of the input pixel located near the temporary pixel may be different from the edge gradient of the interpolation plane. For this reason, the provisional pixel is preferably calculated according to a pattern corresponding to the direction of the edge gradient. Note that the provisional pixels for which the provisional pixel calculation process is executed here are provisional pixels located between the input pixels arranged in the vertical direction.

  FIG. 9A is a diagram illustrating a temporary pixel value calculation method of the first pattern. In the first pattern, the direction of the edge gradient in the temporary pixel estimated in the input pixel located in the vicinity of the temporary pixel is π / 16 <θ ≦ 3π / 16 or −15π / 16 <θ ≦ −13π / 16. It is a provisional pixel value calculation method when included in the angle range. In this case, the temporary pixel edge direction θ ′ is included in an angle range of 9π / 16 <θ ′ ≦ 11π / 16 or −7π / 16 <θ ′ ≦ −5π / 16. Therefore, as shown in FIG. 9A, in the first pattern, the input pixels 1 to 4 that are located in the vicinity of the temporary pixel and are adjacent in the direction within 45 ° from the temporary pixel edge direction are temporary pixels. Is set as a pixel for calculating the pixel value. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 to 4. Specifically, each of the pixel values of the input pixels 1 to 4 is multiplied by a weighting coefficient, and the added value is calculated as the pixel value of the temporary pixel. Note that the weighting factor may be set to 0.25 for the input pixels 1 to 4, and the distance between the temporary pixel and the input pixels 2 and 4 is closer than the distance between the temporary pixel and the input pixels 1 and 3. In consideration, the weighting factor may be set to 0.4 for the input pixels 2 and 4 and the weighting factor may be set to 0.1 for the input pixels 1 and 3. Whether the weighting factors set for the input pixels 1 to 4 are all set to the same value or set to a value corresponding to the distance from the temporary pixel is verified using an image and a preferred method is selected. That's fine.

  FIG. 9B is a diagram illustrating a temporary pixel value calculation method of the second pattern. In the second pattern, the direction of the edge gradient at the temporary pixel estimated at the input pixel located in the vicinity of the temporary pixel is 3π / 16 <θ ≦ 5π / 16 or −13π / 16 <θ ≦ −11π / 16. It is a provisional pixel value calculation method when included in the angle range. In this case, the temporary pixel edge direction θ ′ is included in an angle range of 11π / 16 <θ ′ ≦ 13π / 16 or −5π / 16 <θ ′ ≦ −3π / 16. Therefore, as shown in FIG. 9B, in the second pattern, the input pixels 1 to 4 that are located in the vicinity of the temporary pixel and are adjacent in the direction within 45 ° from the temporary pixel edge direction are temporary pixels. Is set as a pixel for calculating the pixel value. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 to 4. Specifically, each of the pixel values of the input pixels 1 to 4 is multiplied by a weighting coefficient, and the added value is calculated as the pixel value of the temporary pixel. Note that the weighting factor may be set to 0.25 for the input pixels 1 to 4, and the distance between the temporary pixel and the input pixels 2 and 4 is closer than the distance between the temporary pixel and the input pixels 1 and 3. In consideration, the weighting factor may be set to 0.3 for the input pixels 2 and 4 and the weighting factor may be set to 0.2 for the input pixels 1 and 3.

  FIG. 9C is a diagram illustrating a temporary pixel value calculation method of the third pattern. In the third pattern, the direction of the edge gradient at the temporary pixel estimated at the input pixel located in the vicinity of the temporary pixel is 5π / 16 <θ ≦ 7π / 16 or −11π / 16 <θ ≦ −9π / 16. It is a provisional pixel value calculation method when included in the angle range. In this case, the temporary pixel edge direction θ ′ is included in an angle range of 13π / 16 <θ ′ ≦ 15π / 16 or −3π / 16 <θ ′ ≦ −π / 16. Therefore, as shown in FIG. 9C, in the third pattern, the input pixels 1 and 2 that are located in the vicinity of the temporary pixel and are adjacent to the temporary pixel in the temporary pixel edge direction are It is set as a pixel for calculating a pixel value. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 and 2. Specifically, the pixel value of the input pixel 1 and the pixel value of the input pixel 2 are each multiplied by a weighting factor, and the added value is calculated as the pixel value of the temporary pixel. Here, the weighting factor is set to 0.5 for the input pixel 1 and the input pixel 2.

  FIG. 9D is a diagram showing a fourth pattern provisional pixel value calculation method. In the fourth pattern, the direction of the edge gradient in the temporary pixel estimated in the input pixel located in the vicinity of the temporary pixel is 9π / 16 ≦ θ <11π / 16 or −7π / 16 ≦ θ <−5π / 16. It is a provisional pixel value calculation method when included in the angle range. In this case, the provisional pixel edge direction θ ′ is included in an angle range of −π / 16 ≦ θ ′ <π / 16 or 15π / 16 ≦ θ ′ <17π / 16. Therefore, as shown in FIG. 9D, in the fourth pattern, the input pixels 1 and 2 that are located in the vicinity of the temporary pixel and are adjacent to the temporary pixel in the temporary pixel edge direction are It is set as a pixel for calculating a pixel value. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 and 2. Specifically, the pixel value of the input pixel 1 and the pixel value of the input pixel 2 are each multiplied by a weighting factor, and the added value is calculated as the pixel value of the temporary pixel. Here, the weighting factor is set to 0.5 for the input pixel 1 and the input pixel 2.

  FIG. 9E is a diagram illustrating a provisional pixel value calculation method of the fifth pattern. In the fifth pattern, the direction of the edge gradient in the temporary pixel estimated at the input pixel located in the vicinity of the temporary pixel is 11π / 16 ≦ θ <13π / 16 or −5π / 16 ≦ θ <−3π / 16. It is a provisional pixel value calculation method when included in the angle range. In this case, the provisional pixel edge direction θ ′ is included in the angle range of π / 16 ≦ θ ′ <3π / 16 or −15π / 16 ≦ θ ′ <− 13π / 16. Therefore, as shown in FIG. 9E, in the fifth pattern, the input pixels 1 to 4 that are located in the vicinity of the temporary pixel and are adjacent in the direction within 45 ° from the temporary pixel edge direction are temporary pixels. Is set as a pixel for calculating the pixel value. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 to 4. Specifically, each of the pixel values of the input pixels 1 to 4 is multiplied by a weighting coefficient, and the added value is calculated as the pixel value of the temporary pixel. It should be noted that the weighting factor may be set to 0.25 for the input pixels 1 to 4 and that the distance between the temporary pixel and the input pixels 1 and 3 is closer than the distance between the temporary pixel and the input pixels 2 and 4. In consideration, the weighting factor may be set to 0.3 for the input pixels 1 and 3 and the weighting factor may be set to 0.2 for the input pixels 2 and 4.

  FIG. 9F is a diagram illustrating a provisional pixel value calculation method of the sixth pattern. In the sixth pattern, the direction of the edge gradient at the temporary pixel estimated at the input pixel located in the vicinity of the temporary pixel is 13π / 16 ≦ θ <15π / 16 or −3π / 16 ≦ θ <−π / 16. It is a provisional pixel value calculation method when included in the angle range. In this case, the temporary pixel edge direction θ ′ is included in an angle range of 3π / 16 ≦ θ ′ <5π / 16 or −11π / 16 ≦ θ ′ <− 11π / 16. Therefore, as shown in FIG. 9F, in the sixth pattern, the input pixels 1 to 4 that are located in the vicinity of the temporary pixel and are adjacent in the direction within 45 ° from the temporary pixel edge direction are temporary pixels. Is set as a pixel for calculating the pixel value. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 to 4. Specifically, each of the pixel values of the input pixels 1 to 4 is multiplied by a weighting coefficient, and the added value is calculated as the pixel value of the temporary pixel. It should be noted that the weighting factor may be set to 0.25 for the input pixels 1 to 4 and that the distance between the temporary pixel and the input pixels 1 and 3 is closer than the distance between the temporary pixel and the input pixels 2 and 4. In consideration, the weighting factor may be set to 0.4 for the input pixels 1 and 3 and the weighting factor may be set to 0.1 for the input pixels 2 and 4.

  FIG. 9G is a diagram showing a provisional pixel value calculation method for the seventh pattern. The seventh pattern is a tentative pixel calculation method when the direction of the edge gradient in the tentative pixel is indeterminate estimated in the input pixel located in the vicinity of the tentative pixel. As shown in FIG. 9G, in the seventh pattern, the input pixels 1 and 2 adjacent in the vertical direction of the temporary pixel are set as pixels for calculating the pixel value of the temporary pixel. The pixel value of the temporary pixel is calculated based on the weighting factor set for each of the input pixels 1 and 2. Specifically, the pixel value of the input pixel 1 and the pixel value of the input pixel 2 are each multiplied by a weighting factor, and the added value is calculated as the pixel value of the temporary pixel. Here, the weighting factor is set to 0.5 for the input pixel 1 and the input pixel 2.

  Here, the interpolation plane can be set using only 4 × 4 input pixels located in the vicinity of the interpolation pixel. However, since the distance between the pixels becomes very long, the correlation with the interpolation pixel becomes low. Therefore, as described in FIGS. 7 to 9, since the temporary pixel is set at a position where no input pixel exists on the interpolation plane, the distance between the pixels on the interpolation plane can be kept moderate. Thereby, an interpolation process using a pixel having a high correlation with the interpolation pixel becomes possible.

  Here, the representative value (for example, the mode value or the average value) of the edge gradient direction θ of the input pixels in the vicinity of the temporary pixel is used as the edge gradient direction of the temporary pixel. However, the present invention is not limited to this, and the edge gradient direction θ of the input pixel closest to the temporary pixel may be used. Alternatively, the direction of the edge gradient in the interpolation pixel used when setting the interpolation plane to which the temporary pixel belongs may be set as the direction of the edge gradient in the temporary pixel. In this case, it is not necessary to calculate the direction of the edge gradient in the temporary pixel separately from the direction of the edge gradient in the interpolation pixel, so that the processing speed can be increased.

  The interpolation processing unit 77 performs an interpolation process on the interpolation plane including the temporary pixel whose pixel value is calculated by the temporary pixel value calculation processing unit 75. The interpolation process is a process for calculating the pixel value of the interpolation pixel included in the interpolation plane. The interpolation process is, for example, a Lanczos interpolation process. The Lanczos interpolation process is a process for executing an interpolation operation for calculating a product sum of a weighting coefficient determined based on the interpolation function expressed by Equation 3 and pixels used for interpolation.

  However, the sinc function is an expression represented by the following equation (4), and n is a variable for controlling the properties of the interpolation function. The number of interpolated pixels used in the Lanczos interpolation process changes according to the variable n. d indicates the distance between the interpolation pixel and a pixel (input pixel or temporary pixel) located in the vicinity thereof.

  The interpolation processing unit 77 performs an interpolation calculation corresponding to the direction of the edge gradient estimated by the edge direction estimation processing unit 71, and calculates the pixel value of the interpolation pixel. Here, the interpolation calculation for each direction of the edge gradient will be described.

  In the following, the interpolation plane shown in FIGS. 6A and 6E is referred to as a first interpolation plane, the interpolation plane shown in FIG. 6B is referred to as a second interpolation plane, and FIG. , (G) is called the third interpolation plane, the interpolation plane shown in FIG. 6 (d) is called the fourth interpolation plane, and the interpolation plane shown in FIG. 6 (f) is the fifth interpolation plane. The interpolation plane shown in FIG. 6 (h) is referred to as a sixth interpolation plane. In addition, since the interpolation processing in the present embodiment uses 4 × 4 pixels located in the vicinity of the interpolation pixel, n = 2 in Equation 3. In this case, the weighting coefficient is determined by the distance from the position of the interpolation pixel, and specifically, a distribution diagram as shown in FIG. In the distribution diagram, the horizontal axis indicates the distance from the interpolation pixel, and the vertical axis indicates the weighting coefficient.

  FIG. 10 is a diagram illustrating parameters used for the interpolation calculation together with the first interpolation plane. Here, a coordinate system is set in which the horizontal direction (lateral direction) in the input image data is the X axis and the vertical direction (vertical direction) is the Y axis. The coordinate system here is a coordinate system based on the distance between the input pixel and the adjacent input pixel. In FIG. 10, the horizontal direction component and the vertical direction component of the distance between the interpolated pixel and the upper left pixel in the vicinity thereof are shown as distX and distY, respectively. Here, distX is a value when the distance between input pixels adjacent in the horizontal direction is 1, and distY is a value when the distance between input pixels adjacent in the vertical direction is 1. The interpolation calculation is executed in the horizontal direction according to Expression (11) using distX and distY shown in FIG.

V (i) = w (−1−distX) × P (0, i)
+ W (−distX) × P (1, i)
+ W (1-distX) × P (2, i)
+ W (2-distX) × P (3, i) Equation (11)
However, 0 ≦ i ≦ 3, and P (X, Y) indicates an input pixel.

  Next, using V (i) calculated by Expression (11), interpolation calculation is performed by Expression (12), and the pixel value OUT of the interpolation pixel is calculated.

OUT = w (−1−distY) × V (0)
+ W (−distY) × V (1)
+ W (1-distY) × V (2)
+ W (2-distY) × V (3) Expression (12)
FIG. 11 is a diagram showing parameters used for the interpolation calculation together with the second interpolation plane. Here, a coordinate system is set in which the horizontal direction of the second interpolation plane is the X ′ axis (second axis) and the direction parallel to the interpolation pixel edge direction is the Y ′ axis (first axis). The coordinate system here is a coordinate system based on the distance between the input pixel and the adjacent temporary pixel. In FIG. 11, the horizontal direction (X-axis direction) component and the vertical direction (Y-axis direction) component of the distance between the interpolation pixel and the upper left pixel in the vicinity thereof are shown as distX and distY, respectively. The interpolation calculation is executed in the horizontal direction according to Expression (13) using distX ′ and distY calculated based on distX and distY shown in FIG.

V (i) = w (−1−distX ′) × P (0, i)
+ W (−distX ′) × P (1, i)
+ W (1-distX ′) × P (2, i)
+ W (2-distX ′) × P (3, i) (13)
However, 0 ≦ i ≦ 3, and P (X ′, Y ′) indicates an input pixel or a temporary pixel. Note that distX ′ is calculated by the equation (14).

distX ′ = 2 × distX + distY (14)
“distX ′” indicates a distance between an axis (an axis passing through P (1, 0) and P (1, 3)) in the direction of about 63.4 degrees obliquely from the horizontal direction passing through the interpolation pixel and the reference pixel. Hereinafter, the reference pixel is the upper left pixel near the interpolation pixel.

  Next, using V (i) calculated by Expression (14), interpolation calculation is performed by Expression (15), and the pixel value OUT of the interpolation pixel is calculated.

OUT = w (−1−distY) × V (0)
+ W (−distY) × V (1)
+ W (1-distY) × V (2)
+ W (2-distY) × V (3) Expression (15)
Here, distX is a value when the distance between input pixels adjacent in the horizontal direction is 1, and distY is a value when the distance between input pixels adjacent in the vertical direction is 1. On the other hand, distX ′ is a value when the distance between pixels adjacent to the X′-axis direction (including both input pixels and temporary pixels) on the interpolation plane including the temporary pixels is 1, and distY ′ is This is a value when the distance between adjacent pixels in the Y′-axis direction on the interpolation plane including the temporary pixel is 1. The same applies to FIGS. 12 to 16 below.

FIG. 12 is a diagram showing parameters used for the interpolation calculation together with the third interpolation surface. Here, the direction parallel to the side along the interpolation pixel edge direction of the third interpolation plane is the X ′ axis (first axis), and the direction parallel to the side not along the interpolation pixel edge direction is the Y ′ axis (first axis). A coordinate system with two axes) is set. The coordinate system here is a coordinate system based on the distance between the input pixel and the adjacent temporary pixel.
In FIG. 12, the horizontal direction component and the vertical direction component of the distance between the interpolation pixel and the upper left pixel in the vicinity thereof are shown as distX and distY, respectively. The interpolation calculation is executed in the horizontal direction by Expression (16) using distX ′ and distY calculated based on distX and distY shown in FIG.

V (i) = w (−1−distX ′) × P (0, i)
+ W (−distX ′) × P (1, i)
+ W (1-distX ′) × P (2, i)
+ W (2-distX ′) × P (3, i) (16)
However, 0 ≦ i ≦ 3, and P (X ′, Y ′) indicates an input pixel or a temporary pixel. Note that distX ′ is calculated by the equation (17).

distX ′ = distX−distY (17)
“distX ′” indicates a distance between an axis (an axis passing through P (0,1) and P (3,1)) in an oblique upper right 45 ° direction from the horizontal direction passing through the interpolation pixel and the reference pixel.

  Next, using V (i) calculated by Expression (17), interpolation calculation is performed by Expression (18), and the pixel value OUT of the interpolation pixel is calculated.

OUT = w (−1−distY ′) × V (0)
+ W (−distY ′) × V (1)
+ W (1-distY ′) × V (2)
+ W (2-distY ′) × V (3) Expression (18)
However, distY ′ is calculated by Expression (19) using distX and distY shown in FIG.

distY ′ = distX + distY (19)
distY ′ represents a distance between an axis (an axis passing through P (1, 0) and P (1, 3)) in a 45 ° diagonally lower right direction than the horizontal direction passing through the interpolation pixel and the reference pixel.

  FIG. 13 is a diagram showing parameters used for the interpolation calculation together with the fourth interpolation plane. Here, the direction parallel to the side along the interpolation pixel edge direction of the fourth interpolation plane is the X ′ axis (first axis), and the direction parallel to the side not along the interpolation pixel edge direction is the Y ′ axis (first axis). A coordinate system with two axes) is set. The coordinate system here is a coordinate system based on the distance between the input pixel and the adjacent temporary pixel. In FIG. 13, the horizontal direction component and the vertical direction component of the distance between the interpolation pixel and the upper left pixel in the vicinity thereof are shown as distX and distY, respectively. The interpolation calculation is executed in the horizontal direction by Expression (20) using distX and distY shown in FIG.

V (i) = w (−1−distX) × P (0, i)
+ W (−distX) × P (1, i)
+ W (1-distX) × P (2, i)
+ W (2-distX) × P (3, i) (20)
Next, using V (i) calculated by Expression (20), interpolation calculation is performed by Expression (21), and the pixel value OUT of the interpolation pixel is calculated.

OUT = w (−1−distY ′) × V (0)
+ W (−distY ′) × V (1)
+ W (1-distY ′) × V (2)
+ W (2-distY ′) × V (3) Equation (21)
However, distY ′ is calculated by Expression (22) using distX and distY shown in FIG.

distY ′ = 2 × distY + distX (22)
distY ′ indicates a distance between an axis (an axis passing through P (0,1) and P (3,1)) in an oblique upper right 26.5 degree direction from the horizontal direction passing through the interpolation pixel and the reference pixel.

  FIG. 14 is a diagram illustrating parameters used for the interpolation calculation together with the fifth interpolation plane. Here, the direction parallel to the side along the interpolation edge direction of the fifth interpolation plane is the X ′ axis (first axis), and the direction parallel to the side not along the interpolation pixel edge direction is the Y ′ axis (second axis). A coordinate system is set. The coordinate system here is a coordinate system based on the distance between the input pixel and the adjacent temporary pixel. In FIG. 14, the horizontal direction component and the vertical direction component of the distance between the interpolation pixel and the adjacent upper left pixel are indicated as distX and distY, respectively. The interpolation calculation is executed in the horizontal direction by the equation (23) using distX and distY shown in FIG.

V (i) = w (−1−distX) × P (0, i)
+ W (−distX) × P (1, i)
+ W (1-distX) × P (2, i)
+ W (2-distX) × P (3, i) (23)
However, 0 ≦ i ≦ 3, and P (X ′, Y ′) indicates an input pixel or a temporary pixel.

  Next, using V (i) calculated by Expression (23), interpolation calculation is performed by Expression (24), and the pixel value OUT of the interpolation pixel is calculated.

OUT = w (−1−distY ′) × V (0)
+ W (−distY ′) × V (1)
+ W (1-distY ′) × V (2)
+ W (2-distY ′) × V (3) Expression (24)
However, distY ′ is calculated by Expression (25) using distX and distY shown in FIG.

distY ′ = 2 × distY−distX (Equation 25)
distY ′ represents a distance between an axis (an axis passing through P (0,1) and P (3,1)) in a direction diagonally lower right to −26.5 degrees from the horizontal direction passing through the interpolation pixel and the reference pixel.

  FIG. 15 is a diagram illustrating parameters used for the interpolation calculation together with the sixth interpolation plane. Here, the direction parallel to the side not along the interpolation pixel edge direction of the sixth interpolation plane is the X ′ axis (second axis), and the direction parallel to the side along the interpolation pixel edge direction is the Y ′ axis (first axis). A coordinate system for one axis) is set. The coordinate system here is a coordinate system based on the distance between the input pixel and the adjacent temporary pixel. In FIG. 15, the horizontal direction component and the vertical direction component of the distance between the interpolated pixel and the upper left pixel in the vicinity thereof are shown as distX and distY, respectively. The interpolation calculation is executed in the horizontal direction by Expression (26) using distX ′ and distY calculated based on distX and distY shown in FIG.

V (i) = w (−1−distX ′) × P (0, i)
+ W (−distX ′) × P (1, i)
+ W (1-distX ′) × P (2, i)
+ W (2-distX ′) × P (3, i) Equation (26)
However, 0 ≦ i ≦ 3, and P (X ′, Y ′) indicates an input pixel or a temporary pixel. Here, distX ′ is calculated by the equation (27).

distX ′ = 2 × distX−distY (27)
distX ′ represents a distance between an axis (an axis passing through P (1, 0) and P (1, 3)) in the lower right-63.4 degrees direction from the horizontal direction passing through the interpolation pixel and the reference pixel.

  Next, using V (i) calculated by Expression (26), interpolation calculation is performed by Expression (28), and the pixel value OUT of the interpolation pixel is calculated.

OUT = w (−1−distY) × V (0)
+ W (−distY) × V (1)
+ W (1-distY) × V (2)
+ W (2-distY) × V (3) Expression (28)
Through the above processing, it is possible to perform an interpolation processing calculation using an optimal interpolation plane according to the direction of the edge gradient.

  Here, the case where the interpolation calculation is performed using the Lanczos interpolation processing is described as an example, but bi-three-dimensional interpolation (bicubic interpolation) represented by the following equation (29) may be used.

  Further, the interpolation plane may be set in a block larger than the block shown in FIG. In this case, interpolation calculation is performed with n of the Lanczos function set to 3. Here, FIG. 17 shows a case where an interpolation plane is set in a block including interpolation pixels. FIG. 17 is different from FIG. 6 in that the interpolation plane is enlarged by the amount of enlargement of the block. Therefore, detailed description will not be repeated here. Here, a case where interpolation calculation is performed on the interpolation plane set in the blocks shown in FIGS. 17A and 17E will be described as an example. This interpolation plane is referred to as a seventh interpolation plane.

  FIG. 18 is a diagram showing parameters used for the interpolation calculation together with the seventh interpolation plane. Here, a coordinate system is set in which the horizontal direction of the seventh interpolation plane is the X axis and the vertical direction is the Y axis. The coordinate system here is a coordinate system based on the distance between the input pixel and the adjacent input pixel. In FIG. 18, the horizontal direction component and the vertical direction component of the distance between the interpolation pixel and the upper left pixel in the vicinity thereof are shown as distX and distY, respectively. The interpolation calculation is executed in the horizontal direction according to Expression (30) using distX and distY shown in FIG.

V (i) = w (-2-distX) × P (0, i)
+ W (-1−distX) × P (1, i)
+ W (−distX) × P (2, i)
+ W (1-distX) × P (3, i)
+ W (2-distX) × P (4, i)
+ W (3-distX) × P (5, i) (30)
However, 0 ≦ i ≦ 6, and P (X, Y) indicates an input pixel.

  Next, using V (i) calculated by Expression (30), interpolation calculation is performed by Expression (31), and an interpolation pixel value OUT is calculated.

OUT = w (-2-distY) × V (0)
+ W (-1−distY) × V (1)
+ W (−distY) × V (2)
+ W (1-distY) × V (3)
+ W (2-distY) × V (4)
+ W (3-distY) × V (5) Equation (31)
Instead of setting the interpolation plane corresponding to each of the edge gradient directions shown in FIGS. 6A to 6H, the number of interpolation plane patterns to be set may be reduced. For example, any one of four interpolation planes may be set in the block shown in FIG. 19 according to the direction of the edge gradient. Further, by changing the peripheral pixels to be referred to when calculating the temporary pixels as shown in FIG. 8, it is possible to perform the interpolation calculation in consideration of the direction of the edge gradient in each pixel while reducing the calculation amount.

FIG. 20 is a flowchart showing the flow of processing executed by the video signal processing circuit. As shown in FIG. 20, the edge direction estimation processing unit 71 estimates the edge direction of the interpolation pixel (step S31). The direction of the edge gradient is a direction perpendicular to the edge direction, and the edge direction estimation processing unit 71 specifically estimates the direction of the edge gradient of the interpolation pixel. As the direction of the edge gradient, the direction of the edge gradient calculated for the input pixel closest to the interpolation pixel may be used, or the direction of the edge gradient calculated for the input pixel located in the vicinity of the interpolation pixel may be used. A representative value may be used.

  In the next step S32, the interpolation plane setting unit 73 determines whether the direction of the edge gradient estimated in step S31 is within a predetermined angle range. If the edge gradient direction is within the predetermined range, the process proceeds to step S33; otherwise, the process proceeds to step S36. The predetermined angle ranges are π / 16 <θ ≦ 7π / 16 and 9π / 16 ≦ θ <15π / 16. If there is no edge and the edge direction is indefinite, the edge gradient direction is determined to be within a predetermined range.

  In step S33, for the inside of a rectangular block formed of 4 × 4 input pixels located in the vicinity of the interpolation pixel to be interpolated by the enlargement / reduction ratio of the video data according to the number of pixels of the display unit 13, An interpolation plane is set along the direction of the edge gradient calculated in step S31. The interpolation plane is an area determined based on the size of the block, the edge direction, and the number of pixels in the area where the interpolation plane is set. Specifically, the interpolation plane has the largest size that can be accommodated in the block. It is a parallelogram having two sides parallel to the first axis along the interpolation pixel edge direction and two sides parallel to the second axis in a direction different from the first axis. The parallelogram interpolation plane includes 4 × 4 pixels having the same number of pixels as the pixels in the block, which are lattice points arranged along the first axis and the second axis. In some cases, the plurality of pixels included in the interpolation plane include a provisional pixel having no pixel value in addition to the input pixel. The provisional pixel will be described later. The interpolation plane is set so that the interpolation pixel is included in a region surrounded by 2 × 2 pixels at the center of 4 × 4 pixels arranged along the first axis and the second axis.

  The pixels located in the vicinity of the interpolation pixel may be 4 × 4 pixels or 6 × 6 pixels. When the pixels located in the vicinity of the interpolation pixel are 4 × 4 pixels, the interpolation plane corresponding to the direction of the edge gradient is set among the interpolation planes shown in FIGS. If the direction of the edge gradient is indefinite, an interpolation plane corresponding to FIG. 6A or FIG. 6C may be set uniquely, or the edge gradient direction in the neighboring pixels is referred to. Thus, among the interpolation planes shown in FIGS. 6A to 6D, one corresponding to the direction of the referenced edge gradient may be set.

  When the pixels located in the vicinity of the interpolation pixel are 6 × 6 pixels, one corresponding to the direction of the edge gradient is set from the interpolation planes shown in FIGS. If the direction of the edge gradient is indefinite, an interpolation plane corresponding to FIG. 17A or FIG. 17C may be set uniquely, or the direction of the edge gradient in the neighboring pixels is referred to. Then, among the interpolation planes shown in FIGS. 17A to 17D, one corresponding to the direction of the referenced edge gradient may be set.

  In the next step S34, the provisional pixel calculation unit 75 performs provisional pixel calculation processing for calculating the pixel value of the provisional pixel specified in the interpolation plane set in step S33 based on the pixels located in the vicinity thereof. Run. The temporary pixel calculation process is a process of calculating the pixel value of the temporary pixel according to a calculation method defined for each direction of the edge gradient estimated with respect to the pixel located in the vicinity of the temporary pixel. The pixel located near the temporary pixel may be, for example, an input pixel located near the interpolation pixel, two input pixels located near the temporary pixel, or near the temporary pixel. There may be four input pixels located at.

  In the next step S35, the interpolation processing unit 77 calculates the interpolation pixel value by executing the first interpolation process, and ends the process. The first interpolation process is a process for executing an interpolation operation corresponding to the direction of the edge gradient calculated in step S31 on the interpolation plane after the provisional pixel is calculated in step S34.

  Step S36 is a case where the direction of the edge gradient of the interpolated pixel calculated in step S31 is not included in the predetermined angle range. In step S36, the interpolation processing unit 77 calculates the pixel value of the interpolation pixel by executing the second interpolation process, and ends the process. The second interpolation process is a process of executing a conventionally well-known interpolation operation on the interpolation plane after the provisional pixel is calculated in step S34. Specifically, the pixel value of the interpolation pixel is calculated by the method shown in FIG.

<Modification>
The scaler processing unit in the modified example executes correction processing in addition to the processing executed by the scaler processing unit in the present embodiment.

  FIG. 21 is a block diagram illustrating a configuration of a scaler processing unit in a modified example. The scaler processing unit 61A shown in FIG. 21 is different from the scaler processing unit 61 shown in FIG. 3 in that a correction processing unit 79 is added. Other configurations and functions are the same as those of scaler processing unit 61, and thus description thereof will not be repeated here.

  As shown in FIG. 21, the correction processing unit 79 performs correction processing on the image data after the interpolation calculation by the interpolation processing unit 77. The correction process is a process for correcting the blur generated by the interpolation calculation to make the edge clear.

  FIG. 22 is a diagram illustrating a configuration of the correction processing unit. As illustrated in FIG. 22, the correction processing unit 79 includes a maximum value / minimum value calculation processing unit 81, a filter processing unit 83, and a saturation processing unit 85.

  The maximum value / minimum value calculation processing unit 81 calculates the maximum pixel value maxVal and the minimum pixel value minVal from the image data after the interpolation calculation by the interpolation processing unit 77. For the calculation of the maximum pixel value maxVal and the minimum pixel value minVal, a pixel to be processed is set from the image data, and M × N pixels including the set pixel and pixels located in the vicinity thereof are targeted.

  The filter processing unit 83 performs a filter process as a process for sharpening edges on pixels in a predetermined range to be processed by the maximum value / minimum value calculation process unit 81, and sets the pixel value after the filter process. Calculated as the filter processing result “filVal”. The filter processing result “filVal” is calculated according to Expression (32), for example, assuming that the pixel of the image data after being interpolated by the interpolation processing unit 77 is “pixVal”. The pixels in the predetermined range may be M × N pixels to be processed by the maximum value / minimum value calculation processing unit 81, or may be m × n pixels having a smaller range. For example, M × N pixels are 7 × 7 pixels, and m × n pixels are 5 × 5 pixels.

  Here, coef (i, j) represents a filter coefficient.

  The saturation processing unit 85 performs an interpolation operation by the maximum pixel value maxVal and the minimum pixel value minVal calculated by the maximum value / minimum value calculation processing unit 81, the filter processing result fileVal calculated by the filter processing unit 83, and the interpolation processing unit 77. Based on the pixel pixVal of the subsequent image data, a saturation process for controlling the degree of edge sharpening is executed. In the saturation processing, a comparison result based on the filter processing result fillVal and the maximum pixel value maxVal or a comparison result based on the filter processing result fillVal and the minimum pixel value minVal according to the comparison result obtained by comparing the filter processing result fillVal and the pixel pixVal. Is a process for controlling the occurrence of overshoot or undershoot.

  More specifically, in addition to the first comparison result obtained by comparing the filter processing result filVal and the pixel pixVal, the second result obtained by comparing the filter processing result filVal and a value obtained by subtracting a predetermined coefficient α from the minimum pixel value minVal. The correction processing result is determined on the basis of the third comparison result obtained by comparing the comparison result or the filter processing result filVal with the value obtained by adding the predetermined coefficient β to the maximum pixel value maxVal.

  If the first comparison result indicates that the filter processing result fillVal is smaller than the pixel pixVal, the second comparison result is determined, and if not, the third comparison result is determined. In the determination of the second comparison result, when the filter processing result filVal indicates that it is equal to or larger than the value obtained by subtracting the predetermined coefficient α from the minimum pixel value minVal, the filter processing result filVal is determined as the correction processing result. In this case, the minimum pixel value minVal is determined as the correction processing result. In the determination of the third comparison result, when the filter processing result fillVal indicates that it is equal to or less than the value obtained by adding the predetermined coefficient β to the maximum pixel value maxVal, the filter processing result fillVal is determined as the correction processing result, but not so. In this case, the maximum pixel value maxVal is determined as the correction process result.

  The coefficients α and β can be changed according to the degree of emphasizing the edge. Specifically, the occurrence and suppression of undershoot can be controlled by changing the coefficient α, and the occurrence and suppression of overshoot can be controlled by changing the coefficient β. For example, when the coefficient α is set to 0, the occurrence of undershoot can be suppressed, and when the coefficient α is set to 8, an undershoot with a level that is not disturbing to human eyes can be generated. In addition, when the coefficient β is set to 0, the occurrence of overshoot can be suppressed, and when the coefficient β is set to 8, an overshoot of a level that is not disturbing to human eyes can be generated.

  In addition, the saturation process is very effective for correcting the outline of a character or the like, but there is a possibility that the gradation reproduction of the detail portion of the image is impaired. Therefore, when a dynamic range calculated by the difference between the maximum pixel value maxVal and the minimum pixel value minVal has a certain value (for example, 64) in order to identify a contour and a detail portion with strong edge strength, the contour is present. It is possible to determine and perform the correction process, and otherwise, the correction process may not be performed.

  FIG. 23 is a diagram illustrating an example of the effect of suppressing overshoot and undershoot. FIG. 23A is a diagram illustrating an example of image data of one frame among a plurality of frames included in video data. FIG. 23B is a first diagram illustrating an example of image data that has been subjected to filter processing (enhancement processing) without performing correction processing. FIG. 23C is a second diagram illustrating an example of the image data after the correction process. When the filtering process is executed on the image data shown in FIG. 23A, it can be confirmed that overshoot and undershoot occur as shown in FIG. On the other hand, when the correction process was executed on the image data shown in FIG. 23A, it was confirmed that no overshoot and undershoot occurred as shown in FIG.

  FIG. 24 is a flowchart showing a flow of processing executed by the video signal processing circuit in the modification. The process shown in FIG. 24 is different from the process shown in FIG. 20 in that steps S41 to S44 are added. Other processing is the same as the processing shown in FIG. 20, and therefore description thereof will not be repeated here.

  As shown in FIG. 24, in step S41, the correction processing unit 79 determines whether or not to execute the correction process. Specifically, it is determined whether or not the dynamic range calculated from the difference between the maximum pixel value maxVal and the minimum pixel value minVal is a value that is somewhat large (for example, 64). If the dynamic range is a somewhat large value (for example, 64), the process proceeds to step S42; otherwise, the process ends.

  In step S42, the maximum / minimum value calculation processing unit 81 calculates the maximum pixel value maxVal and the minimum pixel value minVal from the image data after the interpolation calculation in step 35 or step S36. For the calculation of the maximum pixel value maxVal and the minimum pixel value minVal, a pixel to be processed is set from the image data, and M × N pixels including the set pixel and pixels located in the vicinity thereof are targeted.

  In step S43, the filter processing unit 83 performs a filter process as a process for sharpening edges on pixels in a predetermined range including the pixels to be processed in step S42 in the image data. The pixel value after processing is calculated as the filter processing result “filVal”. The filter processing result “filVal” is calculated according to the equation (32) using, for example, the pixel “pixVal” of the image data after the interpolation calculation in step S35 or step S36.

  In step S44, the saturation processing unit 85 executes saturation processing and ends the processing.

  FIG. 25 is a flowchart showing the flow of saturation processing. The saturation process is a process executed by the saturation processing unit 85 in step S54 of the process shown in FIG. As shown in FIG. 25, it is determined whether or not the filter processing result fillVal is smaller than the pixel pixVal (step S51). If the filter processing result fillVal is smaller than the pixel pixVal, the process proceeds to step S55; otherwise, the process proceeds to step S52.

  In step S52, it is determined whether or not the filter processing result “filVal” is equal to or smaller than the value obtained by adding the coefficient β to the maximum pixel value “maxVal”. If the filter processing result fillVal is equal to or smaller than the value obtained by adding the coefficient β to the maximum pixel value maxVal, the process proceeds to step S53; otherwise, the process proceeds to step S54.

  The coefficient β may be a fixed value. In this case, the fixed value is 16, for example. Further, it may be a value obtained by multiplying the dynamic range by a predetermined coefficient. Specifically, it is a value obtained by multiplying the difference between the maximum pixel value maxVal and the minimum pixel value minVal by a predetermined coefficient (for example, 0.1).

  In step S53, the filter processing result filVal is determined as the correction processing result, and in step S54, the maximum pixel value maxVal is determined as the correction processing result, and the saturation processing ends.

  Step S55 is a case where it is determined in step S51 that the filter processing result “filVal” is greater than or equal to the pixel “pixVal”. In step S55, it is determined whether or not the filter processing result “filVal” is equal to or greater than the value obtained by subtracting the coefficient α from the minimum pixel value “minVal”. If the filter processing result fillVal is equal to or larger than the value obtained by subtracting the coefficient α from the minimum pixel value minVal, the process proceeds to step S56; otherwise, the process proceeds to step S57.

  The coefficient α may be a fixed value. In this case, the fixed value is 16, for example. Further, it may be a value obtained by multiplying the dynamic range by a predetermined coefficient. Specifically, it is a value obtained by multiplying the difference between the maximum pixel value maxVal and the minimum pixel value minVal by a predetermined coefficient (for example, 0.1).

  In step S56, the minimum pixel value minVal is determined as the correction process result, and in step S57, the filter process result fillVal is determined as the correction process result, and the saturation process ends.

  Therefore, the correction processing described above sharpens the video data and suppresses the occurrence of overshoot and undershoot (over-emphasis), so that it is possible to suppress blurring of the video generated by the interpolation calculation.

  In the present embodiment, the case of the television receiver 1 has been described as an example. However, as shown in FIG. 26, the television receiver 1 is a multi-display in which a plurality of television receivers 1 are arranged in the vertical direction and the horizontal direction. Also good.

  In this embodiment, the direction of the edge gradient is calculated by the canny method using the Sobel filter, but the method of calculating the direction of the edge gradient is not limited to this. A Prewitt filter may be used, or a Scherr filter may be used.

  FIG. 27 is a diagram illustrating a Prewitt filter. FIG. 27A shows a Prewitt filter corresponding to the horizontal direction. FIG. 27B is a diagram illustrating a Prewitt filter corresponding to the vertical direction. FIG. 28 is a diagram illustrating a Scherr filter. FIG. 28A is a diagram illustrating a Scherr filter corresponding to the horizontal direction. FIG. 28B is a diagram illustrating a Scherr filter corresponding to the vertical direction. The edge direction estimation processing unit 71 is similar to the Sobel filter shown in FIGS. 4A and 4B, and the Prewitt filter shown in FIGS. 27A and 27B or the Prewitt filter shown in FIGS. 28A and 28B. The direction of the edge gradient may be calculated according to the equation shown in Equation 7 using a Scherr filter.

  At this time, since the edge direction φ is perpendicular to the direction θ of the edge gradient, the edge direction φ can be calculated according to the equation shown in Equation 8.

  Further, the method of calculating the direction of the edge gradient may be template matching. In this case, the template used for template matching has a predetermined angle for estimating the direction of the edge gradient. Here, a method for calculating the direction of the edge gradient using template matching will be specifically described.

  FIG. 29 is a diagram illustrating an example of a template. Here, templates corresponding to 0, 45, 90, 135, 180 (= 0 degrees), 225 (= 45 degrees), 270 (= 90 degrees), and 315 degrees (= 135 degrees) are shown. The direction of the edge gradient is estimated by matching each of the plurality of templates with image data. Specifically, based on the edge amounts calculated by matching, the angle corresponding to the template that most closely matches the image data among the plurality of templates is specified, and the sum of the edge amounts for each template is calculated. . Then, the direction of the edge gradient is specified based on the specified angle and the total sum of the calculated edge amounts. The template that most closely matches the image data among the plurality of templates is the template that has the maximum edge amount calculated by matching among the plurality of templates. Here, the specification of the edge direction will be specifically described.

  Of the plurality of templates, the degree of matching of the template corresponding to 0 degree is the highest, and the edge amount calculated based on the template corresponding to 0 degree is sufficiently larger than the edge amount calculated based on the other templates. (For example, more than 40% of the total amount of edges), the direction of the edge gradient is in the range of −π / 16 ≦ θ ≦ π / 16 or 15π / 16 ≦ θ ≦ 17π / 16. judge.

  Of the plurality of templates, the template corresponding to 45 degrees has the highest degree of matching, and the edge amount calculated based on the template corresponding to 45 degrees is sufficiently larger than the edge amount calculated based on the other templates. Is larger, the edge gradient direction is determined to be in the range of 3π / 16 <θ ≦ 5π / 16 or −13π / 16 <θ ≦ -11π / 16.

  Of the plurality of templates, the degree of matching of the template corresponding to 90 degrees is the highest, and the edge amount calculated based on the template corresponding to 90 degrees is sufficiently larger than the edge amount calculated based on the other templates. Is larger, the edge gradient direction is determined to be in the range of 7π / 16 <θ ≦ 9π / 16 or −9π / 16 <θ ≦ −7π / 16.

  Of the plurality of templates, the template corresponding to 135 degrees has the highest degree of matching, and the edge amount calculated based on the template corresponding to 135 degrees is sufficiently larger than the edge amount calculated based on the other templates. Is larger, the edge gradient direction is determined to be in the range of 13π / 16 ≦ θ <15π / 16 or −3π / 16 ≦ θ <−π / 16.

  On the other hand, the matching degree of the template corresponding to 0 degree or 45 degrees is the largest among the plurality of templates, but the edge amount calculated based on the template corresponding to 0 degree or 45 degrees is calculated based on the other templates. If the edge amount is not sufficiently large (for example, about 30% of the sum of the edge amounts), the direction of the edge gradient is 3π / 16 <θ ≦ 5π / 16, or −13π / 16 <θ ≦ -11π. It is determined that it is within the range of / 16.

  Of the plurality of templates, the template corresponding to 45 degrees or 90 degrees has the highest degree of coincidence, but the edge amount calculated based on the template corresponding to 0 degree is compared with the edge amount calculated based on the other templates. If it is not sufficiently large, it is determined that the direction of the edge gradient is in the range of 5π / 16 <θ ≦ 7π / 16 or -11π / 16 <θ ≦ -9π / 16.

  Of the plurality of templates, the template corresponding to 90 degrees or 135 degrees has the highest degree of matching, but the edge amount calculated based on the template corresponding to 90 degrees or 135 degrees is sufficiently larger than other edge amounts. Otherwise, it is determined that the direction of the edge gradient is in the range of 9π / 16 <θ ≦ 11π / 16 or −7π / 16 <θ ≦ −5π / 16.

  Of the plurality of templates, the template corresponding to 135 degrees or 0 degrees has the highest degree of coincidence, but the edge amount calculated based on the template corresponding to 135 degrees or 0 degrees is calculated based on another template. If it is not sufficiently larger than the amount, it is determined that the direction of the edge gradient is in the range of 13π / 16 ≦ θ <15π / 16 or −3π / 16 ≦ θ <−π / 16.

  Note that the edge direction may be estimated using template matching. In this case, since the edge direction is perpendicular to the direction of the edge gradient, an angle range including an angle perpendicular to the angle specified by matching may be estimated as the edge direction. For example, if the template corresponding to 0 degrees among the plurality of templates has the highest degree of matching, and the edge amount calculated based on the template corresponding to 0 degrees is sufficiently larger than the other edge amounts ( It is determined that the edge direction is in the range of 7π / 16 <θ ≦ 9π / 16 or −9π / 16 <θ ≦ −7π / 16).

In the present embodiment, the case of performing interpolation using Lanczos interpolation processing or bicubic interpolation (bicubic interpolation) has been described as an example. However, interpolation using spline interpolation processing may be used. Equation (35) is a cubic spline interpolation equation, and the interpolation processing unit 77 uses the values of the temporary pixels calculated by the temporary pixel value calculation processing unit to use the coefficients a _i , b _i , c _i , d _i. Is calculated.

S _i (x) = a _i × (x−x _i ) ³ + b _i × (x−x _i ) ²
+ C _i × (x−x _i ) + d _i (35)
The interpolation processing unit 77 calculates the coefficients a _i , b _i , c _i , d _i according to Expression (35) using six input pixels based on the following conditions.
-Pass through all input pixels.
• Each piecewise interpolation equation has a continuous first derivative of the boundary point.
・ Each piecewise interpolation formula has a continuous second derivative of the boundary point.

The interpolation processing unit 77 uses the calculated coefficients a _i , b _i , c _i , and d _i to calculate an interpolation pixel based on a cubic spline interpolation formula. Note that the pixel indicated by a circle 91 in FIG. 30 is calculated according to Expression (36). Here, distX indicates the distance between the position of the interpolated pixel and the position of the reference input pixel.

S _i (x) = a _i × distX ³ + b _i × distX ²
+ C _i × distX + d _i (36)
The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

  Finally, each block of the television receiver 1, particularly the scaler processing units 61 and 61A, may be configured by hardware logic or may be realized by software using a CPU as follows.

  A recording medium in which a program code (execution format program, intermediate code program, source program) of a control program of the television receiver 1 which is software for realizing the above-described functions is recorded so as to be readable by a computer is recorded on the television receiver 1. This can also be achieved by reading the program code recorded on the recording medium and executing it by the computer (or CPU or MPU).

  Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  Further, the television receiver 1 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Further, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The present invention can be used for image processing such as television.

1 Television receiver (image processing device)
11 Central processing unit (CPU)
21 Video signal processing circuit 51 Video decoder 53 IP conversion processing unit 55 Noise reduction processing unit 57 Sharpness processing unit 59 Color adjustment processing unit 61, 61A Scaler processing unit 71 Edge direction estimation processing unit 73 Interpolation plane setting unit (interpolation region setting unit)
75 Temporary pixel value calculation processing unit 77 Interpolation processing unit 79 Correction processing unit 81 Maximum value / minimum value calculation processing unit 83 Filter processing unit 85 Saturation processing unit

Claims (13)

In the image processing apparatus that enlarges or reduces the image data by providing an interpolation pixel between the input pixels included in the input image data according to the set enlargement / reduction ratio,
For each input pixel, an edge direction indicating an edge direction, or an edge direction estimation processing unit that estimates an edge gradient shifted by a predetermined angle from the edge direction;
When the edge direction of the interpolation pixel specified using the edge direction or edge gradient of the input pixel existing in the vicinity of the interpolation pixel is the interpolation pixel edge direction, the region is along the interpolation pixel edge direction. And an interpolation area setting unit that sets an interpolation area that is an area surrounding the interpolation pixel,
In the interpolation region, a temporary pixel value calculation processing unit that sets a temporary pixel that is a pixel between input pixels and calculates a value of the temporary pixel;
An image processing apparatus comprising: an interpolation processing unit that calculates the value of the interpolation pixel using the values of the input pixel and the temporary pixel included in the interpolation region.   The interpolation region setting unit includes a parallelogram region including a side parallel to the first axis along the interpolation pixel edge direction and two sides parallel to the second axis in a direction different from the first axis. The image processing apparatus according to claim 1, wherein the image processing apparatus is set as the interpolation area.   The interpolation processing unit calculates a value of the interpolated pixel using values of input pixels and temporary pixels arranged along the first axis and the second axis in the interpolation region. The image processing apparatus according to claim 2.   The temporary pixel value calculation processing unit includes a temporary pixel, sets a temporary pixel value calculation area along the interpolation pixel edge direction, and uses the value of the input pixel included in the temporary pixel value calculation area The image processing apparatus according to claim 1, wherein a value of the temporary pixel is calculated.   The provisional pixel value calculation processing unit calculates a value of the provisional pixel using a value of an input pixel adjacent to the provisional pixel in the interpolation pixel edge direction or a direction within a predetermined angle range from the interpolation pixel edge direction. The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus.   When the temporary pixel value calculation processing unit sets the edge direction of the temporary pixel specified using the edge direction or the edge gradient of the input pixel existing in the vicinity of the temporary pixel as the temporary pixel edge direction, the temporary pixel A provisional pixel value calculation area along the provisional pixel edge direction is set, and the value of the provisional pixel is calculated using the value of the input pixel included in the provisional pixel value calculation area. The image processing apparatus according to any one of claims 1 to 3.   When the temporary pixel value calculation processing unit sets the edge direction of the temporary pixel specified using the edge direction or edge gradient of the input pixel existing in the vicinity of the temporary pixel as the temporary pixel edge direction, the temporary pixel 4. The value of the temporary pixel is calculated using a value of an input pixel adjacent in an edge direction or a direction within a predetermined angle range from the temporary pixel edge direction. Image processing apparatus.   The image processing apparatus according to claim 1, further comprising a correction processing unit that sharpens the image data after the interpolation calculation by the interpolation processing unit. The correction processing unit is a maximum value / minimum value calculation process for calculating a maximum pixel value and a minimum pixel value in pixels in a predetermined range including a target pixel from the image data after the interpolation calculation by the interpolation processing unit. And
A filter processing unit for performing a sharpening process;
The maximum pixel value and the minimum pixel value calculated by the maximum / minimum value calculation processing unit, the filter processing result calculated by the filter processing unit, and the image data after the interpolation calculation by the interpolation processing unit The image processing apparatus according to claim 8, further comprising: a saturation processing unit that controls a degree of sharpening based on the number of pixels.   An image display apparatus comprising the image processing apparatus according to claim 1. In an image processing method executed by an image processing apparatus that enlarges or reduces the image data by providing an interpolation pixel between input pixels included in the input image data according to a set enlargement / reduction ratio,
For each input pixel, the image processing apparatus estimates an edge direction indicating an edge direction, or an edge gradient shifted by a predetermined angle from the edge direction;
When the edge direction of the interpolation pixel specified using the edge direction or edge gradient of the input pixel existing in the vicinity of the interpolation pixel is the interpolation pixel edge direction, the region is along the interpolation pixel edge direction. And the step in which the image processing apparatus sets an interpolation area, which is an area surrounding the interpolation pixel,
Setting a temporary pixel that is a pixel between input pixels in the interpolation region, and calculating the value of the temporary pixel by the image processing device ;
Image processing method characterized by comprising the step of the value of the interpolated pixel the image processing apparatus is calculated by using the value of the input pixel and the temporary pixel included in the interpolation region.   A program for operating the image processing apparatus according to any one of claims 1 to 9, wherein the program causes a computer to function as each of the above units. A computer-readable recording medium on which the program according to claim 12 is recorded.

Images