JP4994407B2 - Image processing apparatus and method, and image display apparatus - Google Patents

Description

  The present invention relates to an image processing apparatus and method for emphasizing an input image, and an image display apparatus using the same. For example, when an enlarged image obtained by enlarging an original image is input as an input image, a high frequency component The image enhancement processing is performed so as to obtain an output image with high resolution by generating and adding.

  In general, an image signal representing an image is appropriately subjected to image processing and then reproduced and displayed. In addition, when image enhancement processing is performed on a color image, image enhancement processing is performed on a luminance signal.

  For example, in the image processing apparatus described in Patent Document 1, with respect to a detail image converted to multi-resolution, an enhancement coefficient for a detail image in a desired frequency band is set to a detail image in a lower frequency band than the desired frequency band. The desired frequency band is emphasized by setting based on the above signal.

  Further, in the sharpness enhancement circuit described in Patent Document 2, the first enhancement that emphasizes the luminance component of the input video signal around the frequency band including the highest frequency component of the luminance component. And a second enhancement circuit that enhances the luminance component of the video signal at a lower center frequency than the first enhancement circuit.

JP-A-9-44651 JP 2000-115582 A

  However, in an image processing apparatus that appropriately sets an enhancement coefficient for a detail image in a desired frequency band with respect to a detail image converted to multi-resolution, the enhancement processing is inappropriate or insufficient depending on the input image, and the appropriate image quality Output image could not be obtained.

  For example, when an image that has undergone enlargement processing is input as an input image, a component (folding component) in which a part of the frequency spectrum of the image before enlargement processing is folded is present on the high frequency component side of the frequency spectrum of the input image. appear. Therefore, if the high frequency component is simply emphasized, this aliasing component is emphasized, which is inappropriate processing. In addition, if the frequency band is limited and only the frequency band that does not include the aliasing component is emphasized, when considering the frequency spectrum, emphasis on the high frequency component side is avoided, resulting in insufficient enhancement processing. End up.

  When an image subjected to noise processing is input as an input image, the frequency spectrum on the high frequency component side is lost due to noise processing. Therefore, even if an attempt is made to extract a high frequency component, it cannot be extracted, and image enhancement processing may not be performed sufficiently.

  Further, when emphasis processing is performed on the luminance component of the input video signal, the color may become white (or lighter) near the edge of the chromatic color, or the vicinity of the edge may become black.

The image processing apparatus of the present invention
First intermediate image generation means for generating a first intermediate image obtained by extracting a component of a specific frequency band from a luminance signal of an input image composed of a luminance signal and a color difference signal;
Second intermediate image generation means for generating a second intermediate image based on the first intermediate image;
Luminance color difference addition means for weighted addition of absolute values of the luminance signal and color difference signal of the input image;
First intermediate image processing means for generating a third intermediate image obtained by amplifying the pixel value of the first intermediate image according to a first amplification factor;
Second intermediate image processing means for generating a fourth intermediate image obtained by amplifying the pixel value of the second intermediate image according to a second amplification factor;
In an image processing apparatus having addition means for adding the luminance signal of the input image, the third intermediate image, and the fourth intermediate image,
The second intermediate image generating means includes
A zero-cross determining means for capturing a point where the pixel value of the first intermediate image changes from a positive value to a negative value or from a negative value to a positive value as a zero-cross point;
Signal amplifying means for generating a nonlinear processed image obtained by amplifying a pixel value of a pixel in the vicinity of the zero-cross point among the pixels constituting the first intermediate image at an amplification factor greater than 1,
High-frequency component image generation means for generating a high-frequency component image obtained by extracting only the high-frequency component of the nonlinear processed image;
Outputting the high-frequency component image as the second intermediate image;
The first amplification factor is:
If the sign of the pixel value of the first intermediate image is positive and the output value of the luminance color difference adding means is large, it becomes a small value,
When the sign of the pixel value of the first intermediate image is negative and the output value of the luminance color difference adding means is small, it is determined to be a small value,
The second amplification factor is
If the sign of the pixel value of the second intermediate image is positive and the output value of the luminance color difference adding means is large, it becomes a small value,
When the sign of the pixel value of the second intermediate image is negative and the output value of the luminance / color difference adding means is small, it is determined to be a small value .

Another image processing apparatus of the present invention is
First intermediate image generation means for generating a first intermediate image obtained by extracting a component of a specific frequency band from a luminance signal of an input image composed of a luminance signal and a color difference signal;
Second intermediate image generation means for generating a second intermediate image based on the first intermediate image;
Luminance color difference addition means for weighted addition of absolute values of the luminance signal and color difference signal of the input image;
The pixel value of the first intermediate image becomes a small value when the sign of the pixel value of the first intermediate image is positive and the output value of the luminance color difference adding means is large, and the pixel of the first intermediate image An intermediate image processing means for generating a third intermediate image amplified according to an amplification factor determined to be a small value when the sign of the value is negative and the output value of the luminance color difference adding means is small;
In an image processing apparatus having addition means for adding the luminance signal of the input image, the second intermediate image, and the third intermediate image,
The second intermediate image generating means includes
A zero-cross determining means for capturing a point where the pixel value of the first intermediate image changes from a positive value to a negative value or from a negative value to a positive value as a zero-cross point;
Signal amplifying means for generating a nonlinear processed image obtained by amplifying a pixel value of a pixel in the vicinity of the zero-cross point among the pixels constituting the first intermediate image at an amplification factor greater than 1,
High-frequency component image generation means for generating a high-frequency component image obtained by extracting only the high-frequency component of the nonlinear processed image;
The high frequency component image is output as the second intermediate image .
Still another image processing apparatus according to the present invention provides:
First intermediate image generation means for generating a first intermediate image obtained by extracting a component of a specific frequency band from a luminance signal of an input image composed of a luminance signal and a color difference signal;
Second intermediate image generation means for generating a second intermediate image based on the first intermediate image;
Luminance color difference addition means for weighted addition of absolute values of the luminance signal and color difference signal of the input image;
The pixel value of the second intermediate image is a small value when the sign of the pixel value of the second intermediate image is positive and the output value of the luminance color difference adding means is large, and the pixel of the second intermediate image An intermediate image processing means for generating a third intermediate image amplified according to an amplification factor determined to be a small value when the sign of the value is negative and the output value of the luminance color difference adding means is small;
In an image processing apparatus having an adding means for adding the luminance signal of the input image, the first intermediate image, and the third intermediate image,
The second intermediate image generating means includes
A zero-cross determining means for capturing a point where the pixel value of the first intermediate image changes from a positive value to a negative value or from a negative value to a positive value as a zero-cross point;
Signal amplifying means for generating a nonlinear processed image obtained by amplifying a pixel value of a pixel in the vicinity of the zero-cross point among the pixels constituting the first intermediate image at an amplification factor greater than 1,
High-frequency component image generation means for generating a high-frequency component image obtained by extracting only the high-frequency component of the nonlinear processed image;
The high frequency component image is output as the second intermediate image
It is characterized by that .

  According to the present invention, even when the input image includes a folded component on the high frequency component side in the frequency spectrum, or even when the input image does not sufficiently include the high frequency component, the occurrence of overshoot is sufficiently prevented. Image enhancement processing can be performed, and furthermore, the color does not become unnaturally white (or thin) near the edge of the chromatic color, and the vicinity of the edge does not become black.

It is a block diagram which shows the structure of the image processing apparatus by Embodiment 1 of this invention. It is a block diagram which shows the structural example of the 1st intermediate image generation means 1 of FIG. It is a block diagram which shows the structural example of the 2nd intermediate image generation means 2 of FIG. It is a block diagram which shows the structural example of the 1st intermediate image process means 3M of FIG. It is a block diagram which shows the structural example of the 2nd intermediate image process means 3H of FIG. It is a figure which shows the structural example of the horizontal direction nonlinear processing means 2Ah of FIG. It is a figure which shows the structural example of the vertical direction nonlinear processing means 2Av of FIG. (A)-(E) is a figure which shows arrangement | positioning of the information which shows the arrangement | positioning of the pixel of the brightness | luminance color difference addition image YC, the image D1h, and the image D1v, and the code | symbol of each pixel. It is a block diagram which shows the structural example of the image display apparatus using the image processing apparatus by this invention. It is a block diagram which shows the structural example of the image expansion means U1 of FIG. (A)-(E) is a pixel arrangement | positioning figure which shows operation | movement of the FIG. 10 image expansion means U1. (A)-(D) is a figure which shows the frequency response and frequency spectrum for demonstrating operation | movement of the image expansion means U1 of FIG. (A)-(E) is a figure which shows the frequency response and frequency spectrum for demonstrating operation | movement of the 1st intermediate | middle image generation means 1. FIG. (A)-(C) is a figure which shows the frequency response and frequency spectrum for demonstrating operation | movement of the 2nd intermediate | middle image generation means 2. FIG. (A)-(C) is a figure which shows the value of the signal of the pixel which continues when a step edge and a step edge are sampled by sampling interval S1. (A)-(C) is a figure which shows the value of the signal of the pixel which continues when a step edge and a step edge are sampled by sampling interval S2. (A)-(F) is a figure which shows the value of the signal of the pixel which continues for demonstrating operation | movement of the 1st intermediate image generation means 1 and the 2nd intermediate image generation means 2. FIG. (A) and (B), when the sharpness of the image is increased by appropriately performing the addition of the high frequency component, and when the image quality is deteriorated as a result of excessive addition of the high frequency component It is a figure which shows the value of the signal of the pixel which continues. (A) And (B) is a figure which shows the relationship between the pixel value L of the brightness | luminance color difference addition image YC, and the amplification factor in the 1st intermediate image processing means 3M and the 2nd intermediate image processing means 3H. 3 shows an image processing apparatus according to a second embodiment of the present invention. FIG. 10 is a flowchart illustrating a processing procedure in the image processing apparatus according to the second embodiment. It is a flowchart which shows the process in 1st intermediate | middle image generation step ST1 of FIG. It is a flowchart which shows the process in 2nd intermediate image generation step ST2 of FIG. It is a flowchart which shows the process in horizontal direction nonlinear process step ST2Ah of FIG. FIG. 24 is a flowchart showing processing in the vertical nonlinear processing step ST2Av in FIG. 23. It is a flowchart which shows the process in 1st intermediate | middle image process step ST3M of FIG. It is a flowchart which shows the process in 2nd intermediate image process step ST3H of FIG.

Embodiment 1 FIG.
FIG. 1 is a diagram showing a configuration example of an image processing apparatus according to Embodiment 1 of the present invention. The illustrated image processing apparatus can be used as a part of an image display apparatus, for example.

  An input image IMGIN is input to the illustrated image processing apparatus, and an output image IMGOUT is output. The input image IMGIN is a color image and includes signals YIN (hereinafter referred to as input luminance image YIN) representing luminance components and signals CRIN and CBIN representing color difference components. A signal CRIN (hereinafter referred to as an input CR image CRIN) represents a Cr component of the color difference components, and a signal CBIN (hereinafter referred to as an input CB image CBIN) represents a Cb component of the color difference components. The output image IMGOUT is also a color image, and is composed of signals YOUT (hereinafter referred to as output luminance image YOUT) representing luminance components and signals CROUT and CBOUT representing color difference components. A signal CROUT (hereinafter referred to as an output CR image CROUT) represents a Cr component of the color difference components, and a signal CBOUT (hereinafter referred to as an input CB image CBOUT) represents a Cb component of the color difference components.

  The image processing apparatus according to the first embodiment of the present invention includes a first intermediate image generation unit 1, a second intermediate image generation unit 2, a luminance color difference addition unit 5, a first intermediate image processing unit 3M, Second intermediate image processing means 3H and addition means 4 are provided.

The first intermediate image generating means 1 includes a component in a specific frequency band from the input luminance image YIN (that is, from the first frequency (first predetermined frequency) to the second frequency (second predetermined frequency). An intermediate image (first intermediate image) D1 from which the component is extracted is generated.
The second intermediate image generating means 2 generates an intermediate image (second intermediate image) D2 obtained by performing processing described later on the intermediate image D1.

The luminance color difference adding means 5 generates a luminance color difference addition image YC obtained by weighted addition of the input luminance image YIN, the input CR image CRIN, and the input CB image CBIN as will be described later.
The first intermediate image processing unit 3M generates an intermediate image (third intermediate image) D3M obtained by performing processing described later on the intermediate image D1.
The second intermediate image processing means 3H generates an intermediate image (fourth intermediate image) D3H obtained by performing processing to be described later on the intermediate image D2.
The adding unit 4 adds the input luminance image YIN, the intermediate image D3M, and the intermediate image D3H.

  In the image processing apparatus of the illustrated embodiment, only the luminance component is processed. That is, the output image of the adding means 4 is output as the output luminance image YOUT, while the input CR image CRIN is output as it is as the output CR image CROUT, and the input CB image CBIN is output as it is as the output CB image CBOUT.

  FIG. 2 is a diagram illustrating a configuration example of the first intermediate image generation unit 1. The illustrated first intermediate image generation unit 1 extracts only a high frequency component equal to or higher than the first frequency from the input luminance image YIN. High frequency component image generation means 1A for generating the image D1A, and low frequency component image generation means 1B for generating an image D1B obtained by extracting only low frequency components equal to or lower than the second frequency of the image D1A. The high frequency component image generating means 1A and the low frequency component image generating means 1B constitute band pass filter means for extracting components in a specific frequency band. From the first intermediate image generating means 1, an image D1B is output as an intermediate image D1.

  FIG. 3 is a diagram illustrating a configuration example of the second intermediate image generation unit 2. The illustrated second intermediate image generation unit 2 outputs an image D <b> 2 </ b> A obtained by performing non-linear processing described later on the intermediate image D <b> 1. Non-linear processing means 2A, and high-frequency component image generation means 2B that outputs an image D2B obtained by extracting only high-frequency components equal to or higher than a third frequency (third predetermined frequency) of the image D2A. From the second intermediate image generating means 2, an image D2B is output as an intermediate image D2.

  FIG. 4 is a diagram illustrating a configuration example of the first intermediate image processing unit 3M. The illustrated first intermediate image processing unit 3M includes an amplification factor determination unit 3MA and a pixel value amplification unit 3MB. The amplification factor determination means 3MA determines the amplification factor D3MA based on the pixel value of the luminance / color difference addition image YC described later. The pixel value amplification unit 3MB amplifies the pixel value of the intermediate image D1 with the amplification factor D3MA determined by the amplification factor determination unit 3MA, and outputs the result as the intermediate image D3MB. Then, the intermediate image D3MB is output as the intermediate image D3M from the first intermediate image processing means 3M.

  The amplification factor determination unit 3MA includes a horizontal direction amplification factor determination unit 3MAh and a vertical direction amplification factor determination unit 3MAv. The pixel value amplification unit 3MB includes the horizontal direction pixel value amplification unit 3MBh and the vertical direction pixel value amplification unit. 3 MBv. The horizontal direction gain determining means 3MAh and the horizontal direction pixel value amplifying means 3MBh constitute a first horizontal direction intermediate image processing means 3Mh. The vertical direction gain determining means 3MAv, the vertical direction pixel value amplifying means 3MBv, Thus, the first vertical intermediate image processing means 3Mv is configured.

  FIG. 5 is a diagram showing a configuration example of the second intermediate image processing unit 3H. The illustrated second intermediate image processing unit 3H includes an amplification factor determination unit 3HA and a pixel value amplification unit 3HB. The amplification factor determining means 3HA determines the amplification factor D3HA based on the pixel value of the luminance / color difference addition image YC described later. The pixel value amplifying unit 3HB amplifies the pixel value of the intermediate image D2 with the amplification factor D3HA determined by the amplification factor determining unit 3HA, and outputs the result as the intermediate image D3HB. The intermediate image D3HB is output as the intermediate image D3H from the first intermediate image processing means 3H.

  The amplification factor determination unit 3HA includes a horizontal direction amplification factor determination unit 3HAh and a vertical direction amplification factor determination unit 3HAv. The pixel value amplification unit 3HB includes the horizontal direction pixel value amplification unit 3HBh and the vertical direction pixel value amplification unit. 3HBv. The horizontal direction gain determining means 3HAh and the horizontal direction pixel value amplifying means 3HBh constitute a second horizontal direction intermediate image processing means 3Hh. The vertical direction gain determining means 3HAv, the vertical direction pixel value amplifying means 3HBv, Thus, the second vertical intermediate image processing means 3Hv is configured.

  The adding unit 4 adds the intermediate image D3M and the intermediate image D3H to the input luminance image YIN to generate an output luminance image YOUT.

The detailed operation of the image processing apparatus according to the first embodiment of the present invention will be described below.
First, the detailed operation of the first intermediate image generating unit 1 will be described.
The first intermediate image generating unit 1 generates an image D1A in which only the high frequency component equal to or higher than the first frequency of the input luminance image YIN is extracted in the high frequency component image generating unit 1A.
High frequency components can be extracted by performing high-pass filter processing. High frequency components are extracted in the horizontal and vertical directions of the image. That is, the high frequency component image generating means 1A performs horizontal high-pass filter processing on the input luminance image YIN to generate an image D1Ah in which a high frequency component equal to or higher than the first horizontal frequency is extracted only in the horizontal direction. Direction high-frequency component image generation means 1Ah and vertical high-frequency component image generation means for generating an image D1Av obtained by performing high-pass filtering in the vertical direction and extracting a high-frequency component equal to or higher than the second vertical frequency only in the vertical direction The image D1A includes an image D1Ah and an image D1Av.

  Next, the first intermediate image generation unit 1 generates an image D1B in which only the low frequency component equal to or lower than the second frequency of the image D1A is extracted in the low frequency component image generation unit 1B. The low frequency component can be extracted by performing a low pass filter process. The low frequency component is extracted in each of the horizontal direction and the vertical direction. That is, the low-frequency component image generation means 1B performs horizontal low-pass filter processing on the image D1Ah to generate an image D1Bh in which a low-frequency component below the second horizontal frequency is extracted only in the horizontal direction. Component image generation means 1B and vertical direction low-frequency component image generation that performs low-pass filter processing in the vertical direction on the image D1Av and generates an image D1Bv that extracts a low-frequency component below the second vertical frequency only in the vertical direction Means 1Bv, and the image D1B is composed of an image D1Bh and an image D1Bv. From the first intermediate image generating means 1, the image D1B is output as the intermediate image D1. The intermediate image D1 includes an image D1h corresponding to the image D1Bh and an image D1v corresponding to the image D1Bv.

Next, the detailed operation of the second intermediate image generating means 2 will be described.
First, the second intermediate image generation unit 2 generates an image D2A obtained by performing nonlinear processing described later on the intermediate image D1 in the nonlinear processing unit 2A. Nonlinear processing is performed in each of the horizontal direction and the vertical direction. That is, the nonlinear processing means 2A performs horizontal nonlinear processing means 2Ah that performs nonlinear processing described later on the image D1h to generate an image D2Ah, and vertical that performs nonlinear processing described later on image D1v to generate the image D2Av. The image D2A includes an image D2Ah and an image D2Av.

  The operation of the nonlinear processing means 2A will be described in more detail. The nonlinear processing means 2A includes a horizontal nonlinear processing means 2Ah and a vertical nonlinear processing means 2Av having the same configuration. The horizontal nonlinear processing means 2Ah performs horizontal processing, and the vertical nonlinear processing means 2Av performs vertical processing.

  FIG. 6 is a diagram showing a configuration example of the horizontal nonlinear processing means 2Ah. The illustrated horizontal non-linear processing means 2Ah includes a zero-cross determining means 311h and a signal amplifying means 312h. The image D1h is input to the nonlinear processing means 2Ah as the input image DIN311h.

  The zero-cross determining unit 311h checks the change in the pixel value in the input image DIN 311h along the horizontal direction. A point where the pixel value changes from a positive value to a negative value or from a negative value to a positive value is regarded as a zero-cross point, and pixels (adjacent pixels before and after the zero-cross point) before and after the zero-cross point by a signal D311h. The position is transmitted to the signal amplification means 312h. Here, “front and back” refers to the front and rear in the order in which signals are supplied. When a pixel signal is supplied from left to right in the horizontal direction, it means “left and right”, and the vertical and horizontal directions of the pixel When a signal is supplied, it means “up and down”. In the zero cross determination means 311h, the zero cross determination means 311h in the horizontal non-linear processing means 2Ah recognizes pixels located on the left and right of the zero cross point as pixels located before and after the zero cross point.

  Based on the signal D311h, the signal amplifying unit 312h identifies pixels before and after the zero cross point (adjacent pixels before and after the zero cross point), and amplifies the pixel value only for the pixels before and after the zero cross point (the absolute value is increased). A nonlinear processed image D312h is generated. That is, the amplification factor is set to a value larger than 1 for pixel values of pixels around the zero cross point, and the amplification factor is set to 1 for the pixel values of other pixels.

  A nonlinear processed image D312h is output from the horizontal nonlinear processing means 2Ah as an image D2Ah.

  FIG. 7 is a diagram showing a configuration example of the vertical nonlinear processing means 2Av. The illustrated vertical nonlinear processing means 2Av includes a zero-cross determination means 311v and a signal amplification means 312v. The image D1v is input as the input image DIN311v to the nonlinear processing means 2Av.

  The zero-cross determination unit 311v confirms the change of the pixel value in the input image DIN 311v along the vertical direction. A point where the pixel value changes from a positive value to a negative value or from a negative value to a positive value is regarded as a zero-cross point, and pixels (adjacent pixels before and after the zero-cross point) before and after the zero-cross point by the signal D311v. The position is transmitted to the signal amplification means 312v. In the zero cross determination means 311v in the vertical nonlinear processing means 2Av, pixels located above and below the zero cross point are recognized as pixels located before and after the zero cross point.

Based on the signal D311v, the signal amplifying unit 312v identifies pixels before and after the zero cross point (adjacent pixels before and after the zero cross point), and amplifies the pixel value only for the pixels before and after the zero cross point (the absolute value is increased). A nonlinear processed image D312v is generated. That is, the amplification factor is set to a value larger than 1 for pixel values of pixels around the zero cross point, and the amplification factor is set to 1 for the pixel values of other pixels.
The above is the operation of the nonlinear processing means 2A.

  Next, the second intermediate image generation means 2 generates an image D2B in which only the high frequency component equal to or higher than the third frequency of the image D2A is extracted in the high frequency component image generation means 2B. High frequency components can be extracted by performing high-pass filter processing. High frequency components are extracted in the horizontal and vertical directions of the image. That is, the high frequency component image generation means 2B performs a horizontal high-pass filter process on the image D2Ah to generate an image D2Bh in which a high frequency component equal to or higher than the third horizontal frequency is extracted only in the horizontal direction. Component image generation means 2Bh and vertical high-frequency component image generation for generating an image D2Bv obtained by performing high-pass filter processing in the vertical direction on the image D2Av and extracting a high-frequency component equal to or higher than the third vertical frequency only in the vertical direction Means 2Bv, and the image D2B is composed of an image D2Bh and an image D2Bv. From the second intermediate image generating means 2, the image D2B is output as the intermediate image D2. The intermediate image D2 includes an image D2h corresponding to the image D2Bh and an image D2v corresponding to the image D2Bv.

The detailed operation of the luminance / color difference adding means 5 will be described. The luminance color difference adding means 5 performs weighted addition (by weighted addition only) for each pixel, the pixel value of the input luminance image YIN, the absolute value of the pixel value of the input CR image CRIN, and the absolute value of the pixel value of the input CB image CBIN. ), A luminance color difference addition image YC is generated. That is, the luminance color difference addition image YC is obtained by the following equation using the input luminance image YIN, the input CR image CRIN, and the input CB image CBIN.
YC = Ky · YIN + Kcr · | CRIN | + Kcb · | CBIN | (1)
Here, Ky, Kcr, and Kcb are coefficients for weighting.

Next, a detailed operation of the first intermediate image processing unit 3M will be described.
The first intermediate image processing unit 3M determines the amplification factor D3MA based on the sign sD1 of the pixel value of the luminance / color difference addition image YC and the pixel value of the intermediate image D1 in the amplification factor determination unit 3MA. Here, since the first intermediate image D1 includes the image D1h and the image D1v, the code sD1 includes a code sD1h representing the code of the image D1h and a code sD1v representing the code of the image D1v. As described above, the pixel value of the first intermediate image D1 is amplified based on the amplification factor D3MA. Since the first intermediate image D1 is composed of the image D1h and the image D1v, as the amplification factor D3MA, An amplification factor D3MAh for the image D1h and an amplification factor D3MAv for the image D1v are determined. That is, the amplification factor determination unit 3MA includes a horizontal direction amplification factor determination unit 3MAh and a vertical direction amplification factor determination unit 3MAv. In the horizontal direction amplification factor determination unit 3MAh, the pixel value of the luminance color difference addition image YC and the image D1h The amplification factor D3MAh is determined based on the pixel value code sD1h, and the vertical amplification factor determination means 3MAv determines the amplification factor D3MAv based on the pixel value of the luminance-color difference added image YC and the pixel value code sD1v of the image D1v. Then, amplification factor D3MAh and amplification factor D3MAv are output as amplification factor D3MA from amplification factor determination means 3MA.

  The operations of the horizontal direction gain determining means 3MAh and the vertical direction gain determining means 3MAv will be described in more detail.

  FIGS. 8A to 8E are diagrams showing the luminance color difference addition image YC, the images D1h and D1v, the pixel value code sD1h of the image D1h, and the pixel value code sD1v of the image D1v. ) Represents the luminance color difference addition image YC, FIG. 8B represents the image D1h, FIG. 8C represents the image D1v, FIG. 8D represents the code sD1h, and FIG. 8E represents the code sD1v. Yes. 8A to 8E show horizontal coordinates, vertical coordinates, and coordinate values according to the horizontal and vertical directions of the image. In addition, for the luminance / color difference added image YC, the pixel value of the pixel at the position of the horizontal coordinate x and the vertical coordinate y is represented by the symbol YC (xy), and for the image D1h, the horizontal coordinate x and the vertical coordinate y The pixel value of the pixel at the position is represented by the symbol D1h (xy), and for the image D1v, the pixel value of the pixel at the position of the horizontal coordinate x and the vertical coordinate y is represented by the symbol D1v (xy). Has been. For the sign sD1h of the pixel value of the image D1h, the sign of the pixel at the position of the horizontal coordinate x and the vertical coordinate y is represented by the symbol sD1h (xy), and for the sign sD1v of the pixel value of the image D1v, The code | symbol of the pixel in the position of the coordinate x and the vertical coordinate y is represented by the symbol sD1v (xy).

  The horizontal direction amplification factor determination means 3MAh determines the amplification factor for each pixel of the image D1h based on the sign of the pixel value of the same coordinate of the luminance color difference addition image YC and the pixel value of the same coordinate of the image D1h. That is, the amplification factor for the pixel value D1h (11) is determined based on the pixel value YC (11) and the symbol sD1h (11), and the amplification factor for the pixel value D1h (12) is the pixel value YC (12) and the symbol sD1h. If determined based on (12) and generalized, the amplification factor for the pixel value D1h (xy) is determined based on the pixel value YC (xy) and the code sD1h (xy), and so on. The amplification factor is determined based on the sign of the pixel value at the same coordinate in the image YC and the pixel value at the same coordinate in the image D1h, and the result is output as the amplification factor D3MAh.

  Further, the vertical direction amplification factor determining means 3MAv determines the amplification factor for each pixel of the image D1v based on the sign of the pixel value of the same coordinate of the luminance color difference addition image YC and the pixel value of the same coordinate of the image D1v. That is, the amplification factor for the pixel value D1v (11) is determined based on the pixel value YC (11) and the symbol sD1v (11), and the amplification factor for the pixel value D1v (12) is the pixel value YC (12) and the symbol sD1v. If determined based on (12) and generalized, the amplification factor for the pixel value D1v (xy) is determined based on the pixel value YC (xy) and the code sD1v (xy), and so on. The amplification factor is determined based on the sign of the pixel value at the same coordinate in the image YC and the pixel value at the same coordinate in the image D1v, and the result is output as the amplification factor D3MAv.

  Next, the pixel value amplifying unit 3MB amplifies the pixel value of the first intermediate image D1 based on the amplification factor D3MA. Since the first intermediate image D1 includes the image D1h and the image D1v, the pixel value amplifying unit 3MB amplifies the pixel value of the image D1v and the horizontal pixel value amplifying unit 3MBh for amplifying the pixel value of the image D1h. Vertical direction pixel value amplifying means 3MBv. That is, the horizontal pixel value amplifying unit 3MBh outputs an image D3MBh obtained by amplifying the pixel value of the image D1h based on the amplification factor D3MAh, and the vertical pixel value amplifying unit 3MBv outputs the pixel of the image D1v based on the amplification factor D3MAv. An image D3MBv whose value has been amplified is output. Then, the image value D3MBh and the image D3MBv are output as the image D3MB from the pixel value amplifying unit 3MB.

The image D3MB is output from the first intermediate image processing means 3M as the intermediate image D3M. The intermediate image D3M includes an image D3Mh corresponding to the image D3MBh and an image D3Mv corresponding to the image D3MBv.
The above is the operation of the first intermediate image processing means 3M.

  Next, the operation of the second intermediate image processing means 3H will be described. Comparing FIG. 4 and FIG. 5, the second intermediate image processing means has the same configuration as the first intermediate image processing means except that the input signals are the luminance color difference addition image YC and the intermediate image D2. The second intermediate image processing unit 3H outputs the intermediate image D3H obtained by performing the same process on the intermediate image D2 as the first intermediate image processing unit 3M performs on the intermediate image D1. To do. Since the detailed operation of the second intermediate image processing unit 3H is also apparent from the detailed operation of the first intermediate image processing unit 3M described above, the detailed operation of the second intermediate image processing unit 3H will be described. Description of is omitted.

  Finally, the operation of the adding means 4 will be described. The adding unit 4 generates an output luminance image YOUT obtained by adding the input luminance image YIN, the intermediate image D3M, and the intermediate image D3H. The output luminance image YOUT of the adding means 4 and the output image IMGOUT composed of the output CR image CROUT and the output CB image CBOUT are output from the image processing apparatus as the final output image.

  The intermediate image D3M is composed of an image D3Mh and an image D3Mv, and the intermediate image D3H is composed of an image D3Hh and an image D3Hv. Therefore, when the input luminance image YIN, the intermediate image D3M, and the intermediate image D3H are added, the input luminance image This means that all of images D3Mh, D3Mv, D3Hh, and D3Hv are added to YIN.

  Here, the addition processing by the adding means 4 is not limited to simple addition, and weighted addition may be performed. That is, each of the images D3Mh, D3Mv, D3Hh, and D3Hv may be amplified with different amplification factors and then added to the input luminance image YIN.

  Hereinafter, an example in which the image processing apparatus according to the present invention is used as a part of an image display apparatus will be described. Through this description, the operation and effect of the image processing apparatus according to the present invention will become clear. In the following description, the symbol Fn represents the Nyquist frequency of the input image IMGIN unless otherwise specified.

  FIG. 9 shows an image display apparatus using the image processing apparatus according to the present invention. In the illustrated image display apparatus, an image corresponding to the original image IMGORG is displayed on the monitor U3.

  When the image size of the original image IMORG is smaller than the image size of the monitor U3, the color image enlarging means U1 outputs an image IMGU1 obtained by enlarging the original image IMORG. Here, the image can be enlarged by, for example, a high cubic method.

  The image processing device U2 in the present invention uses the image IMGU1 as the input image IMGIN, and outputs the image IMGOUT obtained by performing the above-described processing on the input image IMGIN as the image DU2. An image DU2 is displayed on the monitor U3.

Note that the image DU2 is divided into a luminance signal (Y) and color difference signals (Cr, Cb) (hereinafter sometimes referred to as YCbCr format), and therefore, normally, red (R) green before being displayed on the monitor U3. (G) It is converted into a blue (B) color signal (hereinafter also referred to as RGB format). The conversion between the YCbCr format and the RGB format is, for example, a recommendation ITU-R. The conversion from RGB format to YCbCr format is described in BT601 etc. Y = 0.299R + 0.587G + 0.114B
Cr = 0.500R-0.419G-0.081B
Cb = −0.169R−0.331G + 0.500B
... (2)
The conversion from YCbCr format to RGB format is R = 1.000Y + 1.402Cr + 0.000Cb
G = 1.000Y-0.714Cr-0.344Cb
B = 1.000Y + 0.000Cr + 1.772Cb
... (3)
It is done by. Note that the coefficients shown in the equations (2) and (3) are examples, and the present invention is not limited to these. When the input image is 8-bit data, the values of Cr and Cb are usually rounded to a range of −127 to 128.

  Hereinafter, assuming that the number of pixels of the original image IMGORG is half the number of pixels of the monitor U3 in both the horizontal direction and the vertical direction, the operation and action of the color image enlarging means U1 will be described first.

  The color image enlarging means U1 includes an image enlarging means U1Y that generates an image DU1Y obtained by enlarging the image YORG that represents the luminance component of the original image IMORG, an image enlarging means U1CR that generates an image DU1CR obtained by enlarging the image CRORG that represents the Cr component, and Cb. Image enlarging means U1CB that generates an image DU1CB obtained by enlarging the image CBORG representing the component. The configurations of the image enlarging unit U1Y, the image enlarging unit U1CR, and the image enlarging unit U1CB can all be the same as the image enlarging unit U1 shown in FIG.

FIG. 10 is a diagram showing a configuration example of the image enlarging means U1, and the image enlarging means U1 includes a horizontal zero insertion means U1A, a horizontal low frequency component passing means U1B, a vertical zero insertion means U1C, and a vertical low Frequency component passing means U1D.
An input image to the image enlarging means U1 is represented as an original image DORG.
The horizontal zero inserting means U1A appropriately inserts pixels having a pixel value of 0 in the horizontal direction of the original image DORG (one pixel column consisting of pixels having a pixel value of 0 between adjacent pixel columns in the horizontal direction of the original image DORG). The image DU1A is inserted).

The horizontal direction low frequency component passing means U1B generates an image DU1B in which only the low frequency component of the image DU1A is extracted by low pass filter processing.
The vertical zero insertion means U1C appropriately inserts pixels having a pixel value of 0 in the vertical direction of the image DU1B (one pixel row consisting of pixels having a pixel value of 0 between adjacent pixel rows in the vertical direction of the image DU1B). The image DU1C is inserted).
The vertical direction low frequency component passing means DU1D generates an image DU1D obtained by extracting only the low frequency component of the image DU1C.
The image DU1D is output from the image enlarging means U1 as an image DU1 obtained by doubling the original image DORG in both the horizontal and vertical directions.

  11A to 11E are diagrams for explaining the operation of the image enlarging means U1 in detail. FIG. 11A shows the original image DORG, FIG. 11B shows the image DU1A, and FIG. ) Represents the image DU1B, FIG. 11D represents the image DU1C, and FIG. 11E represents the image DU1D. 11A to 11E, squares (squares) represent pixels, and symbols or numerical values written therein represent pixel values of the respective pixels.

  The horizontal direction zero insertion means U1A inserts one pixel having a pixel value of 0 for each pixel in the horizontal direction with respect to the original image DORG shown in FIG. 11A (that is, adjacent to the original image DORG in the horizontal direction). An image DU1A shown in FIG. 11B is generated by inserting one pixel row composed of pixels having a pixel value of 0 between the pixel rows. The horizontal direction low frequency component passing means U1B performs a low-pass filter process on the image DU1A shown in FIG. 11B to generate an image DU1B shown in FIG.

  The vertical zero insertion unit U1C inserts one pixel having a pixel value of 0 for each pixel in the vertical direction into the image DU1B shown in FIG. 11C (that is, adjacent to the image DU1B in the vertical direction). An image DU1C shown in FIG. 11D is generated by inserting one pixel row composed of pixels having a pixel value of 0 between the pixel rows. The vertical low frequency component passing means U1D performs a low pass filter process on the image DU1C shown in FIG. 11D, and generates an image DU1D shown in FIG. With the above processing, an image DU1D in which the original image DORG is doubled in both the horizontal direction and the vertical direction is generated.

  12 (A) to 12 (D) show the effect of processing by the image enlarging means U1 on the frequency space, FIG. 12 (A) is the frequency spectrum of the original image DORG, and FIG. 12 (B) is the image DU1A. FIG. 12C shows the frequency spectrum, FIG. 12C shows the frequency response of the horizontal frequency component passing means U1B, and FIG. 12D shows the frequency spectrum of the image DU1B. 12A to 12D, the horizontal axis represents the frequency axis representing the spatial frequency in the horizontal direction, and the vertical axis represents the frequency spectrum or the intensity of the frequency response.

The number of pixels of the original image DORG is half that of the image DU1 in both the horizontal direction and the vertical direction. In other words, the sampling interval of the original image DORG is twice the sampling interval of the image DU1. Therefore, the Nyquist frequency of the original image DORG is half of the Nyquist frequency of the image DU1.
On the other hand, the image IMGU1 output from the image enlarging means U1 (or the image enlarging means U1Y, the image enlarging means U1CR, and the image enlarging means U1CB) is used as the input image IMGIIN. The number of pixels is the same. That is, the Nyquist frequency of the image DU1 is also Fn. Therefore, the Nyquist frequency of the original image DORG is half of the Nyquist frequency of the input image input image IMGIN, that is, Fn / 2.

  In FIGS. 12A to 12D, only one frequency axis is used to simplify the notation. However, normally, image data consists of pixel values given on a pixel array arranged in a two-dimensional plane, and its frequency spectrum is also given on a plane stretched by a horizontal frequency axis and a vertical frequency axis. It is. Therefore, in order to accurately represent the frequency spectrum of the original picture DORG or the like, it is necessary to describe both the horizontal frequency axis and the vertical frequency axis. However, the shape of the frequency spectrum usually spreads isotropically around the origin on the frequency axis, and as long as the frequency spectrum in the space spanned by one frequency axis is shown, the frequency axis It is easy for those skilled in the art to expand and consider the space spanned by two. Therefore, unless otherwise specified in the following description, the description on the frequency space is performed using a space stretched by one frequency axis.

  As signals input as the original image DORG, there are an image YORG representing a luminance component, and images CRORG and CBORG representing a color difference component. However, the human eye is not sensitive to abrupt changes in color difference components (or high frequency components of color difference signals). Therefore, when the images CRORG and CBORG representing the color difference components are input as the original image DORG, the aliasing component described below does not cause much problem. Therefore, in the following description, it is assumed that an image YORG representing the luminance component of the original image IMGORG is input as the original image DORG.

  First, the frequency spectrum of the original picture DORG will be described. Normally, a natural image is input as the original image DORG. In this case, the spectral intensity of the luminance component is concentrated around the origin of the frequency space. Therefore, the frequency spectrum of the original picture DORG is like the spectrum SPO in FIG.

  Next, the spectral intensity of the image DU1A will be described. The image DU1A is generated by inserting one pixel per pixel and a pixel value of 0 in the horizontal direction with respect to the original image DORG. When such processing is performed, aliasing around the Nyquist frequency of the original picture DORG occurs in the frequency spectrum. That is, since a spectrum SPM in which the spectrum SPO is folded around the frequency ± Fn / 2 is generated, the frequency spectrum of the image DU1A is expressed as shown in FIG.

  Next, the frequency response of the horizontal low frequency component passing means U1B will be described. Since the horizontal low-frequency component passing means is realized by a low-pass filter, the frequency response becomes lower as the frequency becomes higher as shown in FIG.

Finally, the frequency spectrum of the image DU1B will be described. An image DU1B shown in FIG. 12D is obtained by performing low-pass filter processing having a frequency response shown in FIG. 12C on the image DU1A having the frequency spectrum shown in FIG. .
Therefore, as shown in the image DU1B, the frequency spectrum of the image DU1B includes a spectrum SP2 in which the intensity of the spectrum SPM has dropped to some extent and a spectrum SP1 in which the intensity of the spectrum SPO has dropped to some extent. In general, the frequency response of the low-pass filter decreases as the frequency increases. Therefore, when the intensity of the spectrum SP1 is compared with the spectrum SPO, the spectrum intensity on the high frequency component side, that is, in the vicinity of the frequency of ± Fn / 2 is reduced by the horizontal low frequency component passing means U1B.

  Of the processing by the image enlarging means U1, the description of the operation on the frequency space of the processing by the vertical zero insertion means U1C and the vertical low frequency component passing means U1D is omitted, but from the contents of the processing. It can be easily understood that there is an action similar to that described with reference to FIGS. 12A to 12D with respect to the axial direction representing the spatial frequency in the vertical direction. That is, the frequency spectrum of the image DU1D is obtained by spreading the frequency spectrum shown in FIG.

  In the following description, the spectrum SP2 is referred to as a folded component. This aliasing component appears on the image as noise or a false signal having a relatively high frequency component. Such noise or false signals include overshoot, jaggy or ringing.

The operation and effect of the image processing apparatus according to the present invention will be described below.
FIGS. 13A to 13E show a case where the intermediate image D1 is generated from the input luminance image YIN when the image IMGU1 obtained by enlarging the original image IMORG as the input image IMGUIN (or the image IMGU1) is input. It is a diagram schematically showing the action and effect,
13A shows the frequency spectrum of the input luminance image YIN, FIG. 13B shows the frequency response of the high frequency component image generating means 1A, and FIG. 13C shows the frequency response of the low frequency component image generating means 1B. 13D shows the frequency response of the first intermediate image generating means 1, and FIG. 13E shows the frequency spectrum of the intermediate image D1. In FIGS. 13A to 13E, only one frequency axis is used for the same reason as in FIGS. 12A to 12D.

  Further, in FIGS. 13A to 13E, the intensity of the frequency spectrum or the frequency response is shown only in the range where the spatial frequency is 0 or more, but the frequency spectrum or the frequency response in the following description is on the frequency axis. It becomes a symmetric shape around the origin. Therefore, the figure used for description is sufficient to show only the range where the spatial frequency is 0 or more.

  First, the frequency spectrum of the input luminance image YIN will be described. Since the image IMGU1 generated by the enlargement process in the image enlarging means U1Y is input as the input luminance image YIN, the frequency spectrum of the input luminance image YIN has been described with reference to FIG. 12D as shown in FIG. The spectrum SP1 has the same shape as that of the original, and the spectrum SPO of the original image IMGORG has dropped to some extent, and the spectrum SP2 that is the aliasing component.

  Next, the frequency response of the high frequency component image generating unit 1A will be described. Since the high frequency component image generating means 1A is composed of a high-pass filter, the frequency response becomes lower as the frequency becomes lower as shown in FIG.

  Next, the frequency response of the low frequency component image generation means 1B will be described. Since the low frequency component image generating means 1B is composed of a low pass filter, the frequency response thereof becomes lower as the frequency becomes higher as shown in FIG.

  Next, the frequency response of the first intermediate image generating unit 1 will be described. Among the frequency components of the input luminance image YIN, the first frequency component of the low frequency component side region (band of frequencies lower than the “first frequency FL1”) RL1 shown in FIG. Is weakened by the high-frequency component image generation means 1A in the intermediate image generation means 1. On the other hand, for the frequency component of the high frequency component side region (band of frequencies higher than the second frequency FL2) RH1 shown in FIG. 13D, the low frequency component in the first intermediate image generating means 1 is used. It is weakened by the image generation means 1B. Therefore, as shown in FIG. 13D, the frequency response of the first intermediate image generating means 1 is an intermediate region (band region limited by the low frequency component side region RL1 and the high frequency component side region RH1). (Specific frequency band) RM1 has a peak.

  Next, the frequency spectrum of the intermediate image D1 will be described. The input luminance image YIN having the frequency spectrum shown in FIG. 13 (A) passes through the first intermediate image generating means 1 having the frequency response shown in FIG. 13 (D), so that it is shown in FIG. 13 (E). An intermediate image D1 is obtained. Since the frequency response of the first intermediate image generating means 1 has a peak in the intermediate region RM1 band-limited by the region RL1 on the low frequency component side and the region RH1 on the high frequency component side, the intermediate image D1 In the frequency spectrum of the input luminance image YIN, the intensity of the portion included in the low frequency component side region RL1 and the high frequency component side region RH1 is weakened. Therefore, the intermediate image D1 is obtained by removing the spectrum SP2 that is the aliasing component from the high frequency component of the input luminance image YIN. That is, the first intermediate image generating means 1 has an effect of generating the intermediate image D1 by removing the spectrum SP1 that is the aliasing component from the high frequency component of the input luminance image YIN.

  FIGS. 14A to 14C are diagrams showing the operation and effect of the second intermediate image generation unit 2, FIG. 14A shows the frequency spectrum of the nonlinear processed image D2A, and FIG. 14B shows the frequency spectrum. FIG. 14C shows the frequency response of the high frequency component image generating means 2B, and FIG. 14C shows the frequency spectrum of the image D2B. 14A to 14C show the frequency spectrum or frequency response only in the range where the spatial frequency is 0 or more for the same reason as in FIGS. 13A to 13E.

  As will be described later, in the nonlinear processed image D2A, a high frequency component corresponding to the region RH2 on the high frequency component side is generated. FIG. 14A is a diagram schematically showing the state. An image D2B shown in FIG. 14C is generated by passing the nonlinear processed image D2A through the high frequency component image generating means 2B. The high-frequency component image generating means 2B is composed of a high-pass filter that passes components of the third frequency FL3 or higher, and the frequency response becomes higher as the frequency becomes higher as shown in FIG. 14B. Accordingly, the frequency spectrum of the image D2B is obtained by removing the component corresponding to the region RL2 on the low frequency component side (frequency component lower than the third frequency FL3) from the frequency spectrum of the nonlinear processed image D2A as shown in FIG. It will be a thing. In other words, the nonlinear processing means 2A has an effect of generating a high frequency component corresponding to the region RH2 on the high frequency component side, and the high frequency component image generating means 2B has only the high frequency component generated by the nonlinear processing means 2A. There is an effect to take out. In the illustrated example, the third frequency FL3 is substantially equal to Fn / 2.

The above operations and effects will be described in more detail.
FIGS. 15A to 15C and FIGS. 16A to 16C are diagrams illustrating luminance signals obtained when sampling step edges.
FIG. 15A shows the step edge and the sampling interval S1, FIG. 15B shows the luminance signal obtained when the step edge is sampled at the sampling interval S1, and FIG. 15 (B) represents a high frequency component of the signal. On the other hand, FIG. 16A shows a sampling interval S2 wider than the step edge and the sampling interval S1, and FIG. 16B shows a signal obtained when the step edge is sampled at the sampling interval S2. FIG. 16C shows high frequency components of the signal shown in FIG. In the following description, it is assumed that the length of the sampling interval S2 is twice the length of the sampling interval S1.

  As shown in FIGS. 15C and 16C, the center of the step edge appears as a zero cross point Z in the signal representing the high frequency component. In addition, the slope of the signal representing the high frequency component near the zero cross point Z becomes steeper as the sampling interval is short, and the position of the point giving the local maximum value and minimum value near the zero cross point Z is also The shorter the sampling interval, the closer to the zero cross point Z.

  That is, even if the sampling interval changes, the position of the zero-cross point of the signal representing the high frequency component does not change in the vicinity of the edge, but the higher the frequency component in the vicinity of the edge, the smaller the sampling interval (or the higher the resolution). The slope becomes steep, and the position of the point giving the local maximum and minimum values approaches the zero cross point.

  FIGS. 17A to 17F are diagrams showing the operation and effect when the signal obtained by sampling the step edge at the sampling interval S2 is doubled and then input to the image processing apparatus according to the present invention. In particular, the operations and effects of the first intermediate image generation unit 1 and the second intermediate image generation unit 2 are shown. As described above, since the processes in the first intermediate image generation unit 1 and the second intermediate image generation unit 2 are performed in each of the horizontal direction and the vertical direction, the process is performed one-dimensionally. Accordingly, in FIGS. 17A to 17F, the contents of the processing are represented using a one-dimensional signal.

  FIG. 17A shows the luminance signal when the step edge is sampled at the sampling interval S2 as in FIG. FIG. 17B shows a signal obtained by enlarging the signal shown in FIG. That is, when the original image DORG includes an edge as shown in FIG. 17A, a signal as shown in FIG. 17B is input as the input luminance image YIN. Note that when the signal is doubled, the sampling interval becomes half that before the expansion, so the sampling interval of the signal shown in FIG. 17B is the same as the sampling interval S1 in FIGS. Become. In FIG. 17A, the position represented by the coordinate P3 is a boundary portion on the low luminance side (low level side) of the edge signal, and the position represented by the coordinate P4 is the high luminance side (high level) of the edge signal. Side).

  FIG. 17C shows a signal representing the high frequency component of the signal shown in FIG. 17B, that is, a signal corresponding to the image D1A output from the high frequency component image generating means 1A. Note that since the image D1A is obtained by extracting the high frequency component of the input luminance image YIN, the aliasing component is included in the image D1A.

  FIG. 17D shows a signal representing the low frequency component of the signal shown in FIG. 17C, that is, a signal corresponding to the image D1B output from the low frequency component image generating means 1B. Since the image D1B is output as the intermediate image D1 as described above, FIG. 17D corresponds to the intermediate image D1. As shown in FIG. 17D, the local minimum value in the vicinity of the zero cross point Z in the intermediate image D1 appears at the coordinate P3, and the local maximum value appears at the coordinate P4, as shown in FIG. The step edge coincides with the high frequency component extracted from the signal sampled at the sampling interval S2. Further, the aliasing component included in the image D1A is removed by a low-pass filter process performed by the low-frequency component image generation unit 1B.

  FIG. 17E shows an output signal when the signal shown in FIG. 17D is input to the nonlinear processing means 2A, that is, an image output from the nonlinear processing means 2A when the intermediate image D1 is input. D2A is represented. In the nonlinear processing means 2A, the signal values of the coordinates P1 and P2 before and after (adjacent in the front and rear) of the zero cross point Z are amplified. Accordingly, in the image D2A, as shown in FIG. 17E, the magnitude of the signal value at the coordinates P1 and P2 is larger than the other values, and the position where the local minimum value appears in the vicinity of the zero cross point Z is shown. The position where the local maximum value changes from the coordinate P3 to the coordinate P1 closer to the zero cross point Z changes from the coordinate P4 to the coordinate P2 closer to the zero cross point Z. This means that the high-frequency component is generated by the nonlinear processing of amplifying the values of the pixels before and after the zero cross point Z in the nonlinear processing means 2A. In this way, it is possible to generate a high-frequency component by adaptively changing the amplification factor for each pixel or appropriately changing the content of processing according to the pixel. That is, the nonlinear processing means 2A has an effect of generating a high frequency component not included in the intermediate image D1, that is, a high frequency component corresponding to the region RH2 on the high frequency component side shown in FIG. is there.

FIG. 17F shows a signal representing the high frequency component of the signal shown in FIG. 17E, that is, a signal corresponding to the image D2B output from the high frequency component image generating means 2B. The more accurate shape of the image D2B will be described later. As shown in FIG. 17F, the local minimum value (negative peak) in the vicinity of the zero-cross point Z in the image D2B is the maximum value ( The positive side peak) appears at the coordinate P2, and this state coincides with the high frequency component extracted from the signal obtained by sampling the step edge at the sampling interval S1, as shown in FIG. This means that the high frequency component generated in the nonlinear processing means 2A is taken out by the high frequency component image generating means 2B and output as an image D2B.
Further, it can be said that the extracted image D2B is a signal including a frequency component corresponding to the sampling interval S1. In other words, the high frequency component image generation means 2B has an effect of extracting only the high frequency component generated by the nonlinear processing means 2A.

  The above is the effect of the second intermediate image generation processing means 2. To summarize, the non-linear processing means 2A in the second intermediate image generation processing means 2 has a high frequency component corresponding to the region RH2 on the high frequency component side. The high frequency component image generation means 2B in the second intermediate image generation processing means 2 has the effect of extracting only the high frequency component generated by the nonlinear processing means 2A. Since the image D2B is output as the intermediate image D2, the second intermediate image generating means 2 can output the intermediate image D2 having a high frequency component corresponding to the sampling interval S1.

Here, it is possible to perform image enhancement processing by adding the intermediate image D1 and the intermediate image D2 to the input luminance image YIN.
In the present invention, the first and second intermediate images D1 and D2 are not added to the input image YIN, but the effects obtained when the first and second intermediate images are added will be described below. After that, the effect obtained by adding the third and fourth intermediate images D3M and D3H instead of the first and second intermediate images D1 and D2 will be described.

  First, the effect of adding the intermediate image D1 will be described. As described above, the intermediate image D1 is obtained by removing the aliasing component from the high frequency component of the input luminance image YIN, and corresponds to the high frequency component near the Nyquist frequency of the original image DORG as shown in FIG. is doing. As described with reference to FIG. 12D, the spectral intensity in the vicinity of the Nyquist frequency of the original picture DORG is weakened by the enlargement process in the image enlargement means U1, and therefore is weakened by the enlargement process by adding the intermediate image D1. Spectrum intensity can be compensated. Further, since the aliasing component is removed from the intermediate image D1, a false signal such as overshoot, jaggy, or ringing is not emphasized.

  Next, the effect of adding the intermediate image D2 will be described. As described above, the intermediate image D2 is a high-frequency component corresponding to the sampling interval S1. Therefore, by adding the image D2, it is possible to give a high frequency component in a band equal to or higher than the Nyquist frequency of the original image DORG, and therefore, it is possible to increase the resolution of the image.

  In summary, by adding the intermediate image D1 and the image D2 to the input luminance image YIN, it is possible to add a high frequency component without enhancing the aliasing component, and it is possible to improve the resolution of the image.

  By the way, it is possible to increase the sharpness of the image and improve the image quality by adding the high frequency component generated as described above to the input image. However, if the high frequency component is excessively added, On the contrary, the image quality may be degraded.

  FIGS. 18A and 18B are diagrams for explaining the deterioration of image quality due to the addition of high frequency components. FIG. 18A shows the sharpness of an image by appropriately performing the addition of high frequency components. FIG. 18B shows a case where image quality is deteriorated as a result of excessive addition of high frequency components.

  FIG. 18A shows the result of adding the intermediate image D1 shown in FIG. 17D and the intermediate image D2 shown in FIG. 17F to the input luminance image YIN shown in FIG. The boundary portion on the low luminance side of the step edge represented by the coordinate P3 in FIG. 17A is corrected to the position represented by the coordinate P1 in FIG. 18A, and FIG. The boundary portion on the high luminance side of the edge signal represented by the coordinate P4 in A) is corrected to the position represented by the coordinate P2 in FIG. 18A, and as a result, FIG. 17A and FIG. Comparing B), it can be seen that FIG. 18A is closer to the step edge shown in FIG. This represents that the step edge was reproduced by appropriately performing the addition of the high frequency component, and the sharpness of the image was increased.

  18B also adds the intermediate image D1 shown in FIG. 17D and the intermediate image D2 shown in FIG. 17F to the input luminance image YIN shown in FIG. 17B. Unlike the case of FIG. 18A, it shows a case where high frequency components are added excessively. Compared with FIG. 18A, the luminance at the positions represented by the coordinates P1 and P3 is unnaturally lower than the surroundings (undershoot), or the luminance at the positions represented by the coordinates P2 and P4 is the surroundings. It can be seen that the image quality is degraded due to an unnatural increase (overshoot). Overshoot occurs when the sign of the pixel value of the intermediate image D1 or intermediate image D2 is positive, and undershoot occurs when the sign of the pixel value of the intermediate image D1 or intermediate image D2 is negative. is there.

In the following, a problem when an overshoot occurs and a problem when an undershoot occur are considered separately, and how to prevent them according to the present invention will be described.
First, overshoot will be described.

  When overshoot occurs in the input luminance image YIN, the luminance signal becomes larger than necessary. If the value of the luminance signal (Y) is increased from the equation (3), the first term on the right side of each equation representing R, G, B is increased when converted to the RGB format, so that R, G, It can be seen that both B values are large.

  Large values for R, G, and B mean that the color approaches white. In other words, getting closer to white means that the color becomes lighter. Originally, it is relatively inconspicuous even if the color is lightened in a portion close to an achromatic color, but if the color is lightened near the edge of a chromatic color, the color is lightened only around the edge, giving an unnatural feeling.

  In other words, when the sign of the pixel value of the intermediate image D1 or the intermediate image D2 is positive, the magnitude of luminance (hereinafter referred to as a correction amount) added to the chromatic portion by the intermediate image D1 or the intermediate image D2 is more than necessary. If it is increased, it is considered that the deterioration of the image quality is likely to occur. Therefore, it is considered that adjustment should be made so that the correction amount by the intermediate image D1 and the intermediate image D2 does not become larger than necessary in the chromatic color portion.

  As the method, for example, when the sign of the pixel value of the intermediate image D1 or the intermediate image D2 is positive, a portion having a chromatic color and a correction amount given by the intermediate image D1 or the intermediate image D2 is detected. A method is conceivable in which the correction amount is not increased more than necessary by appropriately applying a gain so that the correction amount by the intermediate image D1 or the intermediate image D2 is reduced.

  Whether it is a chromatic color or not can be determined by saturation (which can be expressed by the square root of the sum of squares of Cr and Cb). That is, the saturation is a large value in the chromatic color. The square root of the sum of squares of Cr and Cb can be approximated by the sum of absolute values of Cr and Cb. This is because as the absolute value of Cr or Cb increases, the square root of the square sum of Cr and Cb also increases. Further, since it is easier to calculate the sum of absolute values than to calculate the square root of the sum of squares, the circuit scale can be reduced.

  Whether or not the correction amount given by the intermediate image D1 or the intermediate image D2 becomes large can be determined to some extent based on the pixel value of the input luminance image YIN. The reason is described below.

  The intermediate image D1 is generated by performing high-pass filter processing on the input luminance image YIN and then performing low-pass filter processing. The low-pass filter process is the same as obtaining a local average value of input data. Therefore, if the output value of the high-pass filter process is a large positive value, the output value of the low-pass filter process is likely to be a large positive value, and the correction amount given by the intermediate image D1 tends to be a large value.

  On the other hand, the high-pass filter process corresponds to subtracting a local average value from each pixel value of the input luminance image YIN. Therefore, if the pixel value of the pixel of interest in the input luminance image YIN has a large value and the pixel value of the surrounding pixels is small, the output value after high-pass filter processing given to that pixel is also a large positive value. Value.

  Conversely, if the pixel value of the pixel of interest in the input luminance image YIN has a small value, the output value after the high-pass filter process does not become a large positive value.

  Therefore, if the pixel value of the pixel of interest in the input luminance image YIN is large, the output value after high-pass filter processing given to the pixel is likely to be a large positive value.

  On the other hand, the low-pass filter process is the same as obtaining a local average value of input data. Therefore, if the output value of the high-pass filter process is a large positive value, the output value of the low-pass filter process is likely to be a large positive value.

  In summary, when the pixel value of the pixel of interest in the input luminance image YIN is large, the correction amount given by the intermediate image D1 tends to be a large value.

  The intermediate image D2 is obtained by performing non-linear processing on the intermediate image D1 by the non-linear processing means 2A and then performing high-pass filter processing in the high-frequency component image generating means 2B. Since the non-linear processing means 2A amplifies the intermediate image D1 only in the vicinity of the zero cross point, basically, if the intermediate image D1 has a large positive value, the image D2A output from the non-linear processing means 2A also has a large positive value. It is thought that. When the image D2A has a large positive value, there is a high possibility that the intermediate image D2 that is the result of the high-pass filter processing for the image D2A also has a large positive value.

  In summary, when the pixel value of the input luminance image YIN is large, the pixel values of the intermediate image D1 and the intermediate image D2 are likely to be large positive values. In other words, when the pixel value of the input luminance image YIN is large, it can be determined to some extent that the correction amount given by the intermediate image D1 or the intermediate image D2 increases.

  For the reasons described above, the correction amount is large and the RGB format is increased in a pixel having any one of the pixel value of the input luminance image YIN, the absolute value of the pixel value of the input CR image CRIN, and the absolute value of the pixel value of the input CB image CBIN It is thought that the problem that the color becomes light when it is corrected.

  When any one of the pixel value of the input luminance image YIN, the absolute value of the pixel value of the input CR image CRIN, and the absolute value of the pixel value of the input CB image CBIN is large, the weighted addition value is also a large value.

  Therefore, the present invention weights and adds the pixel value of the input luminance image YIN, the absolute value of the pixel value of the input CR image CRIN, and the absolute value of the pixel value of the input CB image CBIN according to the equation (1) to obtain the luminance / color difference addition image YC. If the sign of the pixel values of the intermediate image D1 and the intermediate image D2 is positive, the amplification factor applied to the intermediate image D1 and the intermediate image D2 decreases as the pixel value of the luminance / color difference addition image YC increases. This is to avoid the problem that the color becomes light in the vicinity of the coloring edge.

と表される。 Such a relationship between the amplification factor (GAIN) and the pixel value (L) of the luminance color difference addition image YC is, for example,

It is expressed.

  FIG. 19A is a diagram showing the relationship between the amplification factor (GAIN) represented by Expression (4) and the pixel value (L) of the luminance / color difference addition image YC. When the pixel value of the luminance / chrominance-added image YC is 0, it takes a predetermined value pB, decreases between 0 and a certain value pA1 with a slope pk1, and between the pixel value pA1 and a certain value pA2. Decreases at a slope pk2, and decreases at a slope pk3 when the pixel value is greater than or equal to pA2.

  In short, the amplification factor D3MA or the amplification factor D3HA is based on a monotonically decreasing function in which the amplification factor decreases as the pixel value of the luminance / color difference addition image YC increases, as shown in FIG. 19A or Equation (4). By determining, it is possible to perform processing for preventing a problem that the color becomes light near the edge of the chromatic color.

Next, the undershoot will be described.
When undershoot occurs in the input luminance image YIN, the luminance signal becomes smaller than necessary. If the value of the luminance signal (Y) is reduced from the equation (3), the first term on the right side of each equation representing R, G, B is reduced when converted to the RGB format, and as a result, R, G, It can be seen that both B values are small.

  When R, G, and B are small values, it means that the color approaches black. When the color becomes black near the edge, a black false contour appears that borders the edge, giving an unnatural feeling.

  In other words, when the sign of the pixel value of the intermediate image D1 or the intermediate image D2 is negative, the magnitude of luminance (hereinafter referred to as a correction amount) subtracted by the intermediate image D1 or the intermediate image D2 becomes a value larger than necessary. It is considered that these deteriorations in image quality are likely to occur. Therefore, when the sign of the pixel value of the intermediate image D1 or the intermediate image D2 is negative, it may be necessary to adjust so that the correction amount does not become larger than necessary.

  For example, when the sign of the pixel value of the intermediate image D1 or the intermediate image D2 is negative, a portion where the correction amount is increased is detected, and the correction amount by the intermediate image D1 or the intermediate image D2 is reduced at the detected location. A method of preventing the correction amount from becoming unnecessarily large by appropriately applying a gain can be considered.

  Whether the correction amount given by the intermediate image D1 or the intermediate image D2 becomes large (becomes a negative value having a large absolute value) can be determined to some extent based on the pixel value of the input luminance image YIN. The reason is described below.

  The intermediate image D1 is generated by performing high-pass filter processing on the input luminance image YIN and then performing low-pass filter processing. Here, the high-pass filter processing corresponds to subtracting a local average value from each pixel value of the input luminance image YIN. Therefore, if the pixel value of the pixel of interest in the input luminance image YIN is small, the output value after high-pass filter processing given to that pixel is likely to be a negative value having a large absolute value.

  On the other hand, the low-pass filter process is the same as obtaining a local average value of input data. Therefore, if the output value of the high-pass filter process is a negative value having a large absolute value, the output value of the low-pass filter process is likely to be a negative value having a large absolute value.

  The intermediate image D2 is obtained by performing non-linear processing on the intermediate image D1 by the non-linear processing means 2A and then performing high-pass filter processing in the high-frequency component image generating means 2B. Since the non-linear processing means 2A amplifies the intermediate image D1 only in the vicinity of the zero cross point, basically, if the intermediate image D1 has a negative value with a large absolute value, the image D2A output from the non-linear processing means 2A also has an absolute value. It is considered to have a large negative value. When the image D2A has a negative value with a large absolute value, there is a high possibility that the intermediate image D2 that is the result of the high-pass filter processing for the image D2A also has a negative value with a large absolute value.

  In summary, when the pixel value of the input luminance image YIN is small, the pixel values of the intermediate image D1 and the intermediate image D2 are likely to be small negative values. In other words, when the pixel value of the input luminance image YIN is small, it can be determined to some extent that the correction amount given by the intermediate image D1 or the intermediate image D2 becomes a large value.

  For the reasons described above, it is considered that the pixel having a small pixel value of the input luminance image YIN has a large correction amount, and may be black when converted to the RGB format.

  In the present invention, when the sign of the pixel value of the intermediate image D1 or the intermediate image D2 is negative, the intermediate color D1 or the intermediate image D2 is reduced as the pixel value of the luminance color difference addition image YC decreases instead of the pixel value of the input luminance image YIN. By reducing the applied amplification factor, the problem that a false contour appears near the edge is avoided. (The reason why the luminance color difference addition image YC may be used instead of the input luminance image YIN will be described later.)

と表される。 Such a relationship between the amplification factor (GAIN) and the pixel value (L) of the luminance color difference addition image YC is, for example,

It is expressed.

  FIG. 19B is a diagram illustrating a relationship between the amplification factor (GAIN) expressed by the equation (5) and the pixel value (L) of the luminance / color difference addition image YC. When the pixel value of the luminance / chrominance-added image YC is 0, a certain predetermined value mB is taken, and the pixel value increases from 0 to a certain value mA1 with a slope mk1, and the pixel value ranges from mA1 to a certain value mA2. Increases at a slope mk2, and increases at a slope mk3 when the pixel value is greater than or equal to mA2.

  By performing the processing as described above, it is possible to prevent the problem that the color approaches black in the vicinity of the achromatic edge. Because, in the case of an achromatic color, the absolute value of the pixel value of the input CR image CRIN and the absolute value of the pixel value of the input CB image CBIN are close to zero, the magnitude relationship between the pixel values of the luminance added image YC remains as it is. This is because it is considered that it represents the magnitude relationship of the pixel values. Therefore, it is considered that there is no problem in using the luminance color difference addition image YC instead of the input luminance image YIN in the vicinity of the achromatic edge.

  On the other hand, in the vicinity of the edge of the chromatic color, the absolute value of the pixel value of the input CR image CRIN and the absolute value of the pixel value of the input CB image CBIN can take a large value. Does not necessarily represent the magnitude relationship between the pixel values of the input luminance image YIN. For example, even if the pixel value of the input luminance image YIN is small, if the absolute value of the pixel value of the input CR image CRIN or the pixel value of the input CB image CBIN is large, the pixel value of the input luminance image YIN is small but the luminance color difference addition image YC. The pixel value of becomes a large value.

  However, since the natural image often has a small luminance and the absolute value of the color difference is also small, the absolute value of the pixel value of the input CR image CRIN or the pixel value of the input CB image CBIN even if the pixel value of the input luminance image YIN is small. It is rare if is large. Therefore, it is considered that there are practically no problems in controlling the amplification factor for the intermediate image D1 and the intermediate image D2 using the pixel value of the luminance / color difference addition image YC instead of the pixel value of the input luminance image YIN as described above. It is done.

  In short, the amplification factor D3MA or the amplification factor D3HA is based on a monotonically increasing function in which the amplification factor increases as the pixel value of the luminance color difference addition image YC increases as shown in FIG. 19B or Equation (5). By determining the above, it becomes possible to perform processing for preventing a problem that the color approaches black in the vicinity of the edge.

  As described above, in the image processing apparatus according to the first embodiment of the present invention, it is possible to perform the image enhancement processing while suppressing the occurrence of a problem that the color becomes light or near black at the edge. When the color becomes light or close to black near the edge, it feels unnatural in terms of visual characteristics. Therefore, the image processing apparatus according to Embodiment 1 of the present invention is also very favorable in terms of visual characteristics.

  In the image processing apparatus according to the first embodiment of the present invention, the first intermediate image processing unit 3M and the second intermediate image processing unit 3H determine the amplification factors for the intermediate image D1 and the intermediate image D2. The information required at this time includes the pixel value of the input luminance image YIN, the absolute value of the pixel value of the input CR image CRIN, and the weighted addition value of the absolute value of the pixel value of the input CB image CBIN and the intermediate image D1 or intermediate image D2. This is only the sign of the pixel value. Therefore, the amplification factor can be determined with a simple circuit, and the increase in circuit scale can be reduced.

を行える演算手段と、演算手段に対して与える係数ｋ１、ｋ２、ｋ３、閾値Ａ１、Ａ２および数値Ｂの値を中間画像Ｄ１もしくは中間画像Ｄ２の画素値の符号に応じて変化させる係数切替手段で、増幅率決定手段３ＭＡもしくは増幅率決定手段３ＭＢを構成することができる。
即ち、切替手段により、画素値の符号が正のときは、−ｐｋ１、−ｐｋ２、−ｐｋ３、ｐＡ１、ｐＡ２、ｐＢを、画素値の符号が負のときは、ｍｋ１、ｍｋ２、ｍｋ３、ｍＡ１、ｍＡ２、ｍＢを選択し、それぞれｋ１、ｋ２、ｋ３、Ａ１、Ａ２、Ｂとして用いて、式（６）による演算を行うこととすれば良い。 For example, when Equation (4) and Equation (5) are compared, they differ only in parameters such as coefficients and thresholds used therein. That is, the following calculation expressed using coefficients k1, k2, k3, threshold values A1, A2 and numerical value B

And a coefficient switching means for changing the values of the coefficients k1, k2, k3, thresholds A1, A2 and numerical value B given to the calculation means according to the sign of the pixel value of the intermediate image D1 or the intermediate image D2. The amplification factor determining means 3MA or the amplification factor determining means 3MB can be configured.
That is, when the sign of the pixel value is positive by the switching means, -pk1, -pk2, -pk3, pA1, pA2, pB, and when the sign of the pixel value is negative, mk1, mk2, mk3, mA1, It is only necessary to select mA2 and mB and use them as k1, k2, k3, A1, A2, and B, respectively, to perform the calculation according to equation (6).

  In the above embodiment, the first intermediate image processing unit 3M determines the amplification factor D3MA based on the output YC of the luminance / color difference addition unit 5 and the code sD1 of the pixel value of the first intermediate image D1. In the second intermediate image processing means 3H, the amplification factor D3HA is determined based on the output YC of the luminance / color difference addition means 5 and the sign sD2 of the pixel value of the second intermediate image D2. Only in one of the image processing means 3M and the second intermediate image processing means 3H, the amplification factor may be determined by the above method, and in the other, the amplification factor may be determined by another method.

  Furthermore, in the image processing apparatus according to the present invention, the first intermediate image generation unit 1 and the second intermediate image generation unit 2 perform the processing relating to the horizontal direction and the processing relating to the vertical direction of the image in parallel. The above-described effects can be obtained not only in the horizontal direction or in the vertical direction but also in any direction.

  In the image processing apparatus according to the present invention, the input luminance image YIN has a frequency band from the origin to Fn in the vicinity of the Nyquist frequency ± Fn / 2 of the original image DORG (or a specific frequency band) in the frequency space. The image D2B corresponding to the high frequency component in the vicinity of the Nyquist frequency ± Fn of the input luminance image YIN is generated based on the components that are present. Therefore, for some reason, even if the frequency component near the Nyquist frequency ± Fn is lost in the input luminance image YIN (or the input image IMGIN), the frequency component near the Nyquist frequency ± Fn is given by the image D2B. Is possible. In other words, since a higher frequency component side frequency component is given to the input luminance image YIN, the resolution of the output luminance image YOUT (or the output image IMGOUT) can be increased.

  In addition, the location used as a specific frequency band is not limited to the vicinity of ± Fn / 2. That is, the frequency band to be used can be changed by appropriately changing the frequency response of the high-frequency component image generating unit 1A and the low-frequency component image generating unit 1B.

  In the above description, the image enlargement process is given as an example in which the frequency component near the Nyquist frequency Fn is lost. However, the cause of the loss of the frequency component near the Nyquist frequency Fn with respect to the input luminance image YIN is not limited thereto. In addition, noise removal processing can be considered. Therefore, the application of the image processing apparatus in the present invention is not limited after the image enlargement process.

  In addition, the relationship between the amplification factor determined by the first intermediate image processing unit 3M and the second intermediate image processing unit 3H and the pixel value of the luminance / color difference addition image YC is not limited to that described in the present embodiment. When the sign of the image D1 or D2 is positive, it is sufficient that the amplification factor decreases as the pixel value of the luminance / color difference addition image YC increases. When the sign of the intermediate image D1 or D2 is negative, the luminance / color difference addition image YC It suffices if the amplification factor decreases as the pixel value decreases.

Embodiment 2. FIG.
In the first embodiment, the present invention has been described as being realized by hardware, but part or all of the configuration shown in FIG. 1 can also be realized by software. Processing in that case will be described with reference to FIG. 20 and FIGS.

  FIG. 20 shows an image processing apparatus according to the second embodiment. The illustrated image processing apparatus includes a CPU 11, a program memory 12, a data memory 13, a first interface 14, a second interface 15, and a bus 16 that connects them. For example, the image processing apparatus illustrated in FIG. Instead of the processing device, for example, it can be used as the image processing unit U1 in the display device shown in FIG.

The CPU 11 operates according to a program stored in the program memory 12. Various data are stored in the data memory 13 in the course of operation. For example, the image IMGU1 output from the color image enlarging means U1 shown in FIG. 9 is supplied as the input image IMGIIN through the interface 14, and the CPU 11 performs the same processing as the image processing apparatus in FIG. The generated output image IMGOUT is supplied as an image DU2 to the monitor U3 in the image display device shown in FIG. 9 through the interface 15, and used for display by the monitor 9.
Hereinafter, processing performed by the CPU 11 will be described with reference to FIGS.

  FIG. 21 is a diagram illustrating a flow of an image processing method performed by the image processing apparatus of FIG. 20, and the illustrated image processing method includes a luminance / color difference signal addition step ST0, a first intermediate image generation step ST1, and a second step. Intermediate image generation step ST2, first intermediate image processing step ST3M, second intermediate image processing step ST3H, and addition step ST4.

  Note that the image processing method according to the second embodiment also performs image processing on the input image IMGIN input in the YCbCr format as in the first embodiment. That is, the input image IMGIN input in an image input step (not shown) is a color image, and includes a signal YIN representing a luminance component (hereinafter referred to as an input luminance image YIN) and signals CRIN and CBIN representing color difference components. A signal CRIN (hereinafter referred to as an input CR image CRIN) represents a Cr component of the color difference components, and a signal CBIN (hereinafter referred to as an input CB image CBIN) represents a Cb component of the color difference components.

  As shown in FIG. 22, the first intermediate image generation step ST1 includes a high frequency component image generation step ST1A and a low frequency component image generation step ST1B.

  The high frequency component image generation step ST1A includes a horizontal direction high frequency component image generation step ST1Ah and a vertical direction high frequency component image generation step ST1Av. The low frequency component image generation step ST1B includes a horizontal direction low frequency component image generation step ST1Bh, And a vertical high-frequency nest component image ST1Bv.

  As shown in FIG. 23, the second intermediate image generation step ST2 includes a non-linear processing step ST2A and a high frequency component image generation step ST2B.

  The non-linear processing step ST2A includes a horizontal non-linear processing step ST2Ah and a vertical non-linear processing step ST2Av, and the high frequency component image generation step ST2B includes a horizontal high frequency component passing step ST2Bh and a vertical high frequency component passing step ST2Bv. including.

  As shown in FIG. 24, the horizontal direction nonlinear processing step ST2Ah includes a zero cross determination step ST311h and a signal amplification step ST312h. The vertical direction nonlinear processing step ST2Av includes a zero cross determination step ST311v and a signal as shown in FIG. An amplification step ST312v is included.

  As shown in FIG. 26, the first intermediate image processing step ST3M includes an amplification factor determining step ST3MA and a pixel value changing step ST3MB.

  As shown in FIG. 27, the second intermediate image processing step ST3H includes an amplification factor determining step ST3HA and a pixel value changing step ST3HB.

  First, the operation of the luminance / color difference signal addition step ST0 will be described. In the luminance color difference addition step ST0, the pixel value of the input luminance image YIN, the absolute value of the pixel value of the input CR image CRIN, and the absolute value of the pixel value of the input CB image CBIN are weighted and added for each pixel, and the luminance color difference addition image YC is obtained. Generate. The relationship between the luminance color difference addition image YC, the input luminance image YIN, the input CR image CRIN, and the input CB image CBIN is expressed by Expression (1). This operation is equivalent to the luminance color difference adding means 5.

Next, the operation of the first intermediate image generation step ST1 will be described according to the flow of FIG.
In the high frequency component image generation step ST1A, the following processing is performed on the input luminance image YIN.
First, in the horizontal high-frequency component image generation step ST1Ah, an image D1Ah obtained by extracting the high-frequency component in the horizontal direction from the input luminance image YIN is generated by the high-pass filter processing in the horizontal direction.
In the vertical high-frequency component image step ST1Av, an image D1Av obtained by extracting the high-frequency component in the vertical direction from the input luminance image YIN is generated by high-pass filtering in the vertical direction.
That is, the high frequency component image generation step ST1A performs the same processing as the high frequency component image generation unit 1A, and generates an image D1A composed of the image D1Ah and the image D1Av from the input luminance image YIN. This operation is equivalent to the high frequency component image generating means 1A.

In the low frequency component image generation step ST1B, the following processing is performed on the image D1A. First, in the horizontal direction low frequency component image generation step ST1Bh, an image D1Bh obtained by extracting a horizontal low frequency component from the image D1Ah is generated by a horizontal low-pass filter process.
In the vertical direction low frequency component image generation step ST1Bv, an image D1Bv obtained by extracting the low frequency component in the vertical direction from the image D1Av is generated by the low pass filter processing in the vertical direction.
That is, the low frequency component image generation step ST1B performs the same processing as the low frequency component image generation means 1B, and generates an image D1B composed of the image D1Bh and the image D1Bv from the image D1A. This operation is equivalent to the low frequency component image generation means 1B.

  The above is the operation of the first intermediate image generation step ST1, and the first intermediate image generation step ST1 outputs the image D1Bh as the image D1h, the image D1Bv as the image D1v, and the intermediate image D1 composed of the image D1h and the image D1v. To do. The above operation is the same as that of the first intermediate image generating means 1.

Next, the operation of the second intermediate image generation step ST2 will be described according to the flow of FIGS.
First, in the nonlinear processing step ST2A, the following processing is performed on the intermediate image D1.

  First, in the horizontal non-linear processing step ST2Ah, an image D2Ah is generated from the image D1h by processing according to the flow shown in FIG. The processing in the flow shown in FIG. 24 is as follows. First, in the zero cross determination step ST311h, a change in pixel value in the image D1h is confirmed along the horizontal direction. Then, a point where the pixel value changes from a positive value to a negative value or from a negative value to a positive value is regarded as a zero cross point, and the pixels located to the left and right of the zero cross point are notified to the signal amplification step ST312h. In the signal amplification step ST312h, for the image D1h, the pixel value of the pixel notified to be positioned on the left and right of the zero cross point is amplified, and the image is output as the image D2Ah. That is, the nonlinear processing step ST2Ah performs the same process as the horizontal nonlinear processing means 2Ah on the image D1h to generate the image D2Ah.

  Next, in the vertical nonlinear processing step ST2Av, an image D2Av is generated from the image D1v by processing according to the flow shown in FIG. The processing in the flow shown in FIG. 25 is as follows. First, in the zero cross determination step ST311v, a change in pixel value in the image D1v is confirmed along the vertical direction. A portion where the pixel value changes from a positive value to a negative value or from a negative value to a positive value is regarded as a zero cross point, and the pixels located above and below the zero cross point are notified to the signal amplification step ST312v. In the signal amplification step ST312v, the pixel value of the pixel notified to be positioned above and below the zero cross point is amplified for the image D1v, and the image is output as the image D2Av. That is, in the nonlinear processing step ST2Av, the image D1v is subjected to the same processing as the vertical nonlinear processing means 2Av to generate the image D2Av.

  The above is the operation of the non-linear processing step ST2A, and the non-linear processing step ST2A generates an image D2A composed of the image D2Ah and the image D2Av. The operation is equivalent to the nonlinear processing means 2A.

Next, in the high frequency component image generation step ST2B, the following processing is performed on the image D2A.
First, in the horizontal high-frequency component image generation step ST2Bh, an image D2Bh is generated by performing a high-pass filter process in the horizontal direction on the image D2Ah. That is, the horizontal direction high frequency component image generation step ST2Bh performs the same processing as the horizontal direction high frequency component image generation means 2Bh.

  Next, in the vertical direction high-frequency component image generation step ST2Bv, an image D2Bv obtained by performing vertical high-pass filter processing on the image D2Av is generated. That is, the vertical high frequency component image generation step ST2Bv performs the same processing as the vertical high frequency component image generation means 2Bv.

  The above is the operation of the high frequency component image generation step ST2B, and the high frequency component image generation step ST2B generates an image D2B composed of the image D2Bh and the image D2Bv. The operation is the same as that of the high frequency component image generating means 2B.

  The above is the operation of the second intermediate image generation step ST2, and the second intermediate image generation step ST2 outputs the image D2B as the intermediate image D2. That is, an intermediate image D2 is output in which the image D2Bh is the image D2h and the image D2Bv is the image D2v. This operation is equivalent to the second intermediate image generating means 2.

Next, the operation of the first intermediate image processing step ST3M will be described according to the flow of FIG.
First, in the first intermediate image processing step ST3M, the amplification factor for the pixel value of each pixel of the intermediate image D1 is determined in the amplification factor determination step ST3MA. Here, since the intermediate image D1 includes the image D1h and the image D1v, the amplification factor is determined for each pixel of the image D1h and the image D1v. That is, for the image D1h, the amplification factor for each pixel is determined in the horizontal direction amplification factor determination step ST3MAh, and for the image D1v, the amplification factor for each pixel is determined in the vertical direction amplification factor determination step ST3MAv. Here, the operation of the horizontal direction gain determination step ST3MAh is the same as that of the horizontal direction gain determination unit 3MAh, and the operation of the vertical direction gain determination step ST3MAv is the same as that of the vertical direction gain determination unit 3MAv.

  Next, in the pixel value changing step ST3MB, the pixel value of each pixel of the intermediate image D1 is amplified based on the amplification factor determined in the amplification factor determination step ST3MA. Here, since the intermediate image D1 includes the image D1h and the image D1v, the amplification of the pixel value is performed for each of the image D1h and the image D1v. That is, each pixel value of the image D1h is amplified based on the amplification factor determined in the horizontal direction amplification factor determination step ST3MAh, and an image D3MBh is generated. Further, each pixel value of the image D1v is amplified based on the amplification factor determined in the vertical direction amplification factor determination step ST3MAv, and an image D3MBv is generated. This operation is the same as that of the pixel value changing means 3MB.

  Then, an intermediate image D3M composed of an image D3Mh corresponding to the image D3MBh and an image D3Mv corresponding to the image D3MBv is generated by the first intermediate image processing step ST3M. The above is the operation of the first intermediate image processing step ST3M, and the operation is the same as that of the first intermediate image processing means 3M.

Next, the operation of the second intermediate image processing step ST3H will be described according to the flow of FIG.
First, in the second intermediate image processing step ST3H, in the amplification factor determination step ST3HA, the amplification factor for the pixel value of each pixel of the intermediate image D2 is determined. Here, since the intermediate image D2 includes the image D2h and the image D2v, the amplification factor is determined for each pixel of the image D2h and the image D2v. That is, for the image D2h, the amplification factor for each pixel is determined in the horizontal direction amplification factor determination step ST3HAh, and for the image D2v, the amplification factor for each pixel is determined in the vertical direction amplification factor determination step ST3HAv. Here, the operation of the horizontal direction gain determination step ST3HAh is the same as that of the horizontal direction gain determination unit 3HAh, and the operation of the vertical direction gain determination step ST3HAv is the same as that of the vertical direction gain determination unit 3HAv.

  Next, in the pixel value changing step ST3HB, the pixel value of each pixel of the intermediate image D2 is amplified based on the amplification factor determined in the amplification factor determination step ST3HA. Here, since the intermediate image D2 includes the image D2h and the image D2v, the amplification of the pixel value is performed for each of the image D2h and the image D2v. That is, each pixel value of the image D2h is amplified based on the amplification factor determined in the horizontal direction amplification factor determination step ST3HAh, and an image D3HBh is generated. Also, each pixel value of the image D2v is amplified based on the amplification factor determined in the vertical direction amplification factor determination step ST3HAv, and an image D3HBv is generated. This operation is the same as that of the pixel value changing means 3HB.

  Then, an intermediate image D3H composed of an image D3Hh corresponding to the image D3HBh and an image D3Hv corresponding to the image D3HBv is generated by the second intermediate image processing step ST3H. The above is the operation of the second intermediate image processing step ST3H, and the operation is the same as that of the second intermediate image processing means 3H.

In the addition step ST4, the input luminance image YIN, the intermediate image D3M, and the intermediate image D3H are added to generate an output luminance image YOUT. Since the intermediate image D3M is composed of the image D3Mh and the image D3Mv, and the intermediate image D3H is composed of the image D3Hh and the image D3Hv, all of the images D3Mh, D3Hv, D3Mh, and D3Hv are added to the input luminance image YIN in the addition step ST4. At this time, the images D3Mh, D3Hv, D3Mh, and D3Hv may be simply added to the input luminance image YIN or may be weighted. The output luminance image YOUT is output as the final output image of the image processing method according to the present invention. The above is the operation of the adding step ST4, and this operation is equivalent to the operation of the adding means 4.
The above is the operation of the image processing method according to the present invention.

  As is apparent from the description, the operation of the image processing method according to the present invention is equivalent to that of the image processing apparatus according to the first embodiment of the present invention. Therefore, the image processing method according to the present invention has the same effect as the image processing apparatus according to the first embodiment of the present invention. Further, in the image display device shown in FIG. 9, for example, the image processing method is performed inside the image processing device U2, so that the image processed by the image processing method is displayed on the image display device shown in FIG. You can also

  DESCRIPTION OF SYMBOLS 1 1st intermediate image generation means, 2 2nd intermediate image generation means, 3M 1st intermediate image processing means, 3H 2nd intermediate image processing means, 4 Addition means, 5 Luminance color difference addition means, IMGIN input image, YIN input luminance image, CRIN input CR image, CBIN input CB image, D1 intermediate image, D2 intermediate image, D3M intermediate image, D3H intermediate image, IMGOUT output image, YOUT output luminance image, CROUT output CR image, CBOUT output CB image.

Claims (9)

First intermediate image generation means for generating a first intermediate image obtained by extracting a component of a specific frequency band from a luminance signal of an input image composed of a luminance signal and a color difference signal;
Second intermediate image generation means for generating a second intermediate image based on the first intermediate image;
Luminance color difference addition means for weighted addition of absolute values of the luminance signal and color difference signal of the input image;
First intermediate image processing means for generating a third intermediate image obtained by amplifying the pixel value of the first intermediate image according to a first amplification factor;
Second intermediate image processing means for generating a fourth intermediate image obtained by amplifying the pixel value of the second intermediate image according to a second amplification factor;
In an image processing apparatus having addition means for adding the luminance signal of the input image, the third intermediate image, and the fourth intermediate image,
The second intermediate image generating means includes
A zero-cross determining means for capturing a point where the pixel value of the first intermediate image changes from a positive value to a negative value or from a negative value to a positive value as a zero-cross point;
Signal amplifying means for generating a nonlinear processed image obtained by amplifying a pixel value of a pixel in the vicinity of the zero-cross point among the pixels constituting the first intermediate image at an amplification factor greater than 1,
High-frequency component image generation means for generating a high-frequency component image obtained by extracting only the high-frequency component of the nonlinear processed image;
Outputting the high-frequency component image as the second intermediate image;
The first amplification factor is:
If the sign of the pixel value of the first intermediate image is positive and the output value of the luminance color difference adding means is large, it becomes a small value,
When the sign of the pixel value of the first intermediate image is negative and the output value of the luminance color difference adding means is small, it is determined to be a small value,
The second amplification factor is
If the sign of the pixel value of the second intermediate image is positive and the output value of the luminance color difference adding means is large, it becomes a small value,
The image processing apparatus, wherein when the sign of the pixel value of the second intermediate image is negative and the output value of the luminance / color difference adding means is small, it is determined to be a small value. The first amplification factor is:
If the sign of the pixel value of the first intermediate image is positive and the output value of the luminance / color difference adding means is larger than a first predetermined value, the output value of the luminance / color difference adding means decreases as the output value increases,
When the sign of the pixel value of the first intermediate image is negative and the output value of the luminance color difference adding means is smaller than a second predetermined value, it is determined to decrease as the output value of the luminance color difference adding means decreases. ,
The second amplification factor is
If the sign of the pixel value of the second intermediate image is positive and the output value of the luminance / color difference adding means is greater than a third predetermined value, the output value of the luminance / color difference adding means decreases as the output value increases,
When the sign of the pixel value of the second intermediate image is negative and the output value of the luminance color difference adding means is smaller than a fourth predetermined value, it is determined to decrease as the output value of the luminance color difference adding means decreases. The image processing apparatus according to claim 1, wherein: The first amplification factor is:
When the sign of the pixel value of the first intermediate image is positive and the output value of the luminance / color difference adding means is larger than the fifth predetermined value and smaller than the sixth predetermined value, the output value of the luminance / color difference adding means increases. Decreases as you do,
When the sign of the pixel value of the first intermediate image is negative and the output value of the luminance / color difference adding means is larger than the seventh predetermined value and smaller than the eighth predetermined value, the output value of the luminance / color difference adding means decreases. Decided to decrease with time,
The second amplification factor is
When the sign of the pixel value of the second intermediate image is positive and the output value of the luminance / color difference adding means is larger than the ninth predetermined value and smaller than the tenth predetermined value, the output value of the luminance / color difference adding means increases. Decreases as you do,
When the sign of the pixel value of the second intermediate image is negative and the output value of the luminance color difference adding means is larger than the eleventh predetermined value and smaller than the twelfth predetermined value, the output value of the luminance color difference adding means decreases. The image processing apparatus according to claim 1, wherein the image processing apparatus is determined to decrease as the time increases.   The image processing apparatus according to claim 1, wherein the adding unit weights and adds the luminance signal of the input image, the third intermediate image, and the fourth intermediate image. The first intermediate image generating means includes
First horizontal high-frequency component image generation means for generating a first horizontal high-frequency component image obtained by extracting a high-frequency component equal to or higher than the first horizontal frequency from the luminance signal of the input image;
Horizontal low frequency component image generation means for generating a first horizontal intermediate image obtained by extracting only low frequency components equal to or lower than a second horizontal frequency from the first horizontal high frequency component image;
The first intermediate image includes the first horizontal intermediate image;
The zero-cross determining means has a horizontal zero-cross determining means for determining, as a zero-cross point, a point where the pixel value of the first horizontal intermediate image changes from positive to negative or from negative to positive along the horizontal direction. And
The signal amplifying unit generates a horizontal non-linearly processed image obtained by amplifying a pixel value of a pixel existing in the vicinity of the zero cross point among the pixels constituting the first horizontal intermediate image with an amplification factor larger than 1. Having horizontal signal amplification means;
The high frequency component image generation means generates a second horizontal high frequency component image that extracts only a high frequency component equal to or higher than a third horizontal frequency of the horizontal nonlinear processed image. Having component image generation means;
The image processing apparatus according to claim 1, wherein the second intermediate image includes the second horizontal high-frequency component image. The first intermediate image generating means includes
First vertical high frequency component image generation means for generating a first vertical high frequency component image obtained by extracting a high frequency component equal to or higher than the first vertical frequency from the luminance signal of the input image;
Vertical low frequency component image generation means for generating a first vertical intermediate image obtained by extracting only a low frequency component equal to or lower than a second vertical frequency from the first vertical high frequency component image;
The first intermediate image includes the first vertical intermediate image;
The zero-cross determining unit includes a vertical zero-cross determining unit that determines, as a zero-cross point, a portion where the pixel value of the first vertical intermediate image changes from positive to negative or from negative to positive along the vertical direction. And
The signal amplifying unit generates a vertical nonlinear processed image obtained by amplifying a pixel value of a pixel existing in the vicinity of the zero-cross point among the pixels constituting the first vertical intermediate image with an amplification factor larger than 1. Having vertical signal amplification means;
The high-frequency component image generation means generates a second vertical high-frequency component image by extracting only a high-frequency component equal to or higher than a third vertical frequency of the vertical non-linear processing image. Having component image generation means;
The image processing apparatus according to claim 1, wherein the second intermediate image includes the second vertical high-frequency component image. The image display apparatus comprising the image processing apparatus according to any one of claims 1 to 6. A first intermediate image generation step of generating a first intermediate image obtained by extracting a component of a specific frequency band from a luminance signal of an input image composed of a luminance signal and a color difference signal;
A second intermediate image generating step for generating a second intermediate image based on the first intermediate image;
Luminance color difference addition step for weighted addition of absolute values of the luminance signal and color difference signal of the input image;
A first intermediate image processing step of generating a third intermediate image obtained by amplifying the pixel value of the first intermediate image according to a first amplification factor;
A second intermediate image processing step of generating a fourth intermediate image obtained by amplifying the pixel value of the second intermediate image according to a second amplification factor;
In an image processing method including an addition step of adding the luminance signal of the input image, the third intermediate image, and the fourth intermediate image,
The second intermediate image generation step includes
A zero-cross determination step in which a point where the pixel value of the first intermediate image changes from a positive value to a negative value or from a negative value to a positive value is regarded as a zero-cross point;
A signal amplification step for generating a non-linearly processed image obtained by amplifying a pixel value of a pixel in the vicinity of the zero-cross point among the pixels constituting the first intermediate image with an amplification factor greater than 1,
A high-frequency component image generation step for generating a high-frequency component image obtained by extracting only the high-frequency component of the nonlinear processed image;
Outputting the high-frequency component image as the second intermediate image;
The first amplification factor is:
When the sign of the pixel value of the first intermediate image is positive and the output value of the luminance color difference addition step is large, it becomes a small value,
When the sign of the pixel value of the first intermediate image is negative and the output value of the luminance color difference addition step is small, it is determined to be a small value,
The second amplification factor is
When the sign of the pixel value of the second intermediate image is positive and the output value of the luminance color difference addition step is large, it becomes a small value,
When the sign of the pixel value of the second intermediate image is negative and the output value of the luminance color difference addition step is small, the image processing method is determined to be a small value. An image display apparatus that displays an image processed by the image processing method according to claim 8 .

Images