JP2011049868A - Image processing apparatus and method, and image display device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a magnified image with a resolved feeling while uniformly supplying high frequency components regardless of the direction of the edge of images. <P>SOLUTION: For a second horizontal direction intermediate image (D32Bh), nonlinear processing is performed to a magnified image of an image (D1h) for which the high frequency components in the horizontal direction of an input image (Din) are extracted. For a second vertical direction intermediate image (D32Bv), the nonlinear processing is performed to a magnified image of an image (D1v) for which the high frequency components in the vertical direction of the input image (Din) are extracted. To an image (D2A) for which the input image (Din) is magnified, an image (D33B) for which the second horizontal direction intermediate image (D32Bh) and the second vertical direction intermediate image (D32Bv) are weighted and added is added. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and method for enlarging a digitized image, and an image display apparatus and method, and generates a high frequency component when enlarging an image, thereby providing a high resolution feeling. An enlarged image is obtained.

  In general, when the number of pixels of the output image is larger than the number of pixels of the input image, the image processing apparatus must enlarge the image. In a conventional image processing apparatus, an image is enlarged by weighting and adding pixel values of pixels in the vicinity of the pixel of interest.

  For example, in the image processing apparatus described in Patent Document 1, five adjacent pixel data in the main scanning direction output from each shift register are multiplied by a predetermined weighting constant, and the multiplication result in each pixel data is obtained. An arithmetic circuit for adding is provided, and when image data enlargement processing is performed, the arithmetic result in the arithmetic circuit is selected and output by the selector as pixel data at the center of these pixel data.

JP-A-6-31346 (FIG. 1)

  Weighting and adding pixel values of pixels in the vicinity of the target pixel is a low-pass filter process that passes only the low-frequency component of the input image. Therefore, the above-described conventional technique has a problem that the resolution of the enlarged image is lost because a high-frequency component cannot be sufficiently applied to the enlarged image.

The present invention has been made to solve the above-described problems, and the image processing apparatus of the present invention includes:
In an image processing apparatus for enlarging an input image,
First image enlarging means for enlarging the input image and outputting a first enlarged image;
First high frequency component image generation means for generating a first high frequency component image obtained by extracting the high frequency component of the input image;
Second image enlarging means for enlarging the first high frequency component image and outputting a second enlarged image;
High-frequency component image processing means for generating a second high-frequency component image from the second enlarged image;
In the image processing apparatus having first addition means for adding the first enlarged image and the second high-frequency component image,
The first high frequency component image generation means includes:
First horizontal high frequency component image generation means for generating a first horizontal high frequency component image obtained by extracting a horizontal high frequency component of the input image;
First vertical high frequency component image generation means for generating a first vertical high frequency component image obtained by extracting a vertical high frequency component of the input image;
The second enlarged image is
A third enlarged image obtained by enlarging the first horizontal high-frequency component image, and a fourth enlarged image obtained by enlarging the first vertical high-frequency component image,
The high frequency component image processing means includes
A second horizontal high-frequency component image generating means for generating a first horizontal intermediate image obtained by extracting a high-frequency component of the third enlarged image; and a first of extracting a high-frequency component of the fourth enlarged image. Second vertical high frequency component image generating means for generating a vertical intermediate image of
A horizontal non-linearly processed image generating means for generating a second horizontal intermediate image obtained by performing non-linear processing on the third enlarged image;
Vertical non-linear processing image generation means for generating a second vertical intermediate image obtained by performing non-linear processing on the fourth enlarged image;
First weighted addition means for generating a third intermediate image obtained by weighted addition of the first horizontal intermediate image and the first vertical intermediate image;
Second weighted addition means for generating a fourth intermediate image obtained by weighted addition of the second horizontal intermediate image and the second vertical intermediate image;
And a second adding means for adding the third intermediate image and the fourth intermediate image.

  According to the present invention, high-frequency components can be uniformly applied regardless of the direction of the edge of the image, and an enlarged image with a sense of resolution can be obtained.

It is a block diagram which shows the structure of the image processing apparatus of Embodiment 1 of this invention. 1 is a block diagram illustrating a configuration example of an image display device using the image processing device according to a first embodiment. (A)-(d) is a pixel arrangement | positioning figure which shows operation | movement of the image expansion means 2A. It is a block diagram which shows the structural example of 2 A of image expansion means. It is a block diagram which shows the structural example of the image expansion means 2B. It is a block diagram which shows the structural example of the horizontal direction nonlinear processing means 31h. It is a block diagram which shows the structural example of the vertical direction nonlinear processing means 31v. It is a block diagram which shows the structural example of the addition ratio determination means 33C of FIG. It is a figure showing an example of how to determine the weighting coefficient D33C with respect to the edge direction estimated amount D33C1 by the weighting coefficient determining means 33C2 of FIG. (A)-(d) is a figure which shows the frequency spectrum and frequency response for demonstrating the process of obtaining enlarged image D2A. (A)-(f) is a figure which shows the frequency spectrum and frequency response for demonstrating the process of obtaining intermediate | middle image D32A. (A)-(c) is a figure which shows the frequency spectrum and frequency response for demonstrating the process of obtaining intermediate image D32B. (A)-(e) is a figure which shows the signal intensity | strength obtained when a step edge signal and a step edge signal are sampled with a different sampling frequency, and the signal strength of the high frequency component. (A)-(f) shows the signal strength for demonstrating operation | movement of the nonlinear processing means 31 and the high frequency component image generation means 32B. It is explanatory drawing of the frequency spectrum of the expansion image Dout. (A)-(d) is a figure which shows the signal strength for demonstrating the effect which adds intermediate image D32A, D32B. (A)-(k) is a figure which shows the direction of the edge of an input image, and the signal for demonstrating operation | movement when the direction of the edge of an input image is horizontal as shown to Fig.17 (a). It is a figure which shows intensity | strength. (A)-(k) is a figure which shows the direction of the edge of an input image, and the signal for demonstrating operation | movement when the direction of the edge of an input image is perpendicular | vertical as shown to Fig.18 (a). It is a figure which shows intensity | strength. (A)-(k) is a figure which shows the direction of the edge of an input image, and the signal for demonstrating operation | movement when the direction of the edge of an input image is diagonal as shown to Fig.19 (a) It is a figure which shows intensity | strength. It is a figure explaining the relationship between the direction of an edge, and the value of each signal. FIG. 6 is a block diagram illustrating a configuration of one modification of the image processing apparatus according to the first embodiment. FIG. 10 is a block diagram illustrating a configuration of another modification of the image processing apparatus according to the first embodiment. (A)-(e) is a figure which shows the signal intensity | strength obtained when a step edge signal and a step edge signal are sampled with a different sampling frequency, and the signal strength of the high frequency component. (A)-(f) is a figure which shows the strength of the signal for demonstrating operation | movement of the nonlinear processing means 31 and the high frequency component image generation means 32B. It is a block diagram which shows the structure of the image processing apparatus of Embodiment 2 of this invention. 10 is a flowchart illustrating an image processing method according to Embodiment 2. FIG. It is a flowchart which shows high frequency component image generation step ST1. It is a flowchart which shows image expansion step ST2B. It is a flowchart which shows high frequency component image processing step ST3. It is a flowchart which shows horizontal direction nonlinear process step ST31h. It is a flowchart which shows vertical direction nonlinear processing step ST31v. It is a flowchart which shows addition ratio determination step ST33. It is a block diagram which shows the structure of the image processing apparatus of Embodiment 3 of this invention. FIG. 34 is a block diagram illustrating a configuration example of an addition ratio determination unit 33CA in FIG. 33. It is a figure showing an example of the determination method of the weighting coefficient D33CA with respect to edge direction estimation amount D33CA1 by the weighting coefficient determination means 33CA2 of FIG. FIG. 34 is a block diagram illustrating a configuration example of an addition ratio determining unit 33CB in FIG. 33. It is a figure showing an example of how to determine the weighting coefficient D33CB with respect to the edge direction estimation amount D33CB1 by the weighting coefficient determination means 33CB2 of FIG. It is a flowchart which shows high frequency component image processing step ST3 in the image processing method of Embodiment 4 of this invention. It is a flowchart which shows addition ratio determination step ST33CA of FIG. It is a flowchart which shows addition ratio determination step ST33CB of FIG.

Embodiment 1 FIG.
FIG. 1 is a diagram showing a configuration of an image processing apparatus according to Embodiment 1 of the present invention, and can be used as a part of the image display apparatus shown in FIG. 2, for example. Here, the image display apparatus shown in FIG. 2 includes an image processing apparatus U1 including the image processing apparatus shown in FIG. 1 and a display unit 9, and an image DU1 obtained as an output for the image DORG in the image processing apparatus U1. It is displayed on the display unit 9.

The image processing apparatus according to Embodiment 1 includes an image enlarging means 2A, a high frequency component image generating means 1, an image enlarging means 2B, a high frequency component image processing means 3, and an adding means 4.
The image enlarging means 2A enlarges the input image Din and generates an enlarged image D2A.
The high frequency component image generation means 1 extracts only the high frequency component of the input image Din and generates a high frequency component image D1.

The image enlarging means 2B enlarges the high frequency component image D1 output from the high frequency component image generating means 1 to generate an enlarged image (high frequency component enlarged image) D2B.
The high frequency component image processing means 3 performs a process described later on the enlarged image D2B output from the image enlargement means 2B, and generates a high frequency component image (high frequency component processed image) D3.

  The adding means 4 adds the high frequency component image D3 output from the high frequency component image processing means 3 to the enlarged image D2A output from the image enlarging means 2A, and the result is the final enlarged image, that is, the output Output as an image Dout. The output of the adding means 4 is supplied as, for example, an image DU1 to the display unit 9 of the image display device shown in FIG.

  In this specification, it is described that processing such as enlargement, high-frequency component generation, and high-frequency component processing is performed on an “image”. Specifically, it is performed on digital data representing an image. Is called. In addition, the description of “image” may specifically mean “image data”.

  Detailed operations of the image enlarging unit 2A, the high frequency component image generating unit 1, the image enlarging unit 2B, and the high frequency component image processing unit 3 will be described later. The frequency component of the high frequency component image D3 is obtained by the enlarged image D2A. The frequency component is higher than the frequency component. Therefore, by adding the high frequency component image D3 to the enlarged image D2A in the adding means 4, an enlarged image Dout containing a large amount of high frequency components can be obtained.

  The configuration of the image enlarging means 2A, the high frequency component image generating means 1, the image enlarging means 2B, and the high frequency component image processing means 3 will be described in more detail.

  The image enlarging means 2A is for enlarging an image in at least one of the horizontal direction and the vertical direction. For example, the image enlarging means 2A enlarges the image in the horizontal direction and the vertical direction at the same magnification, but instead has different magnifications in the horizontal direction and the vertical direction. It is also possible to perform enlargement with. Further, enlargement may be performed only in one of the horizontal direction and the vertical direction. For example, when the display screen is horizontally long with respect to the input image, enlargement may be performed only in the horizontal direction.

  The high frequency component image generation means 1 extracts a high frequency component (a component higher than the predetermined frequency Fb) of the input image Din and generates a high frequency component image D1. A horizontal high-frequency component image generating unit 1h and a vertical high-frequency component image generating unit 1v that generate a frequency component image D1h and a vertical high-frequency component image D1v are provided. The high frequency component image D1 is composed of the horizontal high frequency component image D1h and the vertical high frequency component image D1v.

  The image enlarging means 2B includes an image enlarging means 2Bh that generates an enlarged image D2Bh obtained by enlarging the horizontal high-frequency component image D1h, and an image enlarging means 2Bv that generates an enlarged image D2Bv obtained by enlarging the vertical high-frequency component image D1v. The enlarged image D2B is composed of the enlarged image D2Bh and the enlarged image D2Bv.

  When the image enlarging means 2A enlarges in both the horizontal and vertical directions, the image enlarging means 2Bh enlarges the horizontal high frequency component image D1h in both the horizontal and vertical directions, and the image enlarging means 2Bv The direction high frequency component image D1v is enlarged in both the horizontal direction and the vertical direction. The enlargement of the horizontal high-frequency component image D1h and the vertical high-frequency component image D1v by the image enlargement means 2Bh and 2Bv is performed at the same magnification for each of the enlargement by the image enlargement means 2A and the horizontal and vertical directions.

  The high frequency component image processing means 3 includes a high frequency component image generating means 32A, a nonlinear processed image generating means 30, and an adding means 34.

  The high frequency component image generating means 32A extracts the high frequency component (component higher than the predetermined frequency Fd) of the enlarged image D2B and outputs an intermediate image (high frequency component image) D32A, which is included in the enlarged image D2Bh. A horizontal high-frequency component image generating unit 32Ah that generates a horizontal intermediate image D32Ah that extracts only the high-frequency component in the horizontal direction, and a vertical intermediate image D32Av that extracts only the high-frequency component in the vertical direction of the enlarged image D2Bv. A vertical high-frequency component image generating unit 32Av is generated, and an intermediate image D32A composed of a horizontal intermediate image D32Ah and a vertical intermediate image D32Av is output from the high-frequency component image generating unit 32A.

The nonlinear processed image generating means (edge sharpened image generating means) 30 outputs an intermediate image (edge sharpened image) D32B obtained by performing processing including nonlinear processing on the enlarged image D2B. And high frequency component image generation means 32B.
The nonlinear processing means 31 generates a nonlinear processed image D31 obtained by performing nonlinear processing for edge sharpening, which will be described later, on the enlarged image D2B.
The high frequency component image generating means 32B outputs an intermediate image D32B obtained by extracting the high frequency component (component higher than the predetermined frequency Ff) included in the nonlinear processed image D31.

  The non-linear processing means 31 generates a non-linear processed image D31h that is non-linearly processed with respect to the enlarged image D2Bh, and a vertical non-linear process that generates a non-linearly processed image D31v that is non-linearly processed with respect to the enlarged image D2Bv. Means 31v is provided, and the nonlinear processed image D31 is composed of a nonlinear processed image D31h and a nonlinear processed image D31v.

  The high frequency component image generation unit 32B extracts the high frequency component from the nonlinear processed image D31h, extracts the high frequency component from the horizontal processed high frequency component image generation unit 32Bh that generates the horizontal intermediate image D32Bh, and the nonlinear processed image D31v, A vertical high frequency component image generating means 32Bv for generating a vertical intermediate image D32Bv is provided, and the intermediate image D32B is composed of a horizontal intermediate image D32Bh and a vertical intermediate image D32Bv.

  The horizontal direction nonlinear processing means 31h and the horizontal direction high frequency component image generation means 32Bh constitute a horizontal direction nonlinear processing image generation means, and the vertical direction nonlinear processing means 31v and the vertical direction high frequency component image generation means 32Bv constitute a vertical direction nonlinearity. Processed image generation means is configured.

  The adding unit 34 includes horizontal / vertical adding units 33A and 33B, an addition ratio determining unit 33C, and an adding unit 33D.

The addition ratio determination means 33C determines and outputs a weighting coefficient D33C by a process described later.
The horizontal / vertical addition means 33A outputs an intermediate image (corrected image) D33A obtained by weighted addition of the horizontal direction intermediate image D32Ah and the vertical direction intermediate image D32Av by processing described later.
The horizontal / vertical addition means 33B outputs an intermediate image (corrected image) D33B obtained by weighted addition of the horizontal direction intermediate image D32Bh and the vertical direction intermediate image D32Bv by processing described later.
The adding means 33D adds the intermediate image D33A and the intermediate image D33B, and outputs a high frequency component image D3.

  Hereinafter, the operation of each component will be described in more detail by taking as an example the case of generating an enlarged image Dout in which the input image Din is doubled in both the horizontal direction and the vertical direction. Through this description, the operation and effect of the present invention will become clearer.

  First, the operation of the image enlarging means 2A will be described. The image enlarging means 2A generates an enlarged image D2A obtained by enlarging the input image Din twice in both the horizontal direction and the vertical direction. FIGS. 3A to 3D are diagrams schematically showing an example of a procedure for generating the enlarged image D2A in the image enlarging unit 2A, and FIG. 4 is a diagram showing an example of the image enlarging unit 2A.

  The image enlarging unit 2A includes a zero insertion unit 21A and a low frequency component passing unit 22A. Hereinafter, the operations of the zero insertion means 21A and the low frequency component passing means 22A will be described with reference to FIGS. 3A shows the input image Din (particularly, the arrangement of pixels constituting a part of the image), FIG. 3B shows the zero insertion image D21A generated by the zero insertion means 21A, and FIG. FIG. 3D shows the filter coefficient used when the enlarged image D2A is generated by the low frequency component passing means 22A, and FIG. 3D shows the enlarged image D2A generated by the low frequency component passing means 22A. 3A, 3B, and 3D show horizontal coordinates X and vertical coordinates Y corresponding to the positions of the pixels.

In the zero insertion means 21A, one pixel per pixel (in the input image Din) having a pixel value 0 with respect to the input image Din (one between two adjacent pixels) in the horizontal direction, and one in the vertical direction A zero-inserted image D21A is generated in which one (one between two adjacent lines) is inserted per line (of the input image Din).
If “PXY” represents the pixel value of the pixel at the coordinates (X, Y) of the input image Din, and “P′XY” represents the pixel value of the pixel at the coordinates (X, Y) of the zero-inserted image D21A, then zero. The pixel value represented by P ′ (2X−1) (2Y−1) in the insertion image D21A is equal to PXY in the input image Din, and P ′ (2X−1) (2Y), P ′ (2X) (2Y). ), P ′ (2X) (2Y−1), the pixel value is equal to zero.

The low frequency component passing means 22A performs the filter operation represented by the filter coefficient shown in FIG. 3C on the zero insertion image D21A to generate the enlarged image D2A shown in FIG. .
For example, the pixel value QXY of the pixel at coordinates (X, Y) included in the enlarged image D2A is calculated as in the following equation (1).

QXY = (4/16) ×
{P ′ (X−1) (Y−1) + 2P′X (Y−1) + P ′ (X + 1) (Y−1)
+ 2P ′ (X−1) Y + 4P′XY + 2P ′ (X + 1) Y
+ P ′ (X−1) (Y + 1) + 2P′X (Y + 1) + P ′ (X + 1) (Y + 1)}
... (1)

  Since the filter coefficient represented in FIG. 3C represents a low-pass filter, the processing in the low-frequency component passing means 22A represented by the equation (1) is performed by the low-frequency component (predetermined frequency Fa This corresponds to taking out the following components).

In the formula (1), P ′ (X−1) (Y−1), 2P′X (Y−1), P ′ (X + 1) (Y−1), 2P ′ (X−1) Y, P 'XY, 2P' (X + 1) Y, P '(X-1) (Y + 1), P'X (Y + 1), P' (X + 1) (Y + 1)
Some of them have a value of 0, and the others have pixel values themselves of the input image Din. Therefore, the enlargement process is the same as the process of appropriately weighting and adding pixel values in the vicinity of the pixel of interest in the input image Din.

Next, operations of the horizontal direction high frequency component image generating unit 1h and the vertical direction high frequency component image generating unit 1v will be described.
The horizontal high-frequency component image generating means 1h applies a high-pass filter using, for example, a predetermined number of pixels in the vicinity of each pixel of the input image Din and the horizontal direction to the input image Din, and applies the horizontal high-frequency component image. D1h is generated.
On the other hand, the vertical high frequency component image generation unit 1v applies a high pass filter to the input image Din by applying a high pass filter using, for example, a predetermined number of pixels in the vicinity of each pixel of the input image Din and the vertical direction. A component image D1v is generated.

  Applying a high-pass filter corresponds to extracting a high-frequency component, and the horizontal high-frequency component image D1h includes a high-frequency component in the horizontal direction of the input image Din (consisting of a component higher than a predetermined horizontal frequency). The vertical high frequency component image D1v includes a high frequency component in the vertical direction of the input image Din (consisting of a component higher than a predetermined vertical frequency).

As a process for applying a high-pass filter performed by the horizontal high-frequency component image generating means 1h, for example, a low-frequency component in the horizontal direction (or a predetermined line aligned in the horizontal direction with respect to each pixel) from the input signal to the means 1h. By subtracting a simple average value or a weighted average value of pixel values in a local region composed of a number of pixels, it is possible to perform processing for extracting a high frequency component.
Similarly, as a process of applying a high-pass filter performed by the vertical high frequency component image generating means 1v, for example, from the input signal to the means 1v, the vertical low frequency component (or the vertical direction for each pixel). By subtracting a simple average value or a weighted average value of pixel values in a local region composed of a predetermined number of aligned pixels, it is possible to perform processing for extracting a high frequency component.

  Next, the operation of the image enlarging means 2Bh and 2Bv will be described. The image enlarging means 2Bh generates an enlarged image D2Bh obtained by enlarging the horizontal high-frequency component image D1h twice in both the horizontal and vertical directions, and the image enlarging means 2Bv converts the vertical high-frequency component image D1v in both the horizontal and vertical directions. An enlarged image D2Bv enlarged twice is generated.

Each of the image enlarging means 2Bh and the image enlarging means 2Bv can be configured similarly to the image enlarging means 2A described with reference to FIG. Therefore, the image enlarging means 2B composed of the image enlarging means 2Bh and the image enlarging means 2Bv can be shown as shown in FIG.
The input of the image enlarging means 2Bh is a horizontal high frequency component image D1h, and the output is an enlarged image D2Bh. The input of the image enlarging means 2Bv is a vertical high frequency component image D1v, and the output is an enlarged image D2Bv.

The image enlarging unit 2Bh includes a zero insertion unit 21Bh and a low frequency component passing unit 22Bh, and the image enlarging unit 2Bv includes a zero insertion unit 21Bv and a low frequency component passing unit 22Bv.
Each of the zero insertion means 21Bh and the zero insertion means 21Bv is similar to the zero insertion means 21A of FIG. 4, and each of the low frequency component passage means 22Bh and the low frequency component passage means 22Bv is a low frequency component of FIG. This is the same as the passing means 22A.

The zero insertion image D21B as the output of the zero insertion means 21B is composed of the zero insertion image D21Bh output from the zero insertion means 21Bh and the zero insertion image D21Bv output from the zero insertion means 21Bv.
The enlarged image D2B as the output of the low frequency component passing means 22B is composed of the enlarged image D2Bh outputted from the low frequency component passing means 22Bh and the enlarged image D2Bv outputted from the low frequency component passing means 22Bv. The output of the low frequency component passing means 22B is obtained by extracting the low frequency component (component below Fc) of the zero insertion image D21B.

Next, the operation of the high frequency component image generating means 32A will be described.
The horizontal high-frequency component image generation means 32Ah applies a high-pass filter in the horizontal direction to the enlarged image D2Bh to extract a high-frequency component composed of components of a predetermined horizontal frequency or more, and generates a horizontal intermediate image D32Ah.
On the other hand, the vertical high-frequency component image generating means 32Av applies a high-pass filter in the vertical direction to the enlarged image D2Bv to extract a high-frequency component composed of components having a frequency equal to or higher than a predetermined vertical frequency, and generates a vertical intermediate image D32Av. .
Then, an intermediate image D32A composed of the horizontal intermediate image D32Ah and the vertical intermediate image D32Av is output from the high frequency component image generating means 32A.

The high-pass filter processing performed by the horizontal high-frequency component image generation unit 32Ah is performed in the same manner as the horizontal high-frequency component image generation unit 1h, and the high-pass filter performed by the vertical high-frequency component image generation unit 32Av. The processing to be applied can be performed in the same manner as the processing in the vertical high frequency component image generation means 1v.
That is, the processing for applying the high-pass filter performed by the horizontal high-frequency component image generation means 32Ah is, for example, from the input signal to the means 32Ah in the horizontal direction, as in the processing by the horizontal high-frequency component image generation means 1h. To extract a high frequency component by subtracting the low frequency component of (or a simple average value or a weighted average value of pixel values in a local region composed of a predetermined number of pixels aligned in the horizontal direction with respect to each pixel). it can.

  Similarly, as a process for applying a high-pass filter performed by the vertical high frequency component image generating means 32Av, for example, from the input signal to the means 32Av, a low frequency component in the vertical direction (or in a direction perpendicular to each pixel). By subtracting a simple average value or a weighted average value of pixel values in a local region composed of a predetermined number of aligned pixels, it is possible to perform processing for extracting a high frequency component.

  Next, the operation of the nonlinear processing means 31 will be described. The nonlinear processing means 31 includes a horizontal nonlinear processing means 31h and a vertical nonlinear processing means 31v. The horizontal non-linear processing means 31h and the vertical non-linear processing means 31v are configured in the same manner. However, the horizontal non-linear processing means 31h performs horizontal processing, and the vertical non-linear processing means 31v performs vertical processing.

  FIG. 6 is a diagram showing the internal configuration of the horizontal nonlinear processing means 31h. The illustrated horizontal non-linear processing means 31h includes a zero-cross determining means 311h and a signal amplifying means 312h.

The zero cross determination means 311h confirms the change in the pixel value in the inputted enlarged image D2Bh along the horizontal direction. Then, a point where the pixel value changes from a positive value to a negative value or from a negative value to a positive value is regarded as a zero cross point, and pixels before and after the zero cross point by the signal D311h (in the illustrated example, each of the pixels immediately before and after 1 pixel) is transmitted to the signal amplifying means 312h.
In the horizontal non-linear processing means 31h, pixels located on the left and right of the zero cross point are recognized as pixels before and after the zero cross point.

The horizontal direction signal amplifying unit 312h amplifies the pixel value of the third enlarged image D2Bh with an amplification factor determined according to the determination result of the horizontal direction zero cross determining unit 311h. Specifically, the signal amplifying unit 312h specifies pixels before and after the zero cross point (pixels existing in a predetermined region including the zero cross point) based on the signal D311h, and regarding the pixels before and after the zero cross point. Only the pixel value is amplified (the absolute value is increased) to generate a nonlinear processed image D31h. That is, the amplification factor for the pixel values of the pixels before and after the zero cross point is set to a value larger than 1, and the amplification factors for the pixel values of the other pixels are set to 1.
By such processing, edge sharpening including stepwise changes in signal values of pixels arranged in the horizontal direction is performed.

  FIG. 7 is a diagram showing the internal configuration of the vertical nonlinear processing means 31v. The illustrated vertical non-linear processing means 31v includes a zero-cross determining means 311v and a signal amplifying means 312v.

The zero-cross determining unit 311v confirms the change of the pixel value in the input enlarged image D2Bv along the vertical direction. A point where the pixel value changes from a positive value to a negative value or from a negative value to a positive value is regarded as a zero cross point, and pixels before and after the zero cross point by the signal D311v (in the illustrated example, the respective pixels immediately before and after 1 pixel) is transmitted to the signal amplifying means 312v.
The vertical nonlinear processing means 31v recognizes pixels located above and below the zero cross point as pixels before and after the zero cross point.

The vertical direction signal amplifying unit 312v amplifies the pixel value of the fourth enlarged image D2Bv with an amplification factor determined according to the determination result of the vertical direction zero cross determining unit 311v. Specifically, the signal amplifying unit 312v identifies pixels (pixels existing within a predetermined region including the zero cross point) before and after the zero cross point based on the signal D311v, and detects the pixels before and after the zero cross point. Only the pixel value is amplified (the absolute value is increased) to generate a nonlinear processed image D31v. That is, the amplification factor for the pixel values of the pixels before and after the zero cross point is set to a value larger than 1, and the amplification factors for the pixel values of the other pixels are set to 1.
By such processing, edge sharpening including step-like changes in signal values of pixels arranged in the vertical direction is performed.

  In the example shown in the figure, the pixel value is amplified only for each one pixel immediately before and after the zero cross point, but a predetermined number of pixels before and after the zero cross point, in other words, the zero cross point is determined. It is also possible to amplify the pixel value for a pixel that exists in a predetermined region. Alternatively, the size of the “predetermined area” (the number of pixels included in the predetermined area) may be changed according to the image enlargement ratio in the image enlargement unit 2B (set to an appropriate value for the enlargement ratio). good.

Next, the operation of the high frequency component image generating means 32B will be described.
The horizontal high-frequency component image generation unit 32Bh applies a high-pass filter in the horizontal direction to the nonlinear processed image D31h to extract a high-frequency component composed of components having a frequency equal to or higher than a predetermined horizontal frequency, and generates a horizontal intermediate image D32Bh. On the other hand, the vertical high-frequency component image generating means 32Bv applies a high-pass filter in the vertical direction to the nonlinear processed image D31v to extract a high-frequency component composed of components of a predetermined vertical frequency or more, and generates a vertical intermediate image D32Bv. To do. An intermediate image D32B composed of the horizontal intermediate image D32Bh and the vertical intermediate image D32Bv generated in this way is output from the high frequency component image generating means 32B.

The high-pass filter processing performed by the horizontal high-frequency component image generation unit 32Bh is performed in the same manner as the processing by the horizontal high-frequency component image generation unit 1h, and the high-pass filter performed by the vertical high-frequency component image generation unit 32Bv. The processing to be applied can be performed in the same manner as the processing in the vertical high frequency component image generation means 1v.
That is, as a process for applying a high-pass filter performed by the horizontal high-frequency component image generating means 32Bh, for example, the horizontal low-frequency component (or the horizontal alignment for each pixel) from the input signal to the means 32Bh. By subtracting the simple average value or the weighted average value of the pixel values in the local area composed of the predetermined number of pixels, it is possible to perform processing for extracting the high frequency component.

  Similarly, as a process for applying a high-pass filter performed by the vertical high frequency component image generating means 32Bv, for example, from the input signal to the means 32Bv, the vertical low frequency component (or the vertical direction for each pixel). By subtracting a simple average value or a weighted average value of pixel values in a local region composed of a predetermined number of aligned pixels, it is possible to perform processing for extracting a high frequency component.

  Next, the operation of the adding means 34 will be described. The adding unit 34 includes an adding ratio determining unit 33C, horizontal and vertical adding units 33A and 33B, and an adding unit 33D.

First, the operation of the addition ratio determination unit 33C will be described with reference to FIGS.
FIG. 8 is a diagram illustrating a configuration example of the addition ratio determination unit 33C of FIG. 1, and the addition ratio determination unit includes an edge direction estimation unit 33C1 and a weighting factor determination unit 33C2.

  The edge direction estimation means 33C1 calculates an edge direction estimation amount D33C1 as an amount corresponding to the edge direction (angle) from the two signals of the enlarged image D2Bh and the enlarged image D2Bv. For example, when the absolute value of each pixel value of the enlarged image D2Bh is dH and the absolute value of each pixel value of the enlarged image D2Bv is dV, the difference dH−dV between these two values is output as the edge direction estimation amount D33C1. .

により算出している。ただし、Ｋαは図９における直線の傾き（従って（ｄＨ−ｄＶ）の増加に対するＤ３３Ｃの変化の割合）を表す正の定数である。
以上が、加重比率決定手段３３Ｃの動作である。 The weighting coefficient determination unit 33C2 determines the weighting coefficient D33C based on the edge direction estimation amount D33C1. FIG. 9 is a diagram illustrating an example of how to determine the weighting coefficient D33C with respect to the edge direction estimation amount D33C1. In the example shown in FIG. 9, the weighting coefficient D33C is

It is calculated by. However, Kα is a positive constant representing the slope of the straight line in FIG.
The above is the operation of the weight ratio determining means 33C.

と表される。 Next, the operation of the horizontal / vertical addition means 33A will be described. The horizontal / vertical adder 33A performs weighted addition of the horizontal intermediate image D32Ah and the vertical intermediate image D32Av, and outputs the result as an intermediate image D33A. The weighting coefficient D33C is used as the weighted addition ratio. That is, the weighted addition performed by the horizontal / vertical addition means 33A is

It is expressed.

と表される。 Next, the operation of the horizontal / vertical addition means 33B will be described. The horizontal / vertical adder 33B performs weighted addition of the horizontal intermediate image D32Bh and the vertical intermediate image D32Bv, and outputs the result as an intermediate image D33B. The weighting coefficient D33C is used as the weighted addition ratio. That is, the weighted addition performed by the horizontal / vertical addition means 33B is

It is expressed.

Next, the operation of the adding means 33D will be described. The adding unit 33D adds the intermediate image D33A and the intermediate image D33B. The result is output from the adding means 34 as a high frequency component image D3. Note that the addition process here is not limited to simple addition, and may be an addition process after applying individual gains to the intermediate image D33A and the intermediate image D33B.
The above is the operation of the adding means 34.

  Finally, the operation of the adding means 4 will be described. The adding means 4 adds the enlarged image D2A and the high frequency component image D3. Then, an image obtained as a result of adding the enlarged image D2A and the high frequency component image D3 in the adding means 4 is output from the image processing apparatus as a final enlarged image Dout. The addition process here is not limited to simple addition, and may be a process of adding each image with an individual weight.

The operation and effect of the image processing apparatus according to the present invention will be described below.
In the embodiment of the present invention, when the intermediate images D33A and D33B are generated, the horizontal intermediate image D32Ah and the vertical intermediate image D32Av are simply added, or the horizontal intermediate image D32Bh and the vertical intermediate image D32Bv are simply added. Although not necessarily described below, an effect obtained by simply adding will be described below, and then an effect obtained by performing weighted addition instead of simply adding will be described.

  The enlarged image D2A includes a frequency component corresponding to a frequency lower than the Nyquist frequency Fn of the input image Din, and the high frequency component image D3 includes a frequency component corresponding to a frequency equal to or higher than the Nyquist frequency Fn of the input image Din. Therefore, the magnified image Dout generated by adding the magnified image D2A and the high frequency component image D3 has frequency components over all frequency regions that reach the Nyquist frequency after the image is magnified.

  First, it will be described that the enlarged image D2A has a frequency component corresponding to a frequency lower than the Nyquist frequency Fn of the input image Din.

  FIGS. 10A to 10D are diagrams schematically showing the operation when the enlarged image D2A is generated from the input image Din. FIG. 10A shows the frequency spectrum of the input image Din. FIG. 10C shows the frequency spectrum of the low-frequency component passing means 22A, and FIG. 10D shows the frequency spectrum of the enlarged image D2A.

  The frequency spectrum of the input image Din will be described. Normally, natural images and the like are input from the input image Din, but the spectral intensities of these images are concentrated around the origin of the frequency space. Therefore, the frequency spectrum of the input image Din can be expressed as shown in FIG. Here, the vertical axis in FIG. 10A represents the spectral intensity, the horizontal axis represents the spatial frequency, and Fn represents the Nyquist frequency of the input image Din.

  Since the normal input image Din is a two-dimensional image, its frequency spectrum is also represented in a two-dimensional frequency space, but its shape is isotropic with the frequency spectrum shown in FIG. It will spread to. Therefore, in order to describe the frequency spectrum, it is only necessary to show the shape of one dimension at a minimum. In the future, unless otherwise specified, the shape of the frequency space will be described by showing only one dimension.

  Next, the frequency spectrum of the zero insertion image D21A will be described. By inserting one pixel per pixel (of the input image Din) and a pixel value of 0 with respect to the input image Din by the zero insertion means 21A, folding around the frequency Fn occurs in the frequency space. As a result, the frequency spectrum of the zero insertion image D21A is as shown in FIG.

Next, the frequency response of the low frequency component passing means 22A will be described. As described above, since the calculation in the low-frequency component passage means 22A is a low-pass filter, the frequency response of the low-frequency component passage means 22A becomes lower as the frequency becomes higher, as shown in FIG.
In the example shown in the figure, the low frequency component passing means 22A mainly passes the frequency component equal to or lower than the first frequency Fa (= Fn).

  Finally, the frequency spectrum of the enlarged image D2A will be described. An enlarged image D2A is generated by passing the zero insertion image D21A having the frequency spectrum shown in FIG. 10B through the low-frequency component passing means 22A having the frequency response shown in FIG. Accordingly, the frequency spectrum of the enlarged image D2A is obtained by removing the region R2AH on the high frequency side indicated by the oblique lines from the frequency spectrum of the zero insertion image D21A.

  Therefore, the enlarged image D2A mainly has a frequency component corresponding to a frequency lower than the Nyquist frequency Fn of the input image Din.

  Next, it will be described that the high frequency component image D3 mainly has a frequency component corresponding to a frequency equal to or higher than the Nyquist frequency Fn of the input image Din. The high frequency component image D3 is obtained by adding the intermediate image D32A and the intermediate image D32B. The intermediate image D32A has a frequency component corresponding to a frequency close to the Nyquist frequency Fn of the input image Din, and the intermediate image D32B is particularly input. Since it has a frequency component corresponding to a frequency higher than the Nyquist frequency Fn of the image Din and the frequency component of the intermediate images D32A and D32B is added in the high frequency component image D3, a frequency component equal to or higher than the Nyquist frequency Fn of the input image Din is added. Will have.

First, the frequency spectrum of the intermediate image D32A will be described.
FIGS. 11A to 11F are diagrams schematically showing the action when generating the intermediate image D32A, and FIG. 11A shows the frequency response of the high-frequency component image generating means 1, FIG. b) shows the frequency spectrum of the high frequency component image D1 (or D1h or D1v), and FIG. 11C shows the zero insertion image D21B (or D21Bh or D21Bv) generated by the zero insertion means 21B in the image enlargement means 2B. 11D shows the frequency spectrum of the enlarged image D2B (or D2Bh or D2Bv), FIG. 11E shows the frequency response of the high-frequency component image generating means 32A (or 32Ah or 32Av), and FIG. (F) shows the frequency spectrum of the intermediate image D32A (or D32Ah or D32Av) output from the high frequency component image generating means 32A. It represents the torque.

  First, the frequency response of the high frequency component image generation means 1 and the frequency spectrum of the high frequency component image D1 will be described. Since the high-frequency component image generating means 1 generates the high-frequency component image D1 using a high-pass filter that mainly passes a component having a predetermined frequency Fb or higher in the input image Din, as shown in FIG. The frequency response of the high frequency component image generation means 1 increases as the frequency increases. The input image Din having the frequency spectrum shown in FIG. 10A passes through the high-pass filter having the frequency response shown in FIG. 11A, so that a high frequency component image D1 is obtained. In the illustrated example, the frequency spectrum of the high frequency component image D1 is small in the low frequency region (region lower than the frequency Fb) and the high frequency region (region of the frequency Fb or higher) as shown in FIG. It will have a certain level of strength only.

  Next, the frequency spectrum of the zero insertion image D21B in the image enlarging means 2B will be described. Similarly to the description of the zero insertion means 21A of the image enlargement means 2A, since the aliasing occurs by the zero insertion means 21B, the frequency spectrum of the zero insertion image D21B in the image enlargement means 2B is as shown in FIG. It becomes like this.

Next, the frequency spectrum of the enlarged image D2B will be described.
When the enlarged image D2B is generated, the frequency spectrum on the high frequency component side of the zero insertion image D21B (for example, a component in a region higher than the predetermined frequency Fc) is removed by the low frequency component passage unit 22B. As shown in FIG. 11D, the frequency spectrum is obtained by removing the high frequency side region (region higher than the frequency Fc) R32AH.

  Finally, the frequency response of the high frequency component image generating means 32A and the frequency spectrum of the intermediate image D32A will be described. Since the high-frequency component image generating means 32A is a high-pass filter that mainly passes components of a predetermined frequency Fd or higher, the frequency response becomes higher as the frequency becomes higher as shown in FIG. The intermediate image D32A is generated when the enlarged image D2B having the frequency spectrum shown in FIG. 11D passes through the high-pass filter having the frequency response shown in FIG. Accordingly, as shown in FIG. 11 (f), the frequency response of the intermediate image D 32 A is further removed from the frequency spectrum of the enlarged image D 2 B shown in FIG. 11 (d) by the lower frequency side region (region lower than the frequency Fd) R 32 AL. It will be.

  Therefore, the intermediate image D32A mainly has a frequency component (frequency component from the frequency Fd to the frequency Fc) corresponding to a frequency close to the Nyquist frequency Fn of the input image Din.

Next, the frequency spectrum of the intermediate image D32B will be described.
12 (a) to 12 (c) are diagrams schematically showing the action when the intermediate image D32B is generated. FIG. 12 (a) shows a case where a high frequency component is generated by the nonlinear processing means 31 (or 31h or 31v). FIG. 12 (b) shows the frequency response of the high frequency component image generation means 32B, and FIG. 12 (c) shows the frequency spectrum of the intermediate image D32B.

  As will be described later, a high frequency component corresponding to a frequency higher than the Nyquist frequency Fn of the input image Din is generated in the nonlinear processed image D31. FIG. 12A is a diagram schematically showing the state. In the illustrated example, a component having a frequency of Fe or higher is generated. The intermediate image D32B is generated when the nonlinear processed image D31 passes through the high frequency component image generating means 32B. The high-frequency component image generating means 32B is a high-pass filter that mainly passes a component having the frequency Ff or higher, and its frequency response becomes higher as the frequency becomes higher as shown in FIG. Accordingly, as shown in FIG. 12C, the frequency spectrum of the intermediate image D32B is higher than the Nyquist frequency Fn of the input image Din because the low-frequency region R32BL is removed from the frequency spectrum of the nonlinear processed image D31. It corresponds to the frequency.

  The frequency spectrum of the intermediate image D32B will be described in more detail using FIGS. 13 (a) to 13 (e) and FIGS. 14 (a) to 14 (f). In addition, in order to simplify description, each is described as a one-dimensional signal.

  FIGS. 13A to 13E show a step edge signal representing an image (step image) in which component values such as luminance and saturation change stepwise, and the step edge signal is sampled at different sampling frequencies. The signal strength of the obtained signal and its high frequency component signal are shown. FIG. 13A shows a step edge signal.

FIG. 13B shows a signal obtained by sampling the step edge signal at the sampling interval S1, and FIG. 13C shows a high frequency component of the signal obtained by sampling the step edge signal at the interval S1, and FIG. d) shows a signal obtained by sampling the step edge signal at the sampling interval S2, and FIG. 13 (e) shows a high frequency component of the signal obtained by sampling the step edge signal at the interval S1.
Note that the sampling interval S1 is shorter than the sampling interval S2, and shortening the sampling interval is the same as enlarging the image.
Note that the relationship between the sampling intervals S1 and S2 for the step edge signal and the high-frequency component, which will be described with reference to FIG. 13, does not depend on the combination of specific sampling intervals. It is the same as the sampling interval, and the sampling interval S1 is half of the sampling interval S2.

  As shown in FIGS. 13B, 13C, 13D, and 13E, the center of the edge appears as a zero cross point Z in the high frequency component signal (FIGS. 13C and 13E). Further, as is apparent from a comparison between FIGS. 13B and 13C and FIGS. 13D and 13E, the slope of the high-frequency component signal before and after the zero-cross point Z decreases as the sampling interval decreases ( Alternatively, the position of the point giving the local maximum and minimum values of the high-frequency component near the zero-cross point Z also approaches the zero-cross point Z (according to the enlargement of the image).

  Therefore, when enlarging the image, the high frequency component of the input image Din is taken out, the change is made steep near the zero cross, and the point giving the local maximum and minimum values near the zero cross is brought close to the zero cross point. A high-frequency component that is not included in the resolution of Din (or higher than the Nyquist frequency of the input image Din) is generated, thereby making it possible to sharpen the edge.

  FIGS. 14A to 14F schematically show a procedure of high frequency component generation by the high frequency component image generation means 1, the image enlargement means 2B, the nonlinear processing means 31, and the high frequency component image generation means 32B. 14A shows an image (step image) in which component values such as luminance and saturation change stepwise, FIG. 14B shows an input image Din corresponding to the step image, and FIG. 14C shows FIG. The high-frequency component image D1, FIG. 14D shows the enlarged image D2B, FIG. 14E shows the nonlinear processed image D31, and FIG. 14F shows the intermediate image D32B.

  14A to 14F, the coordinate P3 is a pixel corresponding to the boundary of the region (low level side) where the signal intensity is low in the vicinity of the edge, and the coordinate P4 is a region (high) where the value is high. This is a pixel corresponding to the boundary on the level side.

The input image Din and the high-frequency component image D1 corresponding to the step image are as described in FIGS. 13A to 13E, and the description thereof is omitted. First, the enlarged image D2B will be described.
In the high frequency component image D1, the local maximum value in the vicinity of the zero cross point Z appears at the boundary on the high level side. Therefore, the local maximum value appears at the pixel represented by the coordinate P4, and conversely the local maximum value appears. Since the minimum value appears at the boundary on the low level side, a local minimum value appears at the pixel represented by the coordinate P3.

  The enlarged image D2B (or D2Bh or D2Bv) is passed through the low-frequency component after inserting one pixel per pixel (image D1) with a pixel value of 0 into the high-frequency component image D1 by the zero insertion means 21B. It is obtained by extracting the low frequency component by means 22B. Extracting the low frequency component is the same as obtaining an average pixel value in the local region for the high frequency component image D1 (FIG. 14C), and the enlarged image D2B (or D2Bh or D2Bv) is shown in FIG. As shown in d), the signal has the same shape as the high-frequency component image D1 and has an increased sampling number.

  Since the image is enlarged, a new pixel represented by the coordinate P1 between the zero-cross point Z and the pixel represented by the coordinate P3 is a new coordinate between the zero-cross point Z and the pixel represented by the coordinate P4. A pixel represented by P2 appears. Also in the enlarged image D2B, the local maximum value in the vicinity of the zero cross point Z appears in the pixel represented by the coordinate P4, and the local minimum value appears in the pixel represented by the coordinate P3.

  Next, the nonlinear processed image D31 will be described. The nonlinear processed image D31 is output as a result of the nonlinear processing means 31 detecting the zero cross point Z in the enlarged image D1 and amplifying the pixel values of the pixels before and after the zero cross point Z. Accordingly, in the nonlinear processed image D31 (or D31h or D31v), the pixel values of the pixels represented by the coordinates P1 and P2 are amplified, and the nonlinear processed image D31 has a signal as shown in FIG. Become.

  Finally, the intermediate image D32B will be described. The intermediate image D32B (FIG. 14 (f)) is obtained by extracting the high frequency components of the nonlinear processed image D31 (FIG. 14 (e)) by the high frequency component image generating means 32B. The high frequency component can be extracted by subtracting the low frequency component of the input signal (or a simple average value or a weighted average value of pixel values in the local region) from the input signal.

  In the nonlinear processed image D31 (FIG. 14E), the pixel values of pixels before and after the zero cross point Z (pixels represented by coordinates P1 and P2) are amplified by the signal amplifying units 312h and 312v. The difference from the average pixel value in the local area becomes large. On the other hand, since the pixel values of other pixels near the zero cross point are not amplified, the difference from the average pixel value in the local region is a small value. Therefore, compared with the enlarged image D2B (FIG. 14D), in the intermediate image D32B (FIG. 14F), the points that give the local maximum value and the minimum value in the vicinity of the zero-cross point Z are coordinates P2 and P1, respectively. And approaches the zero cross point Z more. In addition, since the point that gives the local maximum and minimum values approaches the zero-cross point Z, the signal change near the zero-cross point also becomes abrupt.

  As described above, this means that the intermediate image D32B includes high frequency components that are not included in the resolution of the input image Din. In other words, the nonlinear processing means 31 amplifies pixel values around the zero cross point of the enlarged image D2B, thereby generating a high frequency component corresponding to a frequency higher than the Nyquist frequency Fn of the input image Din.

  Further, since the intermediate image D32B is generated by extracting the high frequency component generated by the nonlinear processing means 31 by the high frequency component image generating means 32B, the high frequency component corresponding to a frequency higher than the Nyquist frequency Fn of the input image Din is obtained. It becomes an image with.

  The frequency components of the enlarged image D2A, intermediate image D32A, and intermediate image D32B are illustrated in FIG. The enlarged image D2A mainly includes a frequency component corresponding to a region RL corresponding to a frequency lower than the Nyquist frequency Fn of the input image Din. On the other hand, the intermediate image D32A includes a frequency component corresponding to a region RM corresponding to a frequency close to the Nyquist frequency Fn of the input image Din, and the intermediate image D32B corresponds to a frequency higher than the Nyquist frequency Fn of the input image Din. The frequency component corresponding to the area | region RH to be included is contained.

  If the high frequency component image D3 is generated by adding the intermediate image D32A and the intermediate image D32B, the high frequency component image D3 includes both the frequency component of the intermediate image D32A and the frequency component of the intermediate image D32B. become. The intermediate image D32A includes a frequency component corresponding to a frequency close to the Nyquist frequency Fn of the input image Din, and the intermediate image D32B includes a frequency component corresponding to a frequency higher than the Nyquist frequency Fn of the input image Din. Therefore, the high frequency component image D3 includes frequency components equal to or higher than the Nyquist frequency Fn of the input image Din. Then, by adding the high frequency component image D3 to the enlarged image D2A to obtain the output image Dout, it becomes possible to give the output image Dout a frequency component equal to or higher than the Nyquist frequency Fn of the input image Din. The resolution can be increased.

  FIGS. 16A to 16D are diagrams for explaining the above effect from another viewpoint. FIG. 16A shows a step edge signal. In the edge shown in FIG. 16A, the luminance on the left side of the edge center is lower than that on the right side. That is, the left side from the edge center is the low level side, and the right side is the high level side. FIG. 16B shows an input image Din obtained by sampling the step edge signal at the sampling interval S2.

  FIG. 16C shows an enlarged image D2A obtained for the input image Din shown in FIG. Since the enlarged image D2A is obtained by performing an interpolation operation on the input image Din, the sampling interval is S1, which is half of S2, but the change in the signal in the vicinity of the edge remains gentle, and the boundary on the low level side Is a pixel represented by the coordinate P3, and the boundary on the high level side remains the pixel represented by the coordinate P4.

  FIG. 16D shows an image obtained by sampling the step edge signal at the sampling interval S1 (indicated by the symbol Din as in the image of FIG. 16B). The boundary on the low level side is the pixel represented by the coordinate P1, and the boundary on the high level side is the pixel represented by the coordinate P2, and the change in the signal near the edge is abrupt compared to the enlarged image D2A. ing.

  By adding the intermediate image D32A and the intermediate image D32B, the inclination of the signal in the vicinity of the edge of the enlarged image D2 is corrected, and an image close to the image obtained by sampling the step edge signal shown in FIG. It can be obtained and the resolution of the image can be enhanced.

  As described above, by adding the intermediate images D32A and D32B (or high frequency components) to the enlarged image D2A, the sharpness of the image can be increased and the image quality can be improved. In the image processing apparatus of the present invention, the high frequency component addition processing is performed in each of the horizontal direction and the vertical direction of the image, so that it is possible to increase the sharpness of the edge in any direction. That is, the sharpness can be increased by the horizontal intermediate images D32Ah and D33Bh in the horizontal direction of the image and the vertical intermediate images D32Av and D32Bv in the vertical direction.

  However, if the horizontal intermediate image D32Ah and the vertical intermediate image D32Av or the horizontal intermediate image D32Bh and the vertical intermediate image D32Bv are simply added, the correction amount is not uniform depending on the direction of the edge included in the input image. Artifacts occur in the processed image.

FIGS. 17A to 17K are schematic diagrams for explaining the cause of the artifact and the effect of the adding means 2C. As shown in FIG. 17A, FIG. 18A, and FIG. 19A, the input image includes (a) a horizontal edge (FIG. 17 (a)).
(B) When a vertical edge is included (FIG. 18A),
(C) When an oblique edge is included (FIG. 19 (a)),
think of. In each case, the intensity of the signal when the pixel signal is supplied in the horizontal direction from left to right (the signal to be processed by the horizontal high-frequency component image generating unit 1h) is shown in FIG. 18 (b) and 19 (b), the signals when the pixel signals are supplied in the order from top to bottom in the vertical direction (subject to processing by the vertical high-frequency component image generating means 1v). The intensity of the signal is shown in FIGS. 17 (c), 18 (c), and 19 (c).
When considering the edge in each case by dividing it into a horizontal component and a vertical component, in (a) and (b), the step edge is composed of only one of the horizontal component and the vertical component. In (c), the step edge is composed of both the horizontal component and the vertical component.

  17 (d), 18 (d), and 19 (d) are horizontal images processed in the horizontal direction with respect to the input images shown in FIGS. 17 (b), 18 (b), and 19 (b). FIG. 17E, FIG. 18E, and FIG. 19E show the signal intensity of the direction high-frequency component image D1h, respectively, as shown in FIG. 17C, FIG. 18C, and FIG. 19C. The signal intensity of the vertical high frequency component image D1v processed in the vertical direction with respect to the input image is represented.

  In the vicinity of the edge, the absolute value of the signal intensity of the horizontal high-frequency component image D1h is large in (A) and (C) and is zero in (B). In addition, in the vicinity of the edge, the absolute value of the signal intensity of the vertical high frequency component image D1v is zero in (A) and large in (B) and (C).

  FIGS. 17 (f), 18 (f), and 19 (f) are obtained for the horizontal high-frequency component image D1h shown in FIGS. 17 (d), 18 (d), and 19 (d). FIG. 17 (g), FIG. 18 (g), and FIG. 19 (g) show the signal strength of the enlarged image D2Bh, and FIG. 17 (e), FIG. 18 (e), and FIG. The signal intensity of the enlarged image D2Bv obtained with respect to the frequency component image D1v is represented.

  In the vicinity of the edge, the feature of the absolute value of the signal intensity of the enlarged image D2Bh is the same as that of the horizontal high-frequency component image D1h, and the feature of the absolute value of the signal intensity of the enlarged image D2Bv is the same as that of the vertical high-frequency component image D1v. It is.

  FIGS. 17 (h), 18 (h), and 19 (h) are horizontal intermediate images obtained for the enlarged image D2Bh shown in FIGS. 17 (f), 18 (f), and 19 (f). 17 (i), FIG. 18 (i), and FIG. 19 (i) show the signal intensity of D32Ah, and the enlarged images D2Bv shown in FIG. 17 (g), FIG. 18 (g), and FIG. Represents the signal intensity of the vertical intermediate image D32Av obtained in this way.

When a signal having the shape shown in FIGS. 17 (f), 18 (g), 19 (f) and 19 (g) is an input signal, the shape of the signal obtained by applying a high-pass filter to the input signal is the input signal. It becomes almost the same. As shown in FIGS. 17 (g) and 18 (f), when the value of the input signal is substantially zero and constant, the output value of the high-pass filter is also substantially zero, and the shape thereof is substantially the same as the input signal. .
Therefore, in the vicinity of the edge, the characteristic of the absolute value of the signal intensity of the horizontal intermediate image D32Ah is the characteristic of the absolute value of the horizontal high frequency component image D1h and the absolute value of the signal intensity of the vertical intermediate image D32Av is the vertical high frequency component. Each is the same as the image D1v.

  FIGS. 17 (j), 18 (j), and 19 (j) are horizontal intermediate images obtained for the enlarged image D2Bh shown in FIGS. 17 (f), 18 (f), and 19 (f). 17 (k), FIG. 18 (k), and FIG. 19 (k) show the signal intensity of D32Bh, and the enlarged image D2Bv shown in FIG. 17 (g), FIG. 18 (g), and FIG. Represents the signal intensity of the vertical intermediate image D32Bv obtained in this way.

  For the intermediate image D32B obtained when the signals having the shapes shown in FIGS. 17 (f), 18 (g), 19 (f) and 19 (g) are input as the enlarged image D2B, FIG. 14 (a). As described in (f). Further, as shown in FIGS. 17G and 18F, when the pixel value of the enlarged image D2B is almost zero and constant, the pixel value of the intermediate image D32B is also almost zero. After all, in the vicinity of the edge, the feature of the absolute value of the signal intensity of the horizontal intermediate image D32Bh is the feature of the absolute value of the signal intensity of the horizontal high frequency component image D1h and the vertical intermediate image D32Bv of the vertical high frequency. This is the same as the component image D1v.

  From the above relationship, if the horizontal intermediate image D32Ah and the vertical intermediate image D32Av are simply added, the correction amount for the edge in the oblique direction becomes about twice as large as the edge in the horizontal direction and the vertical direction. You can see that In addition, if the horizontal intermediate image D32Bh and the vertical intermediate image D32Bv are simply added, the correction amount for the edge in the oblique direction becomes about twice as large as the edge in the horizontal direction and the vertical direction. I understand. As a result, the correction intensity becomes non-uniform depending on the direction of the edge, and problems such as an overshoot becoming large at the edge in the oblique direction occur.

  Therefore, in the present invention, the horizontal intermediate image D32Ah and the vertical intermediate image D32Av, or the horizontal intermediate image D32Bh and the vertical intermediate image D32Bv are not simply added, and these images are set in accordance with the edge direction. Multiply by weighting factor and add. As described above, the adding means 34 calculates the difference between the absolute values of the enlarged image D2Bh and the enlarged image D2Bv as the edge direction estimated amount D33C1 in the edge direction estimating means 33C1 provided in the addition ratio determining means 33C. FIG. 20 shows a table summarizing these relationships for each edge direction. As is apparent from the figure, the edge direction estimation amount D33C1 is an amount corresponding to the edge direction, and is a positive value that is relatively large near the edge in the horizontal direction, and a value close to 0 near the edge in the oblique direction. Thus, it is a negative value in the vicinity of the edge in the vertical direction, and its absolute value takes a relatively large value.

  In the edge direction estimation means 33C1, it is necessary to separately provide means such as a two-dimensional filter for detecting the edge direction by estimating the edge direction from the difference between the absolute values of the enlarged image D2Bh and the enlarged image D2Bv. Therefore, an increase in circuit scale can be prevented.

  In accordance with this difference, the weighting coefficient determination means 33C2 determines the weighting coefficient D33C from the relationship shown in FIG. 9 or Expression (2), and calculates the intermediate image D33A according to Expression (3).

Specifically, the weight coefficient D33C is increased as the difference becomes larger (when the edge direction is horizontal). From Equation (3), it can be seen that in this case, the contribution of the horizontal intermediate image D32Ah is large in the calculation of the intermediate image D33A. On the contrary, the weight coefficient D33C is reduced as the absolute value of the difference is larger (when the edge direction is vertical). In this case, in the calculation of the intermediate image D33A, it can be seen that the contribution of the vertical intermediate image D32Av is large.
Furthermore, the weight coefficient D33C approaches an intermediate value in a possible range as the difference approaches 0 (when the edge direction is oblique). In this case, in the calculation of the intermediate image D33A, the contribution of the horizontal intermediate image D32Ah is approximately half that when the edge direction is horizontal, and the contribution of the vertical intermediate image D32Av is also when the edge direction is vertical. It is almost half. Therefore, the intensity of adding the high frequency component is not doubled at the edge in the diagonal direction as when the horizontal intermediate image D32Ah and the vertical intermediate image D32Av are simply added.

  Further, since the same argument holds for the calculation of the intermediate image D33B, the high frequency component is added at the edge in the oblique direction as when the horizontal intermediate image D32Bh and the vertical intermediate image D32Bv are simply added. There is no doubling of strength. In the example shown in FIG. 1, the weighting factor D33C determined by the addition ratio determining unit 33C is also used when adding the horizontal intermediate image D32Bh and the vertical intermediate image D2Bv of the intermediate image D2B.

  The method of determining the weighting coefficient D33C in the weighting coefficient determining means 33C2 is not limited to the relationship shown in FIG. 9 or the expression (2) as long as it fulfills the above-mentioned purpose. For example, the relationship defined by a smooth curve is used. It may be used. Further, such a curve may be approximated by a broken line. In general, the weighting coefficient D33C for the horizontal direction component may be determined based on a characteristic that monotonously increases with respect to the edge direction estimation amount D33C1.

  Further, the edge direction estimation means 33C1 calculates the above difference as the edge direction estimation amount D33C1, but if it is an amount correlated with the edge direction, it is calculated by any of various other relational expressions. An amount may be used. In that case, the weighting factor determination means 33C2 is also changed and implemented so as to achieve the above-mentioned purpose according to the edge direction estimation amount D33C1.

  In summary, an intermediate image D33A may be generated by weighting and adding the horizontal intermediate image D32Ah and the vertical intermediate image D32Av according to a weighting factor determined according to the edge direction. The horizontal intermediate image D32Bh and the vertical intermediate image D32Bv are generated. What is necessary is just to produce | generate the intermediate image D33B weighted and added according to the weighting coefficient D33C determined according to the direction of an edge.

  In this manner, the image processing apparatus according to the first embodiment obtains an output image Dout having a high resolution while suppressing the occurrence of artifacts depending on the edge direction.

  21 and 22 show a modification of the first embodiment. In the modification shown in FIG. 21, the addition ratio determination unit 33C calculates the weighting coefficient D33C using the horizontal direction intermediate image D32Ah and the vertical direction intermediate image D32Av. In the modification shown in FIG. 22, the addition ratio determination unit 33C calculates the weighting coefficient D33C using the horizontal direction intermediate image D32Bh and the vertical direction intermediate image D32Bv.

  According to FIGS. 17A to 19K, the size of the edge direction and the enlarged image D2Bh, the horizontal intermediate image D32Ah, the horizontal intermediate image D32Bh, the edge direction and the enlarged image D2Bv, and the vertical intermediate image D32Av. The magnitude relationship of the vertical intermediate image D32Bv is substantially the same. Therefore, the edge direction estimation means 33C1 in the addition ratio determination means 33C uses the horizontal direction intermediate image D32Ah, the vertical direction intermediate image D32Av, the horizontal direction intermediate image D32Bh, and the vertical direction intermediate image D32Bv instead of the enlarged images D2Bh and D2Bv. It is also possible to calculate the direction estimation amount D33C1. That is, the absolute value of each pixel of the horizontal intermediate image D32Ah or the horizontal intermediate image D32Bh is dH, the absolute value of each pixel value of the vertical intermediate image D32Av or the vertical intermediate image D32Bv is dV, and the difference dH−dV is The edge direction estimation amount D33C1 can be used. At this time, the operation of the weighting factor determination means 33C2 need not be particularly changed.

  As described above, the high-frequency component image processing unit 3 processes the image enlarging unit 2B obtained by enlarging the high-frequency component image D1 generated by the high-frequency component image generating unit 1 with the image enlarging unit 2B. A high frequency component image D3 including a frequency component corresponding to a frequency equal to or higher than the Nyquist frequency Fn of the image Din can be obtained. Then, in the adding means 4, an enlarged image D2A including a frequency component in a region corresponding to a frequency lower than the Nyquist frequency Fn of the input image Din and a high frequency including a frequency component in a region corresponding to a frequency equal to or higher than the Nyquist frequency Fn of the input image Din. Since the enlarged image Dout is generated by adding the frequency component images D3, a high frequency component can be sufficiently given to the enlarged image Dout, and an enlarged image Dout with a sense of resolution can be obtained.

Further, the high frequency component image generation means 1 generates a horizontal high frequency component image D1h obtained by extracting the horizontal high frequency component and a vertical high frequency component image D1v obtained by extracting the vertical high frequency component. It is possible to generate a frequency component corresponding to a frequency equal to or higher than the Nyquist frequency Fn of the input image Din in any one of the horizontal direction and the vertical direction of the image. That is, by applying a horizontal high-pass filter to the enlarged image D2Bh obtained by enlarging the horizontal high-frequency component image D1h by the image enlarging means 2Bh by the horizontal high-frequency component image generating means 32Ah, the input image Din An intermediate image D32Ah having a frequency component corresponding to a frequency close to the Nyquist frequency Fn is generated, and a vertical high frequency component image generating unit is generated with respect to the enlarged image D2Bv obtained by enlarging the vertical high frequency component image D1v by the image enlarging unit 2Bv. By applying a high-pass filter in the vertical direction at 32 Av, an intermediate image D32Av having a frequency component corresponding to a frequency close to the Nyquist frequency Fn of the input image Din in the vertical direction is generated.
Further, by applying high-pass filters having different characteristics in the horizontal direction and the vertical direction, frequency components corresponding to frequencies close to the Nyquist frequency Fn are included in the horizontal direction and the vertical direction in different levels. You can also

Further, by applying a high-pass filter to the non-linearly processed image D31h generated by performing non-linear processing on the enlarged image D2Bh by the horizontal non-linear processing unit 31h, the horizontal direction high-frequency component image generating unit 32Bh applies an input in the horizontal direction. An intermediate image D32Bh having a frequency component corresponding to a frequency higher than the Nyquist frequency Fn of the image Din is generated, and the non-linearly processed image D31v generated by performing non-linear processing on the enlarged image D2Bv by the non-linear processing means 31v in the vertical direction. By applying a high-pass filter in the vertical high frequency component image generation means 32Bv, an intermediate image D32Bv having a frequency component corresponding to a frequency higher than the Nyquist frequency Fn of the input image Din in the vertical direction is generated.
Further, by performing nonlinear processing and high-pass filtering with different characteristics in the horizontal direction and the vertical direction, frequency components corresponding to frequencies higher than the Nyquist frequency Fn can be included in different levels in the horizontal direction and the vertical direction. .

  Then, by generating an intermediate image D33A obtained by weighted addition of the horizontal intermediate image D32Ah and the vertical intermediate image D32Av according to the edge direction, generation of artifacts depending on the edge direction is suppressed.

  Further, by generating an intermediate image D33B obtained by weighted addition of the horizontal intermediate image D32Bh and the vertical intermediate image D32Bv according to the edge direction, generation of artifacts depending on the edge direction is suppressed.

Note that the description has been made assuming that the enlargement ratio when generating the enlarged image Dout from the input image Din is double in both the horizontal direction and the vertical direction, but the enlargement ratio is not limited to double. That is, the image enlargement unit 2A generates an enlarged image D2A obtained by enlarging the input image Din to a desired magnification both in the horizontal direction and the vertical direction, and the high frequency component generation unit 1 generates the high frequency component image D1 based on the input image Din. The image enlarging means 2B generates an enlarged image D2B obtained by enlarging the high frequency component image D1 to a desired magnification (the same magnification as the magnification in the image enlarging means 2A) in both the horizontal direction and the vertical direction. The processing means 3 generates the high frequency component image D3 based on the enlarged image D2B, and the adding means 4 adds the enlarged image D2A and the high frequency component image D3 to obtain the final enlarged image Dout.
Furthermore, as described above, the horizontal enlargement factor and the vertical enlargement factor may not be the same, and enlargement may be performed only in one of the horizontal direction and the vertical direction.

  In the above description, the amplification factor is increased only for one pixel before and after the zero cross point in both the horizontal direction and the vertical direction. However, the example of the amplification factor control is not limited to this, and is appropriately changed according to, for example, the enlargement factor ( It can also be set to a value according to the enlargement ratio).

  Hereinafter, a case where the enlargement ratio is different from the above example will be described with reference to FIGS. 23 (a) to 23 (e) and FIGS. 24 (a) to 24 (f).

  FIG. 23A shows a step edge signal, FIG. 23B shows a signal obtained by sampling the step edge signal at the sampling interval S1, and FIG. 23C shows a signal obtained by sampling the step edge signal at the sampling interval S1. FIG. 23D shows a signal obtained by sampling a step edge signal at an interval S3 that is three times the interval S1, and FIG. 23E shows a step edge signal sampled at an interval S3. Represents the high frequency component of the resulting signal. In FIGS. 23A to 23E, pixel positions PL1 and PR1 represent step edge signal boundaries (points at which the brightness changes). Usually, in a signal representing a high-frequency component of an image obtained by sampling the step edge signal, the position of the pixel that gives the local maximum value and minimum value in the vicinity of the zero cross point Z substantially coincides with the position of the boundary of the step edge signal. .

  24A to 24F show high frequency component generation by the high frequency component image generation means 1, the image enlargement means 2B, the nonlinear processing means 31, and the high frequency component image generation means 32B when the enlargement ratio is three times. FIG. 24A is an image (step image) in which component values such as luminance and saturation change stepwise, and FIG. 24B is an input corresponding to the step image. Image Din, FIG. 24C shows the high frequency component image D1, FIG. 24D shows the enlarged image D2B, FIG. 24E shows the nonlinear processed image D31, and FIG. 24F shows the intermediate image D32B. In addition, in order to simplify description, each is described as a one-dimensional signal.

As shown in FIG. 24D, pixel positions PL1 and PR1 that give local maximum and minimum values near the zero-cross point Z in the enlarged image D2B are the positions of the boundary of the step edge signal in the enlarged image D2B. Almost matches. Normally, in the enlargement method used in the description of the present embodiment, the positions of PL1 and PR1 do not change, and the number of pixels existing between the positions represented by PL1 and PR1 and the zero-cross point Z increases. Further, the number of pixels existing between the positions represented by PL1 and PR1 and the zero-cross point Z increases as the enlargement ratio when generating the enlarged image D2B is increased (or the sampling interval is shortened).
On the other hand, in the signal representing the high frequency component of the image obtained by sampling the step edge signal at a short sampling interval, the position of the pixel giving the local maximum value and minimum value near the zero cross point Z is closer to the zero cross point Z, and the zero cross A pixel closer to the point than Z has a larger amplitude of a signal representing a high frequency component.

Accordingly, when generating the nonlinear processed image D31 by amplifying only the signals before and after the zero cross point Z, it is preferable to perform processing so that the amplitude becomes larger as the pixel is closer to the zero cross point Z than PL1 and PR1, for example, positions PL1, PR1 By amplifying the pixel value of the enlarged image D2B at a pixel closer to the zero cross point Z, the larger the pixel closer to the zero cross point Z, and at a gain of 1 for a pixel farther from the zero cross point Z than PL1 and PR1, FIG. As shown in (e), it is possible to generate a nonlinear processed image D31 having a larger amplitude as the pixel is closer to the zero cross point Z.
An intermediate image D32B corresponding to the sampling interval S1 as shown in FIG. 24F can be generated by extracting only the high frequency component from the enlarged image D2B generated in this way by high-pass filter processing.

  In summary, since the number of pixels existing between the positions PL1 and PR1 and the zero cross point Z varies depending on the enlargement ratio at the time of generating the enlarged image D2B, the zero cross point Z is generated when generating the nonlinear processed image D31 from the enlarged image D2B. The number of pixels whose amplification factor is greater than 1 before and after may be changed in accordance with the enlargement factor of the image. Further, the amplification factors for these pixels may be changed according to the pixels, for example, according to the distance from the zero cross point Z. For example, the amplification factor may be increased as the pixel is closer to the zero cross point Z.

Further, even by adding the intermediate image D33B to the enlarged image D2A, it is possible to give a high frequency component in a region higher than the Nyquist frequency Fn, so that it is possible to increase the resolution of the image. That is, the high frequency component image processing means 3 does not include the high frequency component image generation means 32A, includes the non-linear processing image generation means 30, and the addition means 34 does not include the horizontal / vertical addition means 33A and the addition means 33D, thereby determining the addition ratio. A configuration including means 33C and horizontal / vertical addition means 33B may be adopted.
In this case, the intermediate image D33B output from the horizontal / vertical addition means 33B is output as the high frequency component image D3.

  The effect is maximized by providing both the horizontal and vertical adding means 33A and 33B, but a configuration having only one of them may be used. For example, when the adding unit 34 does not include the horizontal / vertical adding unit 33A, the horizontal intermediate image D32Ah, the vertical intermediate image D32Av, and the intermediate image D33B are input to the adding unit 33D. In addition means 33D adds horizontal intermediate image D32Ah, vertical intermediate image D32Av, and intermediate image D33B. Conversely, a configuration not including the horizontal / vertical addition means 33B is also possible, and in this case, the addition means 33D adds the intermediate image D33A, the horizontal direction intermediate image D32Bh, and the vertical direction intermediate image D32Bv.

Embodiment 2. FIG.
In the first embodiment, the present invention has been described as being realized by hardware. However, part or all of the configuration shown in FIG. 1 may be realized by software, that is, by a programmed computer. Processing in that case will be described with reference to FIG. 25 and FIGS.

FIG. 25 shows an image processing apparatus according to the second embodiment. The illustrated image processing apparatus includes a CPU 11, a program memory 12, a data memory 13, and a bus 14 for connecting them.
The CPU 11 operates according to a program stored in the program memory 12. Various data are stored in the data memory 13 in the course of operation. The enlarged image Dout generated as a result of the processing is supplied to the display unit 9 via the interface 15 and used for display by the display unit 9.
Hereinafter, processing performed by the CPU 11 will be described with reference to FIGS.

  FIG. 26 is a diagram illustrating a flow of an image processing method performed by the image processing apparatus in FIG. 25. The image processing method illustrated in FIG. 26 includes an image enlargement step ST2A, a high-frequency component image generation step ST1, and an image enlargement. Step ST2B, high frequency component image processing step ST3, and addition step ST4 are included.

  The image enlargement step ST2A generates an enlarged image D2A obtained by enlarging the input image Din input in an image input step (not shown) by the same process as the image enlargement means 2A in FIG.

  As shown in FIG. 27, the high frequency component image generation step ST1 includes a horizontal direction high frequency component image generation step ST1h and a vertical direction high frequency component image generation step ST1v. In the horizontal direction high frequency component image generation step ST1h, the input image Din is processed in the same manner as the horizontal direction high frequency component image generation unit 1h in FIG. 1 to generate a horizontal direction high frequency component image D1h. On the other hand, in the vertical direction high frequency component image generation step ST1v, the input image Din is processed in the same manner as the vertical direction high frequency component image generation means 1v in FIG. 1 to generate the vertical direction high frequency component image D1v.

As shown in FIG. 28, the image enlargement step ST2B has an image enlargement step ST2Bh and an image enlargement step ST2Bv.
In the image enlarging step ST2Bh, the horizontal high-frequency component image D1h generated in the horizontal high-frequency component image generating step ST1h is subjected to the same processing as the image enlarging means 2Bh in FIG. 1 to generate an enlarged image D2Bh.
In the image enlarging step ST2Bv, the vertical high frequency component image D1v generated in the vertical high frequency component image generating step ST1v is subjected to the same processing as the image enlarging means 2Bv in FIG. 1 to generate an enlarged image D2Bv.

Next, the operation of the high frequency component image processing step ST3 will be described.
As shown in FIG. 29, the high frequency component image processing step ST3 includes a high frequency component image generation step ST32A, a non-linear processing image generation step ST30, a high frequency component image correction step ST33A, a high frequency component image correction step ST33B, and an addition step. It has ST34.
The high frequency component passing step ST32A includes a horizontal direction high frequency component image generation step ST32Ah and a vertical direction high frequency component image generation step ST32Av.

The nonlinear process image generation step ST30 includes a nonlinear process step ST31 and a high frequency component image generation step ST32B.
The nonlinear processing step ST31 includes a horizontal nonlinear processing step ST31h and a vertical nonlinear processing step ST31v.
The high frequency component image generation step ST32B includes a horizontal direction high frequency component image generation step ST32Bh and a vertical direction high frequency component image generation step ST32Bv.

  The adding step ST34 includes an adding ratio determining step ST33C, horizontal and vertical adding steps ST33A, ST33B, and an adding step ST33D.

In the horizontal high-frequency component passing step ST32Ah, the enlarged image D2Bh generated in the image enlargement step ST2Bh is subjected to the same processing as the horizontal high-frequency component image generating unit 32Ah in FIG. 1 to generate the horizontal intermediate image D32Ah. . In the vertical high-frequency component passing step ST32Av, the enlarged image D2Bv generated in the image enlargement step ST2Bv is subjected to the same processing as the vertical high-frequency component image generating unit 32Av in FIG. 1 to generate the vertical intermediate image D32Av. .
In the high frequency component image generation step ST32A, an intermediate image D32A composed of the horizontal intermediate image D32Ah and the vertical intermediate image D32Av is generated.
Thus, in the high frequency component passing step ST32A, the same operation as the high frequency component image generating means 32A of FIG. 1 is performed.

As shown in FIG. 30, the horizontal non-linear processing step ST31h includes a zero-cross determination step ST311h and a signal amplification step ST312h.
The operation of the horizontal nonlinear processing step ST31h is as follows.
First, in the zero cross determination step ST311h, a change in the pixel value in the enlarged image D2Bh generated in the image enlargement step ST2Bh is confirmed along the horizontal direction. A location where the pixel value changes from a positive value to a negative value or from a negative value to a positive value is regarded as a zero cross point, and pixels located on the left and right sides of the zero cross point are specified. In the signal amplification step ST312h, the pixel values of the pixels located on the left and right of the zero cross point specified in the zero cross determination step ST311h in the enlarged image D2Bh are amplified, and an image obtained as a result is generated as the nonlinear processed image D31h.

As shown in FIG. 31, the vertical nonlinear processing step ST31v includes a zero-cross determination step ST311v and a signal amplification step ST312v.
The operation of the vertical nonlinear processing step ST31v is as follows.
First, in the zero cross determination step ST311v, a change in pixel value in the enlarged image D2Bv generated in the image enlargement step ST2Bv is confirmed along the vertical direction. A portion where the pixel value changes from a positive value to a negative value or from a negative value to a positive value is regarded as a zero cross point, and pixels located above and below the zero cross point are specified. In the signal amplification step ST312v, the pixel values of the pixels located above and below the zero-cross point specified in the zero-cross determination step ST311v in the enlarged image D2Bv are amplified, and the resulting image is generated as the nonlinear processed image D31v.

In the high frequency component image generation step ST31, a nonlinear processed image D31 composed of the vertical direction nonlinear processed image D31v is generated.
Thus, in the nonlinear processing step ST31, the same operation as that of the nonlinear processing means 31 of FIG. 1 is performed.

In the horizontal high-frequency component passing step ST32Bh, a high-pass filter is applied to the horizontal nonlinear processed image D31h generated in the horizontal nonlinear processing step ST31h to generate a horizontal intermediate image D32Bh. In the vertical direction high-frequency component passing step ST32Bv, a high-pass filter is applied to the vertical nonlinear processing image D31v generated in the vertical nonlinear processing step ST31v to generate a vertical intermediate image D32Bv.
In the high frequency component image generation step ST32B, an intermediate image D32B composed of the horizontal intermediate image D32Bh and the vertical intermediate image D32Bv is generated.
Thus, in the high frequency component passage step ST32B, the same operation as that of the high frequency component image generating means 32B in FIG. 1 is performed.

  The horizontal nonlinear processing step ST31h and the horizontal high frequency component image generation step ST32Bh constitute a horizontal nonlinear processing image generation step, and the vertical nonlinear processing step ST31h and the vertical high frequency component image generation step ST32Bv are vertical nonlinear. A processed image generation step is configured.

Next, the detailed operation of the adding step ST34 will be described.
First, in the addition ratio determination step ST33C of the addition step ST34, the weight coefficient D33C is obtained. Here, the addition ratio determination step ST33C includes an edge direction estimation step ST33C1 and a weight determination step ST33C2, as shown in FIG.
In the edge direction estimation step ST33C1, when the absolute value of each pixel of the enlarged image D2Bh is represented by dH and the absolute value of each pixel of the enlarged image D2Bv is represented by dV, the difference dH−dV between these two values is used as the edge direction estimation amount D33C1. Output. This operation is the same as that of the edge direction estimation means 33C1 in FIG.
The weight determination step DT33C2 calculates a weight coefficient D33C according to the equation (2). This operation is the same as the weight determination means 33C2 in FIG.

  Next, in the horizontal / vertical addition step ST33A, an intermediate image D33A obtained by weighted addition of the horizontal direction intermediate image D32Ah and the vertical direction intermediate image D32Av is obtained. The details of this operation are the same as those of the horizontal / vertical adding means 33A shown in FIG.

  Next, in the horizontal / vertical addition step ST33B, an intermediate image D33B obtained by weighted addition of the horizontal direction intermediate image D32Bh and the vertical direction intermediate image D32Bv is obtained. The details of this operation are the same as those of the horizontal / vertical addition means 33B in FIG.

  Finally, the addition step ST33D generates a high frequency component image D3 obtained by adding the intermediate image D33A and the intermediate image D33B. Since this operation can be the same as that of the adding means 33D of FIG. 1, detailed description thereof is omitted.

  The above is the operation of the adding step ST34, and the operation is equivalent to the operation of the adding means 34 of FIG.

The adding step ST4 generates an image Dout obtained by adding the enlarged image D2A generated in the image enlarging step ST2A and the high frequency component image D3 generated in the high frequency component image processing step ST3. Then, the generated image Dout is output as a final enlarged image through a step (not shown).
This operation is the same as that of the adding means 4 in FIG.

  The above is the operation of the image processing method according to the second embodiment. As is clear from the above description, the image processing method according to the second embodiment can also enlarge an image by the same processing as that of the image processing device according to the first embodiment, so that the same effect as the image processing device according to the first embodiment can be obtained. It is done. Also, the image processing method according to the second embodiment can be modified in the same manner as the image processing apparatus according to the first embodiment, and the effect obtained in that case is the same as that of the image processing apparatus according to the first embodiment. . For example, when the absolute value of each pixel of the horizontal intermediate image D32Ah is expressed as dH and the absolute value of each pixel of the vertical intermediate image D32Ah is expressed as dV in the edge direction estimation step ST33C1, the difference dH−dV between these two values is expressed as dH−dV. You may output as edge direction estimation amount D33C1.

  In addition, the high frequency component image processing step ST3 may be configured by excluding the high frequency component image generation step ST32A, the horizontal / vertical addition step ST33A, and the addition step ST33D. In this case, from the addition step ST34, the intermediate image D33B generated in the horizontal / vertical addition step ST33B is output as the high frequency component image D3.

  In addition to the above example, modifications added to the first embodiment can be added to the image processing method according to the second embodiment. In that case, how to modify each component of the image processing method according to the second embodiment is clear from comparison with the first embodiment. For example, the operation of the nonlinear processed image generation step ST30 can be determined according to the enlargement ratio.

  In addition, since the image processing apparatus according to the second embodiment can be used as a part of the image display apparatus similar to the image processing apparatus described in the first embodiment, the image Dout generated by the image processing apparatus according to the second embodiment. The image display apparatus that displays the same effect as the image processing apparatus described in the first embodiment can also be obtained. Further, the image processing method implemented using the image processing apparatuses of the first and second embodiments and the image display method using the same can obtain the same effects.

Embodiment 3 FIG.
FIG. 33 is a diagram illustrating a configuration of an image processing apparatus according to the third embodiment of the present invention. Similar to the image processing apparatus according to the first embodiment, the image processing apparatus according to the third embodiment can be used as a part of the image display apparatus shown in FIG. Further, when compared with the image processing apparatus according to the third embodiment and the image processing apparatus according to the first embodiment, the configuration and operation of the adding means 34 are different. Therefore, in the following description, the configuration and operation of the adding means 34 will be mainly described.

  33 includes addition ratio calculation means 33CA and 33CB, horizontal / vertical addition means 33A, horizontal / vertical addition means 33B, and addition means 33C.

  34 and 36 show the configuration of the addition ratio calculation means 33CA and 33CB, and FIGS. 35 and 37 show the operation of the addition ratio calculation means 33CA and 33CB.

First, the configuration and operation of the addition ratio determination unit 33CA will be described with reference to FIGS.
FIG. 34 is a diagram illustrating a configuration example of the addition ratio determination unit 33CA. The addition ratio determination unit includes an edge direction estimation unit 33CA1 and a weight coefficient determination unit 33CA2.

  The edge direction estimation means 33CA1 calculates an edge direction estimation amount D33CA1 as an amount corresponding to the edge direction (angle) from two signals of the horizontal direction intermediate image D32Ah and the vertical direction intermediate image D32Av. For example, when the absolute value of each pixel value in the horizontal intermediate image D32Ah is dH and the absolute value of each pixel value in the vertical intermediate image D32Av is dV, the difference dH−dV between these two values is calculated as an edge direction estimation amount. Output as D33CA1.

により算出している。ただし、Ｋβは図３５における直線の傾き（従って（ｄＨ−ｄＶ）の増加に対するＤ３３ＣＡの変化の割合）を表す正の定数である。
以上が、加重比率決定手段３３ＣＡの動作である。 The weighting factor determination unit 33CA2 determines the weighting factor D33CA based on the edge direction estimation amount D33CA1. FIG. 35 is a diagram illustrating an example of how to determine the weighting coefficient D33CA with respect to the edge direction estimation amount D33CA1. In the example shown in FIG. 35, the weighting coefficient D33CA is

It is calculated by. However, Kβ is a positive constant representing the slope of the straight line in FIG. 35 (thus, the ratio of the change in D33CA with respect to the increase in (dH−dV)).
The above is the operation of the weight ratio determining means 33CA.

Next, the configuration and operation of the addition ratio determination unit 33CB will be described with reference to FIGS.
FIG. 36 is a diagram showing the configuration of the addition ratio determination means 33CB, and the addition ratio determination means includes an edge direction estimation means 33CB1 and a weight coefficient determination means 33CB2.

  The edge direction estimation means 33CB1 calculates an edge direction estimation amount D33CB1 as an amount corresponding to the edge direction (angle) from two signals of the horizontal direction intermediate image D32Bh and the vertical direction intermediate image D32Bv. For example, when the absolute value of each pixel value of the horizontal intermediate image D32Bh is dH and the absolute value of each pixel value of the vertical intermediate image D32Bv is dV, the difference dH−dV between these two values is calculated as an edge direction estimation amount. Output as D33CB1.

により算出している。ただし、Ｋγは図３５における直線の傾き（従って（ｄＨ−ｄＶ）の増加に対するＤ３３ＣＢの変化の割合）を表す正の定数である。
以上が、加重比率決定手段３３ＣＢの動作である。 The weighting factor determination unit 33CB2 determines the weighting factor D33CB based on the edge direction estimation amount D33CB1. FIG. 37 is a diagram illustrating an example of how to determine the weighting coefficient D33CB with respect to the edge direction estimation amount D33CB1. In the example shown in FIG. 37, the weighting coefficient D33CB is

It is calculated by. However, Kγ is a positive constant representing the slope of the straight line in FIG. 35 (thus, the ratio of the change in D33CB with respect to the increase in (dH−dV)).
The above is the operation of the weight ratio determining means 33CB.

と表される。 Next, the operation of the horizontal / vertical addition means 33A will be described. The horizontal / vertical adder 33A performs weighted addition of the horizontal intermediate image D32Ah and the vertical intermediate image D32Av, and outputs the result as an intermediate image D33A. The weighting factor D33CA is used as the weighted addition ratio. That is, the weighted addition performed by the horizontal / vertical addition means 33A is

It is expressed.

と表される。 Next, the operation of the horizontal / vertical addition means 33B will be described. The horizontal / vertical adder 33B performs weighted addition of the horizontal intermediate image D32Bh and the vertical intermediate image D32Bv, and outputs the result as an intermediate image D33B. The weighting factor D33CB is used as the weighted addition ratio. That is, the weighted addition performed by the horizontal / vertical addition means 33B is

It is expressed.

  Since the operation of the adding means 33D is the same as that of the first embodiment, the description thereof is omitted.

  The above is the operation of the image processing apparatus according to the third embodiment.

  Since the operation of the addition ratio determination means 33CA and 33CB is the same as the operation of the addition ratio determination means 33C in the modification described in the first embodiment, both the addition means 34 in the third embodiment and the addition means 34 in the first embodiment. It is clear that it has the same effect as. Therefore, the image processing apparatus according to the third embodiment can obtain the same effects as the image processing apparatus according to the first embodiment.

  Furthermore, the description of the modified example described for the first embodiment also applies to the third embodiment.

Embodiment 4 FIG.
38, 39 and 40 are diagrams showing a part of the image processing method according to the fourth embodiment of the present invention. The image processing method according to the fourth embodiment operates in accordance with substantially the same flow as the image processing method according to the second embodiment. That is, the image processing method according to the fourth embodiment operates according to FIGS. 26, 27, 28, 38, 39, and 40. Here, the operations shown in FIG. 26, FIG. 27, and FIG.

  The operation of the high frequency component image processing step ST3 of the image processing method according to the fourth embodiment will be described with reference to FIG. As shown in FIG. 38, the high frequency component image processing step ST3 includes a high frequency component image generation step ST32A, a nonlinear processed image generation step ST30, and an addition step ST34. Here, the operations of the high-frequency component passing step ST32A and the non-linear processing image generation step ST30 are the same as those in the second embodiment, so the description thereof will be omitted and the operation of the adding step ST34 will be described.

  The addition step ST34 includes addition ratio determination steps ST33CA and ST33CB, horizontal and vertical addition steps ST33A and ST33B, and an addition step ST33D.

First, in the addition ratio determination step ST33CA of the addition step ST34, the weight coefficient D33CA is obtained. Here, the addition ratio determining step ST33CA includes an edge direction estimating step ST33CA1 and a weight determining step ST33CA2, as shown in FIG.
In the edge direction estimation step ST33CA1, when the absolute value of each pixel of the horizontal intermediate image D32Ah is represented by dH and the absolute value of each pixel of the vertical intermediate image D32Av is represented by dV, the difference dH−dV between these two values is represented by the edge direction. Output as estimated amount D33CA1. This operation is the same as the edge direction estimating means 33CA1 in FIG.
The weight determination step DT33CA2 calculates a weight coefficient D33CA according to the equation (5). This operation is the same as the weight determination means 33CA2 in FIG.

Next, in addition ratio determination step ST33CB, a weight coefficient D33CB is obtained. Here, the addition ratio determining step ST33CB includes an edge direction estimating step ST33CB1 and a weight determining step ST33CB2, as shown in FIG.
In the edge direction estimation step ST33CB1, when the absolute value of each pixel of the horizontal intermediate image D32Bh is expressed by dH and the absolute value of each pixel of the vertical intermediate image D32Bv is expressed by dV, the difference dH−dV between these two values is expressed in the edge direction. Output as the estimated amount D33CB1. This operation is the same as the edge direction estimating means 33CB1 in FIG.
The weight determination step DT33CB2 calculates a weight coefficient D33CB according to the equation (6). This operation is the same as the weight determining means 33CB2 in FIG.

  Next, in the horizontal / vertical addition step ST33A, an intermediate image D33A obtained by weighted addition of the horizontal direction intermediate image D32Ah and the vertical direction intermediate image D32Av is obtained. The details of this operation are the same as those of the horizontal / vertical adding means 33A shown in FIG.

  Next, in the horizontal / vertical addition step ST33B, an intermediate image D33B obtained by weighted addition of the horizontal direction intermediate image D32Bh and the vertical direction intermediate image D32Bv is obtained. Since the details of this operation are the same as those of the horizontal / vertical adding means 33B of FIG. 33, the description thereof is omitted.

  Finally, the addition step ST33D generates a high frequency component image D3 obtained by adding the intermediate image D33A and the intermediate image D33B. Since this operation can be the same as that of the adding means 33D of FIG. 33, detailed description thereof is omitted.

  The above is the operation of the adding step ST34, and the operation is equivalent to the operation of the adding means 34 of FIG.

Since the operation of the image processing method according to the fourth embodiment is the same as that of the image processing apparatus according to the third embodiment, the same effect as that of the image processing apparatus according to the third embodiment can be obtained.
Furthermore, the description of the modified example described for the second embodiment also applies to the fourth embodiment.

  DESCRIPTION OF SYMBOLS 1 High frequency component image generation means, 2A Image expansion means, 2B Image expansion means, 3 High frequency component image processing means, 4 Addition means, 32A High frequency component image generation means, 30 Non-linear image generation means, 33A High frequency component image correction Means, 33B high frequency component image correcting means, 34 adding means, Din input image, D1 high frequency component image, D2A enlarged image, D2B enlarged image, D33C weighting factor, D32A intermediate image, D32B intermediate image, D33A intermediate image, D33B intermediate Image, D3 high frequency component image, Dout output image.

Claims (23)

In an image processing apparatus for enlarging an input image,
First image enlarging means for enlarging the input image and outputting a first enlarged image;
First high frequency component image generation means for generating a first high frequency component image obtained by extracting the high frequency component of the input image;
Second image enlarging means for enlarging the first high frequency component image and outputting a second enlarged image;
High-frequency component image processing means for generating a second high-frequency component image from the second enlarged image;
In the image processing apparatus having first addition means for adding the first enlarged image and the second high-frequency component image,
The first high frequency component image generation means includes:
First horizontal high frequency component image generation means for generating a first horizontal high frequency component image obtained by extracting a horizontal high frequency component of the input image;
First vertical high frequency component image generation means for generating a first vertical high frequency component image obtained by extracting a vertical high frequency component of the input image;
The second enlarged image is
A third enlarged image obtained by enlarging the first horizontal high-frequency component image, and a fourth enlarged image obtained by enlarging the first vertical high-frequency component image,
The high frequency component image processing means includes
A second horizontal high-frequency component image generating means for generating a first horizontal intermediate image obtained by extracting a high-frequency component of the third enlarged image; and a first of extracting a high-frequency component of the fourth enlarged image. Second vertical high frequency component image generating means for generating a vertical intermediate image of
A horizontal non-linearly processed image generating means for generating a second horizontal intermediate image obtained by performing non-linear processing on the third enlarged image;
Vertical non-linear processing image generation means for generating a second vertical intermediate image obtained by performing non-linear processing on the fourth enlarged image;
First weighted addition means for generating a third intermediate image obtained by weighted addition of the first horizontal intermediate image and the first vertical intermediate image;
Second weighted addition means for generating a fourth intermediate image obtained by weighted addition of the second horizontal intermediate image and the second vertical intermediate image;
An image processing apparatus comprising: a second addition unit that adds the third intermediate image and the fourth intermediate image. In an image processing apparatus for enlarging an input image,
First image enlarging means for enlarging the input image and outputting a first enlarged image;
First high frequency component image generation means for generating a first high frequency component image obtained by extracting the high frequency component of the input image;
Second image enlarging means for enlarging the first high frequency component image and outputting a second enlarged image;
High-frequency component image processing means for generating a second high-frequency component image from the second enlarged image;
In the image processing apparatus having first addition means for adding the first enlarged image and the second high-frequency component image,
The first high frequency component image generation means includes:
First horizontal high frequency component image generation means for generating a first horizontal high frequency component image obtained by extracting a horizontal high frequency component of the input image;
First vertical high frequency component image generation means for generating a first vertical high frequency component image obtained by extracting a vertical high frequency component of the input image;
The second enlarged image is
A third enlarged image obtained by enlarging the first horizontal high-frequency component image, and a fourth enlarged image obtained by enlarging the first vertical high-frequency component image,
The high frequency component image processing means includes
A horizontal non-linearly processed image generating means for generating a second horizontal intermediate image obtained by performing non-linear processing on the third enlarged image;
Vertical non-linear processing image generation means for generating a second vertical intermediate image obtained by performing non-linear processing on the fourth enlarged image;
An image processing apparatus comprising: a second weighted addition unit that generates a fourth intermediate image obtained by weighted addition of the second horizontal intermediate image and the second vertical intermediate image. The high frequency component image processing means further includes an addition ratio determining means for obtaining a weighting factor according to an angle of an edge included in the first enlarged image,
The image processing apparatus according to claim 1, wherein each of the first weighted addition unit and the second weighted addition unit performs the weighted addition based on the weighting coefficient. The high frequency component image processing means further includes an addition ratio determining means for obtaining a weighting factor according to an angle of an edge included in the first enlarged image,
The image processing apparatus according to claim 2, wherein the second weighted addition unit performs the weighted addition based on the weighting coefficient. The addition ratio determining means includes
The image processing apparatus according to claim 3, wherein an angle of an edge included in the first enlarged image is obtained from the third enlarged image and the fourth enlarged image. The addition ratio determining means includes
The image processing apparatus according to claim 3, wherein an angle of an edge included in the first enlarged image is obtained from the first horizontal intermediate image and the first vertical intermediate image. The addition ratio determining means includes
The image processing apparatus according to claim 4, wherein an angle of an edge included in the first enlarged image is obtained from the second horizontal intermediate image and the second vertical intermediate image. The horizontal direction non-linear processing image generating means is
A horizontal non-linear processing means for generating a first non-linear processed image obtained by performing non-linear processing on the third enlarged image;
The image processing apparatus according to claim 1, wherein a high-frequency component is extracted from the first non-linearly processed image and used as the second horizontal intermediate image.   The image processing apparatus according to claim 8, wherein the horizontal nonlinear processing unit amplifies the pixel value of the third enlarged image with an amplification factor that changes in accordance with the pixel. The horizontal non-linear processing means includes:
A horizontal direction zero cross determining means for determining a point where the pixel value of the third enlarged image changes from positive to negative or from negative to positive as a zero cross point;
10. The image according to claim 8, further comprising a horizontal signal amplifying unit that amplifies a pixel value of the third enlarged image with an amplification factor determined according to a determination result of the horizontal direction zero-cross determination unit. Processing equipment. The horizontal direction signal amplification means includes
The amplification factor for the pixel value of the pixel existing in the first region including the zero-cross point determined by the horizontal zero-cross determination unit is set to a value larger than 1, and the amplification factor for the pixel values of the other pixels is set to 1. The image processing apparatus according to claim 10.   The image processing apparatus according to claim 11, wherein the first area is determined according to an enlargement ratio in the second image enlargement unit.   The image processing apparatus according to claim 11, wherein an amplification factor for a pixel existing in the first region is determined according to the pixel. The vertical non-linear processing image generating means is
Vertical direction nonlinear processing means for generating a second nonlinear processed image obtained by performing nonlinear processing on the fourth enlarged image;
The image processing apparatus according to claim 1, wherein a high-frequency component is extracted from the second non-linearly processed image and used as the second vertical intermediate image.   The image processing apparatus according to claim 14, wherein the vertical nonlinear processing means amplifies the pixel value of the fourth enlarged image with an amplification factor that changes in accordance with the pixel. The vertical nonlinear processing means includes:
Vertical zero-cross determination means for determining a point where the pixel value of the fourth enlarged image changes from positive to negative or from negative to positive as a zero-cross point;
16. The image according to claim 14, further comprising vertical direction signal amplification means for amplifying a pixel value of the fourth enlarged image with an amplification factor determined according to a determination result of the vertical direction zero cross determination means. Processing equipment. The vertical signal amplification means includes
The amplification factor for the pixel value of the pixel existing in the second region including the zero-cross point determined by the vertical zero-cross determination unit is set to a value larger than 1, and the amplification factor for the pixel values of other pixels is set to 1. The image processing apparatus according to claim 16.   The image processing apparatus according to claim 17, wherein the second area is determined according to an enlargement ratio in the second image enlargement unit.   The image processing apparatus according to claim 17 or 18, wherein an amplification factor for a pixel existing in the second region is determined according to the pixel.   An image display device comprising the image processing device according to claim 1. In an image processing method for enlarging an input image,
A first image enlargement step of enlarging the input image and outputting a first enlarged image;
A first high frequency component image generating step for generating a first high frequency component image obtained by extracting the high frequency component of the input image;
A second image enlargement step of enlarging the first high frequency component image and outputting a second enlarged image;
A high frequency component image processing step of generating a second high frequency component image from the second enlarged image;
In the image processing method including a first addition step of adding the first enlarged image and the second high-frequency component image,
The first high frequency component image generation step includes:
A first horizontal high-frequency component image generation step of generating a first horizontal high-frequency component image obtained by extracting a horizontal high-frequency component of the input image;
A first vertical high-frequency component image generation step for generating a first vertical high-frequency component image obtained by extracting a vertical high-frequency component of the input image,
The second enlarged image is
A third enlarged image obtained by enlarging the first horizontal high-frequency component image, and a fourth enlarged image obtained by enlarging the first vertical high-frequency component image,
The high frequency component image processing step includes:
A second horizontal high-frequency component image generation step of generating a first horizontal intermediate image obtained by extracting a high-frequency component of the third enlarged image;
A second vertical high-frequency component image generation step for generating a first vertical intermediate image obtained by extracting a high-frequency component of the fourth enlarged image;
A horizontal non-linearly processed image generating step for generating a second horizontal intermediate image obtained by performing non-linear processing on the third enlarged image;
A vertical nonlinear processed image generating step for generating a second vertical intermediate image obtained by performing nonlinear processing on the fourth enlarged image;
A first weighted addition step of generating a third intermediate image obtained by weighted addition of the first horizontal intermediate image and the first vertical intermediate image;
A second weighted addition step of generating a fourth intermediate image obtained by weighted addition of the second horizontal intermediate image and the second vertical intermediate image;
An image processing method comprising: a second addition step of adding the third intermediate image and the fourth intermediate image. In an image processing method for enlarging an input image,
A first image enlargement step of enlarging the input image and outputting a first enlarged image;
A first high frequency component image generating step for generating a first high frequency component image obtained by extracting the high frequency component of the input image;
A second image enlargement step of enlarging the first high frequency component image and outputting a second enlarged image;
A high frequency component image processing step of generating a second high frequency component image from the second enlarged image;
In the image processing method including a first addition step of adding the first enlarged image and the second high-frequency component image,
The first high frequency component image generation step includes:
A first horizontal high-frequency component image generation step of generating a first horizontal high-frequency component image obtained by extracting a horizontal high-frequency component of the input image;
First vertical high frequency component image generation step means for generating a first vertical high frequency component image obtained by extracting a vertical high frequency component of the input image,
The second enlarged image is
A third enlarged image obtained by enlarging the first horizontal high-frequency component image;
Consisting of a fourth magnified image that is an enlargement of the first vertical high frequency component image;
The high frequency component image processing step includes:
A horizontal non-linearly processed image generating means for generating a second horizontal intermediate image obtained by performing non-linear processing on the third enlarged image;
A vertical nonlinear processed image generating step for generating a second vertical intermediate image obtained by performing nonlinear processing on the fourth enlarged image;
An image processing method comprising: a second weighted addition step of generating a fourth intermediate image obtained by weighted addition of the second horizontal intermediate image and the second vertical intermediate image.   An image display apparatus that displays an image generated by the image processing method according to claim 21.

Images