JP2010034959A - Image processing apparatus and method, and image display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that, in an image processing apparatus that properly sets an enhancement coefficient to a detailed portion image of a desired frequency band, enhancement processing may be made inappropriate or insufficient in accordance with an input image and an output image of proper image quality can not be obtained. <P>SOLUTION: An image processing apparatus includes: an intermediate image generating means (1) for generating a first intermediate image (D1) obtained by extracting a component in the vicinity of a specific frequency band from an input image (DIN); an intermediate image processing means (2) for generating a second intermediate image (D2) obtained by processing the first intermediate image (D1); and an adding means (3) for adding the input image (DIN) and the second intermediate image (D2). The intermediate image processing means (2) includes a nonlinear processing means (2A) for changing contents of processing in accordance with pixels of the intermediate image (D1). <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and method for emphasizing an input image, and an image display apparatus using the same. For example, when an enlarged image obtained by enlarging an original image is input as an input image, a high frequency Image enhancement processing is performed so as to obtain an output image with high resolution by generating and adding components.

  In general, an image signal representing an image is appropriately subjected to image processing and then reproduced and displayed.

  For example, in the image processing apparatus described in Patent Document 1, with respect to a detail image converted to multi-resolution, an enhancement coefficient for a detail image in a desired frequency band is set to a detail image in a lower frequency band than the desired frequency band. The desired frequency band is emphasized by setting based on the above signal.

JP-A-9-44651

  However, in an image processing apparatus that appropriately sets an enhancement coefficient for a detail image in a desired frequency band with respect to a detail image converted to multi-resolution, the enhancement processing is inappropriate or insufficient depending on the input image, and the appropriate image quality It was impossible to obtain the output image.

  For example, when an image that has undergone enlargement processing is input as an input image, a component (folding component) in which a part of the frequency spectrum of the image before enlargement processing is folded is present on the high frequency component side of the frequency spectrum of the input image. appear. Therefore, if the high frequency component is simply emphasized, this aliasing component is emphasized, which is inappropriate processing. In addition, if the frequency band is limited and only the frequency band that does not include the aliasing component is emphasized, when considering the frequency spectrum, emphasis on the high frequency component side is avoided, resulting in insufficient enhancement processing. End up.

  When an image subjected to noise processing is input as an input image, the frequency spectrum on the high frequency component side is lost due to noise processing. Therefore, even if an attempt is made to extract a high frequency component, it cannot be extracted, and image enhancement processing may not be performed sufficiently.

The image processing apparatus of the present invention
Intermediate image generating means for generating a first intermediate image obtained by extracting a component in the vicinity of a specific frequency band of the input image;
Intermediate image processing means for generating a second intermediate image based on the first intermediate image;
In an image processing apparatus having a first addition means for adding the input image and the second intermediate image,
The intermediate image processing means includes non-linear processing means for changing processing contents in accordance with pixels of the first intermediate image.

  According to the present invention, even when the input image includes a aliasing component on the high frequency component side in the frequency spectrum or when the input image does not sufficiently include the high frequency component, the image can be sufficiently enhanced. .

Embodiment 1 FIG.
FIG. 1 is a diagram showing a configuration of an image processing apparatus according to Embodiment 1 of the present invention, and can be used as a part of an image display apparatus, for example.

The illustrated image processing apparatus includes an intermediate image generation unit 1, an intermediate image processing unit 2, and an addition unit 3.
The intermediate image generation means 1 generates an intermediate image (first intermediate image) D2 obtained by extracting components near a specific frequency band of the input image DIN.
The intermediate image processing means 2 generates an intermediate image (second intermediate image) D2 obtained by performing processing described later on the first intermediate image D2.
The adding means 3 adds the input image DIN and the second intermediate image D2. The addition result by the adding means 3 is output as an output image DOUT.

  The intermediate image generation unit 1 extracts a high frequency component image generation unit 1A that generates an image (first high frequency component image) D1A obtained by extracting only a high frequency component from the input image DIN, and extracts only a low frequency component of the image D1A. And a low frequency component image generating means 1B for generating the image D1B. The high-frequency component image generating means 1A and the low-frequency component image generating means 1B constitute a band pass filter means. From the intermediate image generating means 1, the image D1B is output as the intermediate image D1.

  The intermediate image processing means 2 outputs a non-linear processing means 2A that outputs an image D2A obtained by performing non-linear processing described later on the intermediate image D1, and a high-frequency component image that outputs an image D2B obtained by extracting only the high-frequency component of the image D2A. A generating unit 2B and an adding unit 2C that outputs an image D2C obtained by adding the intermediate image D1 and the image D2B are provided. From the intermediate image processing means 2, the image D2C is output as the intermediate image D2.

The detailed operation of the image processing apparatus according to the first embodiment will be described below.
First, the detailed operation of the intermediate image generating unit 1 will be described.

  First, the intermediate image generating unit 1 generates an image D1A obtained by extracting only the high frequency component of the input image DIN in the high frequency component image generating unit 1A. High frequency components can be extracted by performing high-pass filter processing. Note that high-frequency components are extracted in each of the horizontal and vertical directions of the image. That is, the high-frequency component image generating unit 1A performs horizontal high-pass filter processing on the input image DIN to generate an image D1Ah in which the high-frequency component is extracted only in the horizontal direction. And vertical direction high frequency component image generating means 1Av that performs high-pass filter processing in the vertical direction to generate an image D1Av obtained from high frequency components only in the vertical direction, and the image D1A is composed of an image D1Ah and an image D1Av.

  Next, the intermediate image generating unit 1 generates an image D1B obtained by extracting only the low frequency component of the image D1A in the low frequency component image generating unit 1B. The low frequency component can be extracted by performing a low pass filter process. The low frequency component is extracted in each of the horizontal direction and the vertical direction. That is, the low-frequency component image generation unit 1B performs horizontal low-frequency component image generation unit 1Bh that generates an image D1Bh obtained by performing horizontal low-pass filter processing on the image D1Ah, and performs low-pass filter processing in the vertical direction on the image D1Av. Vertical direction low frequency component image generation means 1Bv for generating the performed image D1Bv, and the image D1B is composed of an image D1Bh and an image D1Bv. Then, the intermediate image generating unit 1 outputs the image D1B as the intermediate image D1. The intermediate image D1 includes an image D1h corresponding to the image D1Bh and an image D1v corresponding to the image D1Bv.

Next, the detailed operation of the intermediate image processing means 2 will be described.
First, the intermediate image processing unit 2 generates an image D2A obtained by performing a later-described nonlinear process on the intermediate image D1 in the nonlinear processing unit 2A. Nonlinear processing is performed for each of the horizontal direction and the vertical direction. That is, the nonlinear processing means 2A performs horizontal nonlinear processing means 2Ah that performs nonlinear processing described later on the image D1Bh to generate an image D2Ah, and vertical that performs nonlinear processing described later on image D1Bv to generate the image D2Av. The image D2A includes an image D2Ah and an image D2Av.

  The operation of the nonlinear processing means 2A will be described in more detail. The nonlinear processing means 2A includes a horizontal nonlinear processing means 2Ah and a vertical nonlinear processing means 2Av having the same configuration. Here, the horizontal nonlinear processing means 2Ah performs horizontal processing, and the vertical nonlinear processing means 2Av performs vertical processing.

  FIG. 2 is a diagram showing the configuration of the horizontal nonlinear processing means 2Ah. The horizontal non-linear processing unit 2Ah includes a zero-cross determining unit 311h and a signal amplifying unit 312h. Note that the image D1h is input to the nonlinear processing means 2Ah as the input image DIN311h.

  The zero-cross determining unit 311h checks the change in the pixel value in the input image DIN 311h along the horizontal direction. Then, a point where the pixel value changes from a positive value to a negative value or from a negative value to a positive value is regarded as a zero cross point, and the positions of pixels (adjacent pixels in the front and rear) before and after the zero cross point by the signal D311h. The signal is transmitted to the signal amplifying means 312h. For example, pixels located on the left and right of the zero cross point are recognized as pixels located before and after the zero cross point.

  Based on the signal D311h, the signal amplifying unit 312h identifies pixels before and after the zero cross point (adjacent pixels before and after the zero cross point), and amplifies the pixel value only for the pixels before and after the zero cross point (the absolute value is increased). A nonlinear processed image D312h is generated. That is, the amplification factor is set to a value larger than 1 for the pixel values of the pixels before and after the zero cross point, and the amplification factor is set to 1 for the pixel values of the other pixels.

  A non-linearly processed image D312h is output from the horizontal non-linear processing means 24h as an image 24h.

  FIG. 3 is a diagram showing the configuration of the vertical nonlinear processing means 2Av. The vertical nonlinear processing means 2Av includes a zero cross determination means 311v and a signal amplification means 312v. Note that the image D1v is input to the nonlinear processing means 2Av as the input image DIN311v.

  The zero-cross determination unit 311v confirms the change of the pixel value in the input image DIN 311v along the vertical direction. Then, a position where the pixel value changes from a positive value to a negative value or from a negative value to a positive value is regarded as a zero cross point, and the position of pixels (adjacent pixels in the front and rear) before and after the zero cross point is determined by the signal D311v. This is transmitted to the signal amplifying means 312v. For example, pixels positioned above and below the zero cross point are recognized as pixels positioned before and after the zero cross point.

Based on the signal D311v, the signal amplifying unit 312v identifies pixels before and after the zero cross point (adjacent pixels before and after the zero cross point), and amplifies the pixel value only for the pixels before and after the zero cross point (the absolute value is increased). A nonlinear processed image D312v is generated. That is, the amplification factor is set to a value larger than 1 for the pixel values of the pixels before and after the zero cross point, and the amplification factor is set to 1 for the pixel values of the other pixels.
The above is the operation of the nonlinear processing means 2A.

  Next, the intermediate image processing unit 2 generates an image D2B obtained by extracting only the high frequency component of the image D2A in the high frequency component image generation unit 2B. High frequency components can be extracted by performing high-pass filter processing. Note that high-frequency components are extracted in each of the horizontal and vertical directions of the image. That is, the high-frequency component image generation unit 2B performs horizontal high-frequency component image generation unit 2Bh that generates an image D2Bh obtained by performing horizontal high-pass filter processing on the image D2Ah, and performs vertical high-pass filter processing on the image D2Av. Vertical direction high frequency component image generation means 2Bv for generating the performed image D2Bv, and the image D2B includes the image D2Bh and the image D2Bv.

  Next, the adding means 2C adds the intermediate image D1 and the image D2B to generate an image D2C. The intermediate image D1 is composed of the image D1h and the image D1v, and the image D2B is composed of the image D2Bh and the image D2Bv. Therefore, when the intermediate image D1 and the image D2B are added, It means adding all. From the intermediate image processing means 2, the image D2C is output as the intermediate image D2.

  Here, the addition processing in the adding means 2C is not limited to simple addition, and weighted addition may be used. That is, the images D1h, D1v, D2Bh, and D2Bv may be added after being amplified at different amplification factors.

  Finally, the operation of the adding means 3 will be described. The adding means 3 generates an output image DOUT obtained by adding the input image DIN and the intermediate image D2. The output image DOUT is output from the image processing apparatus as a final output image.

  Hereinafter, an example in which the image processing apparatus according to the present invention is used as a part of an image display apparatus will be described. Through this description, the operation and effect of the image processing apparatus according to the present invention will become clear. In the following description, unless otherwise specified, the symbol Fn represents the Nyquist frequency of the input image DIN.

  FIG. 4 shows an image display apparatus using the image processing apparatus according to the present invention, and an image corresponding to the original image DORG is displayed on the monitor U3.

  When the image size of the original image DORG is smaller than the image size of the monitor U3, the image enlargement unit U1 outputs an image DU1 obtained by enlarging the original image DORG. Here, a bicubic method or the like can be used as means for enlarging the image.

  The image processing apparatus U2 in the present invention outputs an image DU2 obtained by performing the above-described processing on the image DU1. The image DU2 is displayed on the monitor U8.

  Hereinafter, assuming that the number of pixels of the original image DORG is half that of the monitor U3 in both the horizontal direction and the vertical direction, the operation and action of the image enlarging means U1 will be described first.

  FIG. 5 is a diagram showing the configuration and operation of the image enlarging means U1. The image enlarging means U1 includes a horizontal direction zero insertion means U1A, a horizontal direction low frequency component passing means U1B, a vertical direction zero insertion means U1C, and a vertical direction low frequency. Component passing means U1D is provided. The horizontal zero insertion means U1A generates an image DU1A in which pixels having a pixel value of 0 in the horizontal direction of the original image DORG are appropriately inserted. The horizontal direction low frequency component passing means U1B generates an image DU1B in which only the low frequency component of the image DU1A is extracted by low pass filter processing. The vertical zero insertion unit U1C generates an image DU1C in which pixels having a pixel value of 0 in the vertical direction of the image DU1B are appropriately inserted. The vertical direction low frequency component passing means DU1D generates an image DU1D obtained by extracting only the low frequency component of the image DU1C. The image DU1D is output from the image enlarging means U1 as an image DU1 obtained by doubling the original image DORG in both the horizontal and vertical directions.

  6A to 6E are diagrams for explaining the operation of the image enlarging means U1 in detail. FIG. 6A shows the original image DORG, FIG. 6B shows the image DU1A, and FIG. ) Represents the image DU1B, FIG. 6D represents the image DU1C, and FIG. 6E represents the image DU1D. 6A to 6E, a square (each square) represents a pixel, and a symbol or a numerical value written therein represents a pixel value of each pixel.

  The horizontal zero insertion means U1A inserts one pixel having a pixel value of 0 for each pixel in the horizontal direction into the original image DORG shown in FIG. 6A (that is, adjacent to the original image DORG in the horizontal direction). An image DU1A shown in FIG. 6B is generated by inserting a single pixel column composed of pixels having a pixel value of 0 between the pixel columns. The horizontal direction low frequency component passing means U1B performs a low-pass filter process on the image DU1A shown in FIG. 6B to generate an image DU1B shown in FIG. The vertical zero insertion means U1C inserts one pixel with a pixel value of 0 for each pixel in the vertical direction into the image DU1B shown in FIG. 6C (that is, adjacent to the image DU1B in the vertical direction). An image DU1C shown in FIG. 6D is generated by inserting one pixel row composed of pixels having a pixel value of 0 between the pixel rows. The vertical low frequency component passing means U1D performs low-pass filter processing on the image DU1C shown in FIG. 6D to generate an image DU1D shown in FIG. Through the above processing, an image DU1D in which the original image DORG is doubled in both the horizontal direction and the vertical direction is generated.

  FIGS. 7A to 7D show the effect of processing by the image enlarging means U1 on the frequency space, FIG. 7A shows the frequency spectrum of the original picture DORG, and FIG. 7B shows the image DU1A. 7C shows the frequency response of the horizontal frequency component passing means U1B, and FIG. 7D shows the frequency spectrum of the image DU1B. 7A to 7D, the horizontal axis is the frequency axis representing the spatial frequency in the horizontal direction, and the vertical axis represents the frequency spectrum or the intensity of the frequency response. Note that the number of pixels of the original image DORG is half that of the input image DIN. In other words, the sampling interval of the original image DORG is twice the sampling interval of the input image DIN. Therefore, the Nyquist frequency of the original image DORG is half of the Nyquist frequency of the input image DIN, that is, Fn / 2.

  In FIGS. 7A to 7D, only one frequency axis is used to simplify the notation. However, usually, image data consists of pixel values given on a pixel array arranged in a two-dimensional plane, and its frequency spectrum is also given on a plane stretched by a horizontal frequency axis and a vertical frequency axis. It is. Therefore, in order to accurately represent the frequency spectrum of the original picture DORG or the like, it is necessary to describe both the horizontal frequency axis and the vertical frequency axis. However, the shape of the frequency spectrum usually spreads isotropically around the origin on the frequency axis, and as long as the frequency spectrum in the space spanned by one frequency axis is shown, the frequency axis It is easy for those skilled in the art to expand and consider the space spanned by two. Therefore, unless otherwise specified in the following description, the description on the frequency space is performed using a space stretched by one frequency axis.

  First, the frequency spectrum of the original picture DORG will be described. Normally, a natural image is input as the original image DORG, but its spectral intensity is concentrated around the origin of the frequency space. Therefore, the frequency spectrum of the original picture DORG is a spectrum SPO represented as shown in FIG.

  Next, the spectral intensity of the image DU1A will be described. The image DU1A is generated by inserting one pixel per pixel and a pixel value of 0 in the horizontal direction with respect to the original image DORG. When such processing is performed, aliasing around the Nyquist frequency of the original picture DORG occurs in the frequency spectrum. That is, since a spectrum SPM in which the spectrum SPO is folded around the frequency ± Fn / 2 is generated, the frequency spectrum of the image DU1A is expressed as shown in FIG.

  Next, the frequency response of the horizontal low frequency component passing means U1B will be described. Since the horizontal low frequency component passing means U1B is realized by a low-pass filter, its frequency response becomes lower as the frequency becomes higher as shown in FIG.

  Finally, the frequency spectrum of the image DU1B will be described. The image DU1B is obtained by performing low-pass filter processing having the frequency response shown in FIG. 7C on the image DU1A having the frequency spectrum shown in FIG. 7B. Therefore, as shown in the image DU1B, the frequency spectrum of the image DU1B includes a spectrum SP2 in which the intensity of the spectrum SPM has dropped to some extent and a spectrum SP1 in which the intensity of the spectrum SPO has dropped to some extent. In general, the frequency response of the low-pass filter decreases as the frequency increases. Therefore, when the intensity of the spectrum SP1 is compared with the spectrum SPO, the spectrum intensity on the high frequency component side, that is, the frequency near ± Fn / 2 is reduced by the horizontal low frequency component passing means U1B.

  Of the processing by the image enlarging means U1, the description of the operation on the frequency space of the processing by the vertical zero insertion means U1C and the vertical low frequency component passing means U1D is omitted, but from the contents of the processing. It can be easily understood that there is an action similar to that described with reference to FIGS. 7A to 7D with respect to the axial direction representing the spatial frequency in the vertical direction. That is, the frequency spectrum of the image DU1D is obtained by spreading the frequency spectrum shown in FIG. 7D two-dimensionally.

  In the following description, the spectrum SP2 is referred to as a folded component. This aliasing component appears on the image as noise or a false signal having a relatively high frequency component. Such noise or false signals include overshoot, jaggy or ringing.

The operation and effect of the image processing apparatus according to the present invention will be described below.
8A to 8E show the operation when the intermediate image D1 is generated from the input image DIN when the image DU1D obtained by enlarging the original image DORG is input as the input image DIN (or image DU1). 8A schematically shows the effect, FIG. 8A shows the frequency spectrum of the input image DIN, FIG. 8B shows the frequency response of the high frequency component image generating means 1, and FIG. 8C shows the frequency response. FIG. 8D shows the frequency response of the low-frequency component image generation means 2, FIG. 8D shows the frequency response of the intermediate image generation means 1, and FIG. 8E shows the frequency spectrum of the intermediate image D1. In FIGS. 8A to 8E, only one frequency axis is used for the same reason as in FIGS. 7A to 7D.

  Further, in FIGS. 8A to 8E, the intensity of the frequency spectrum or the frequency response is shown only in the range where the spatial frequency is 0 or more, but the frequency spectrum or the frequency response in the following description is on the frequency axis. It becomes a symmetric shape around the origin. Therefore, the figure used for description is sufficient to show only the range where the spatial frequency is 0 or more.

  First, the frequency spectrum of the input image DIN will be described. Since the image DU1D is input as the input image DIN, the frequency spectrum of the input image DIN has the same shape as that illustrated in FIG. 7D as shown in FIG. 8A, and the spectrum of the original image DORG. It consists of a spectrum SP1 in which the intensity of SPO has dropped to some extent and a spectrum SP2 that is a folded component.

  Next, the frequency response of the high frequency component image generating unit 1A will be described. Since the high frequency component image generating means 1A is constituted by a high pass filter, the frequency response thereof becomes lower as the frequency becomes lower as shown in FIG. 8B.

  Next, the frequency response of the low frequency component image generation means 1B will be described. Since the low frequency component image generating means 1B is composed of a low-pass filter, the frequency response becomes lower as the frequency becomes higher as shown in FIG. 8C.

Next, the frequency response of the intermediate image generating unit 1 will be described. Among the frequency components of the input image DIN, the frequency components in the low frequency component side region RL1 shown in FIG. 8D are weakened by the high frequency component image generating unit 1A in the intermediate image generating unit 1. On the other hand, the frequency component in the region RH1 on the high frequency component side shown in FIG. 8D is weakened by the low frequency component image generating unit 1B in the intermediate image generating unit 1. Therefore, as shown in FIG. 8D, the frequency response of the intermediate image generating means 1 has a peak in the intermediate region RM1 whose band is limited by the region RL1 on the low frequency component side and the region RH1 on the high frequency component side. It will have.
This intermediate region RM1 does not include the aliasing component generated when a pixel having a pixel value of 0 is inserted into the original image DORG, and is a part of the region below the Nyquist frequency Fn / 2 of the original image DORG. Occupy.

  Next, the frequency spectrum of the intermediate image D1 will be described. The input image DIN having the frequency spectrum shown in FIG. 8A passes through the intermediate image generating means 1 having the frequency response shown in FIG. 8D, whereby the intermediate image D1 is obtained. The frequency response of the intermediate image generating means 1 has a peak in the intermediate region RM1 band-limited by the region RL1 on the low frequency component side and the region RH1 on the high frequency component side, so the frequency spectrum of the intermediate image D1 is In the frequency spectrum of the input image DIN, the intensity of the portion included in the low frequency component side region RL1 and the high frequency component side region RH1 is weakened. Therefore, the intermediate image D1 is obtained by removing the spectrum SP1 that is the aliasing component from the high frequency component of the input image DIN. That is, the intermediate image generating means 1 has an effect of generating the intermediate image D1 by removing the spectrum SP1 that is the aliasing component from the high frequency component of the input image DIN.

  FIGS. 9A to 9C are diagrams showing the operation and effect of the intermediate image processing means 2, FIG. 9A shows the frequency spectrum of the nonlinear processed image D2A, and FIG. 9B shows the high frequency component. FIG. 9C shows the frequency response of the image 2B, and FIG. 9C shows the frequency spectrum of the image D2B. In FIGS. 9A to 9C, for the same reason as in FIGS. 8A to 8E, the intensity of the frequency spectrum or frequency response is shown only in the range where the spatial frequency is 0 or more.

  As will be described later, in the nonlinear processed image D2A, a high frequency component corresponding to the region RH2 on the high frequency component side is generated. FIG. 9A is a diagram schematically showing the state. The image D2B is generated by passing the nonlinear processed image D2A through the high frequency component image generating means 2B. The high frequency component image generating means 2B is composed of a high-pass filter, and the frequency response becomes higher as the frequency becomes higher as shown in FIG. 9B. Therefore, the frequency spectrum of the image D2B is obtained by removing the component corresponding to the region RL2 on the low frequency component side from the frequency spectrum of the nonlinear processed image D2A as shown in FIG. 9C. In other words, the nonlinear processing means 2A has an effect of generating a high frequency component corresponding to the region RH2 on the high frequency component side, and the high frequency component image generating means 2B has only the high frequency component generated by the nonlinear processing means 2A. There is an effect to take out.

The above operations and effects will be described in more detail.
FIGS. 10A to 10C and FIGS. 11A to 11C are diagrams showing step edges and signal values of pixels that are continuous when obtained by sampling the step edges.
FIG. 10A shows the step edge and the sampling interval S1, FIG. 10B shows the signal obtained when the step edge is sampled at the sampling interval S1, and FIG. 10 (B) represents the high frequency component of the signal. On the other hand, FIG. 11A shows a sampling interval S2 that is wider than the step edge and the sampling interval S1, and FIG. 11B shows a signal obtained when the step edge is sampled at the sampling interval S2. FIG. 11C shows high frequency components of the signal shown in FIG. In the following description, it is assumed that the length of the sampling interval S2 is half of the length of the sampling interval S1.

  As shown in FIGS. 10C and 11C, the center of the step edge appears as a zero cross point Z in the signal representing the high frequency component. In addition, the slope of the signal representing the high frequency component near the zero cross point Z becomes steeper as the sampling interval is short, and the position of the point giving the local maximum and minimum values near the zero cross point Z is also The shorter the sampling interval, the closer to the zero cross point Z.

  That is, even if the sampling interval changes, the position of the zero crossing point of the signal representing the high frequency component does not change near the edge, but the slope of the high frequency component near the edge decreases as the sampling interval decreases (or the resolution increases). The position of the point that gives the local maximum and minimum values approaches the zero-cross point.

  12A to 12F show the intermediate image generating means 1 and the intermediate image when the signal obtained by sampling the step edge at the sampling interval S2 is doubled and then input to the image processing apparatus according to the present invention. The operation of the processing means 2 is shown. As described above, since the internal processing of the intermediate image generating unit 1 and the intermediate image processing unit 2 is performed in each of the horizontal direction and the vertical direction, the processing is performed one-dimensionally. Therefore, in FIGS. 12A to 12F, the contents of the processing are represented using a one-dimensional signal.

  FIG. 12A shows a signal obtained by sampling the step edge at the sampling interval S2, as in FIG. 11B. FIG. 12B is a signal obtained by enlarging the signal shown in FIG. That is, when the original image DORG includes an edge as shown in FIG. 12A, a signal as shown in FIG. 12B is input as the input image DIN. Note that when the signal is doubled, the sampling interval is half that before the expansion, and therefore the sampling interval of the signal shown in FIG. 12B is the same as the sampling interval S1 in FIGS. 10A to 10C. Become. In FIG. 12A, the position represented by the coordinate P3 is the boundary portion on the low luminance side of the edge signal, and the position represented by the coordinate P4 is the boundary portion on the high luminance side of the edge signal.

  FIG. 12C shows a signal representing the high frequency component of the signal shown in FIG. 12B, that is, a signal corresponding to the image D1A output from the high frequency component image generating means 1A. Note that the image D1A is obtained by extracting the high-frequency component of the input image DIN, and therefore includes an aliasing component.

  FIG. 12D shows a signal representing the low frequency component of the signal shown in FIG. 12C, that is, a signal corresponding to the image D1B output from the low frequency component image generating means 1B. Since the image D1B is output as the intermediate image D1 as described above, FIG. 12D corresponds to the intermediate image D1. As shown in FIG. 12D, the local minimum value in the vicinity of the zero cross point Z in the intermediate image D1 appears at the coordinate P3, and the local maximum value appears at the coordinate P4, which is shown in FIG. 11C. The step edge coincides with the high frequency component extracted from the signal sampled at the sampling interval S2. Further, the aliasing component included in the image D1A is removed by a low-pass filter process performed by the low-frequency component image generation unit 1B.

  FIG. 12E shows an output signal when the signal shown in FIG. 12D is input to the nonlinear processing means 2A, that is, an image output from the nonlinear processing means 2A when the intermediate image D1 is input. D2A is represented. In the nonlinear processing means 2A, the signal values of the coordinates P1 and P2 before and after the zero cross point Z are amplified. Therefore, as shown in FIG. 12E, in the image D2A, the magnitude of the signal value at the coordinates P1 and P2 is larger than the other values, and the position where the local minimum value appears near the zero cross point Z is the coordinate. The position where the local maximum value appears at the coordinate P1 closer to the zero cross point Z from P3 changes from the coordinate P4 to the coordinate P1 closer to the zero cross point Z. This means that the high frequency component is generated by the nonlinear processing in the nonlinear processing means 2A that amplifies the values of the pixels before and after the zero cross point Z. In this way, it is possible to generate a high-frequency component by adaptively changing the amplification factor for each pixel or appropriately changing the content of processing according to the pixel. In other words, the nonlinear processing means 2A has an effect of generating a high frequency component not included in the intermediate image D1, that is, a high frequency component corresponding to the region RH2 on the high frequency component side shown in FIG. 9A. is there.

  FIG. 12F shows a signal representing the high frequency component of the signal shown in FIG. 12E, that is, a signal corresponding to the image D2B output from the high frequency component image generating means 2B. As shown in FIG. 12F, the local minimum value in the vicinity of the zero-cross point Z in the image D2B appears at the coordinate P1, and the maximum value appears at the coordinate P2. The state is shown by the step edge shown in FIG. This coincides with the high frequency component extracted from the signal sampled at the sampling interval S1. This means that the high frequency component generated in the nonlinear processing means 2A is taken out by the high frequency component image generating means 2B and output as an image D2B. Further, it can be said that the extracted image D2B is a signal including a frequency component corresponding to the sampling interval S1. In other words, the high frequency component image generation means 2B has an effect of extracting only the high frequency component generated by the nonlinear processing means 2A.

  The adding means 2C adds the intermediate image D1 and the image D2B to generate an image D2C. As described above, the intermediate image D1 is obtained by removing the aliasing component from the high frequency component of the input image DIN, and corresponds to the high frequency component near the Nyquist frequency of the original image DORG as shown in FIG. Yes. As described with reference to FIG. 7D, since the spectral intensity near the Nyquist frequency of the original image DORG is weakened by the enlargement process in the image enlargement means U1, it is weakened by the enlargement process by adding the intermediate image D1. Spectral intensity can be supplemented. Further, since the aliasing component is removed from the intermediate image D1, a false signal such as overshoot, jaggy, or ringing is not emphasized. On the other hand, the image D2B is a high frequency component corresponding to the sampling interval S1. Therefore, by adding the image D2C, a high frequency component in a band equal to or higher than the Nyquist frequency of the original image DORG can be given, so that the resolution of the image can be increased. Therefore, by adding the image D2C obtained by adding the intermediate image D1 and the image D2B to the input image DIN, it is possible to add a high-frequency component without enhancing the aliasing component, and the resolution of the image can be enhanced. It becomes.

  In the adding means 3, the image D2C is added to the input image DIN as the intermediate image D2. Therefore, it is possible to add a high frequency component while suppressing an increase in overshoot, jaggy, ringing, or the like due to the aliasing component, thereby enhancing the resolution of the image.

  Further, in the image processing apparatus according to the present invention, the intermediate image generation unit 1 and the intermediate image processing unit 2 perform the processing related to the horizontal direction of the image and the processing related to the vertical direction in parallel, so that only the horizontal direction of the image or the vertical direction. The above effects can be obtained not only in the direction but also in any direction.

  In the image processing apparatus according to the present invention, the component of the input image DIN in the vicinity of the Nyquist frequency ± Fn / 2 (or a specific frequency band) of the original image DORG in the frequency band extending from the origin to Fn in terms of frequency space. Based on the above, an image D2B corresponding to a high frequency component near the Nyquist frequency ± Fn is generated. Therefore, even if the frequency component near the Nyquist frequency ± Fn is lost in the input image DIN for some reason, the image D2B can provide the frequency component near the Nyquist frequency ± Fn.

  The location used as the specific frequency band is not limited to the vicinity of ± Fn / 2. That is, the frequency band to be used can be changed by appropriately changing the frequency response of the high-frequency component image generating unit 1A and the low-frequency component image generating unit 1B.

  In the above description, the image enlargement process is given as an example in which the frequency component near the Nyquist frequency Fn is lost. However, the cause of the loss of the frequency component near the Nyquist frequency Fn with respect to the input image DIN is not limited thereto. Also, noise removal processing or the like can be considered. Therefore, the application of the image processing apparatus in the present invention is not limited after the image enlargement process.

Embodiment 2. FIG.
FIG. 13 is a flowchart showing an image processing method according to the second embodiment of the present invention. The image processing method according to the second embodiment of the present invention includes an intermediate image generation step ST1, an intermediate image processing step ST2, and an addition step ST3. It is realized by.

  As shown in FIG. 14, the intermediate image generation step ST1 includes a high frequency component image generation step ST1A and a low frequency component image generation step ST1B.

  The high frequency component image generation step ST1A includes a horizontal high frequency component image generation step ST1Ah and a vertical high frequency component image generation step ST1Av, and the low frequency component image generation step ST1B includes a horizontal low frequency component image generation step ST1Bh, and The vertical high-frequency nest component image ST1Bv is included.

  As shown in FIG. 15, the intermediate image processing step ST2 includes a non-linear processing step ST2A, a high frequency component image generation step ST2B, and an addition step ST2C.

  The nonlinear processing step ST2A includes a horizontal nonlinear processing step ST2Ah and a vertical nonlinear processing step ST2Av, and the high frequency component image generation step ST2B includes a horizontal high frequency component passing step ST2Bh and a vertical high frequency component passing step ST2Bv. .

  As shown in FIG. 16, the horizontal non-linear processing step ST2Ah includes a zero-cross determination step ST311h and a signal amplification step ST312h. The vertical non-linear processing step ST2Av includes a zero-cross determination step ST311v and a signal amplification step as shown in FIG. Includes ST312v.

First, the operation of the intermediate image generation step ST1 will be described according to the flow of FIG.
In the high frequency component image generation step ST1A, the following processing is performed on the input image DIN input in an image input step (not shown). First, in the horizontal high-frequency component image generation step ST1Ah, an image D1Ah obtained by extracting a horizontal high-frequency component from the input image DIN is generated by horizontal high-pass filter processing. In the vertical high-frequency component image step ST1Av, an image D1Av obtained by extracting the high-frequency component in the vertical direction from the input image DIN is generated by high-pass filtering in the vertical direction. That is, the high frequency component image generation step ST1A performs the same processing as the high frequency component image generation means 1A, and generates an image D1A composed of the image D1Ah and the image D1Av from the input image DIN. This operation is equivalent to the high frequency component image generating means 1A.

  In the low frequency component image generation step ST1B, the following processing is performed on the image D1A. First, in the horizontal direction low frequency component image generation step ST1Bh, an image D1Bh obtained by extracting a horizontal low frequency component from the image D1Ah is generated by a horizontal low-pass filter process. In the vertical direction low frequency component image generation step STBv, an image D1Bv obtained by extracting the low frequency component in the vertical direction from the image D1Av is generated by the low pass filter processing in the vertical direction. That is, the low frequency component image generation step ST1B performs the same processing as the low frequency component image generation means 1B, and generates an image D1B composed of the image D1Bh and the image D1Bv from the image D1A. This operation is equivalent to the low frequency component image generation means 1B.

  The operation of the intermediate image generation step ST1 is as described above. In the intermediate image generation step ST1, the image D1Bh is set as the image D1h, the image D1Bv is set as the image D1v, and the intermediate image D1 composed of the image D1h and the image D1v is output. The above operation is the same as that of the intermediate image generating unit 1.

Next, the operation of the intermediate image processing step ST2 will be described with reference to FIGS.
First, in the nonlinear processing step ST2A, the following processing is performed on the intermediate image D1.

  First, in the horizontal non-linear processing step ST2Ah, an image D2Ah is generated from the image D1h by processing according to the flow shown in FIG. The processing in the flow shown in FIG. 16 is as follows. First, in the zero cross determination step ST311h, a change in pixel value in the image D1h is confirmed along the horizontal direction. Then, a portion where the pixel value changes from a positive value to a negative value or from a negative value to a positive value is regarded as a zero cross point, and the pixels located on the left and right of the zero cross point are notified to the signal amplification step ST312h. In the signal amplification step ST312h, for the image D1h, the pixel value of the pixel notified to be positioned on the left and right of the zero cross point is amplified, and the image is output as the image D2Ah. That is, the non-linear processing step ST2Ah performs the same processing as the horizontal non-linear processing means 2Ah on the image D1h to generate the image D2Ah.

  Next, in the vertical non-linear processing step ST2Av, an image D2Av is generated from the image D1v by processing according to the flow shown in FIG. Processing in the flow shown in FIG. 17 is as follows. First, in the zero cross determination step ST311v, a change in pixel value in the image D1v is confirmed along the vertical direction. Then, a point where the pixel value changes from a positive value to a negative value or from a negative value to a positive value is regarded as a zero cross point, and the pixels located above and below the zero cross point are notified to the signal amplification step ST312v. In the signal amplification step ST312v, the pixel value of the pixel notified to be positioned above and below the zero cross point is amplified for the image D1v, and the image is output as the image D2Av. That is, the nonlinear processing step ST2Av performs the same processing as the vertical nonlinear processing means 2Av on the image D1v to generate the image D2Av.

  The above is the operation of the non-linear processing step ST2A, and the non-linear processing step ST2A generates an image D2A composed of the image D2Ah and the image D2Av. The operation is equivalent to the nonlinear processing means 2A.

  Next, in the high frequency component image generation step ST2B, the following processing is performed on the image D2A.

  First, in the horizontal direction high frequency component image generation step ST2Bh, an image D2Bh obtained by performing a high-pass filter process in the horizontal direction on the image D2Ah is generated. That is, the horizontal direction high frequency component image generation step ST2Bh performs the same processing as the horizontal direction high frequency component image generation means 2Bh.

  Next, in the vertical direction high-frequency component image generation step ST2Bv, an image D2Bv obtained by performing vertical high-pass filter processing on the image D2Av is generated. That is, the vertical high frequency component image generation step ST2Bv performs the same processing as the vertical high frequency component image generation means 2Bv.

  The above is the operation of the high frequency component image generation step ST2B, and the high frequency component image generation step ST2B generates an image D2B composed of the image D2Bh and the image D2Bv. The operation is the same as that of the high frequency component image generating means 2B.

  In the addition step ST2C, the image D2B and the intermediate image D1 are added to generate an image D2C. At this time, the addition of the images D2B and D1 may be weighted addition. This operation is equivalent to the adding means 2C.

  The above is the operation of the intermediate image processing step ST2, and the intermediate image processing step ST2 outputs the image D2C as the intermediate image D2. This operation is equivalent to the intermediate image processing means 2.

In addition step ST3, the input image DIN and the intermediate image D2 are added to generate an output image DOUT. The output image DOUT is output as the final output image of the image processing method in the present invention. That is, the operation of the adding step ST3 is equivalent to the operation of the adding means 3.
The above is the operation of the image processing method according to the present invention.

  As is apparent from the description, the operation of the image processing method according to the present invention is equivalent to that of the image processing apparatus according to the first embodiment of the present invention. Therefore, the image processing method according to the present invention has the same effect as the image processing apparatus according to the first embodiment of the present invention. Further, the image processed by the above-described image processing method can be displayed by the image display device shown in FIG.

It is a block diagram which shows the structure of the image processing apparatus by Embodiment 1 of this invention. It is a block diagram which shows the structure of the horizontal direction nonlinear processing means 2Ah of FIG. FIG. 2 is a block diagram showing a configuration of vertical non-linear processing means 2Av in FIG. 1. It is a block diagram which shows the structure of the image display apparatus using the image processing apparatus of FIG. It is a block diagram which shows the structure of the image expansion means U1 of FIG. (A)-(E) is a pixel arrangement | positioning figure which shows operation | movement of the image expansion means U1 of FIG. (A)-(D) are the frequency response figure and frequency spectrum figure which show operation | movement of the image expansion means U1 of FIG. (A)-(E) are the frequency response figure and frequency spectrum figure which show the operation | movement of the intermediate | middle image generation means 1 of FIG. (A)-(C) are the frequency response figure and frequency spectrum figure which show operation | movement of the intermediate | middle image processing means 2. FIG. . (A)-(C) is a figure which shows the value of the signal of the pixel which continues when a step edge and a step edge are sampled by sampling interval S1. (A)-(C) is a figure which shows the value of the signal of the pixel which continues when a step edge and a step edge are sampled by sampling interval S2. (A)-(F) is a figure which shows the value of the signal of the continuous pixel which shows operation | movement of the intermediate image generation means 1 and the intermediate image processing means 2 of FIG. It is a flowchart which shows the image processing method by Embodiment 2 of this invention. It is a flowchart which shows the process in intermediate | middle image generation step ST1 of FIG. It is a flowchart which shows the process in intermediate image process step ST2 of FIG. It is a flowchart which shows the process in horizontal direction nonlinear process step ST2Ah of FIG. FIG. 16 is a flowchart showing processing in the vertical nonlinear processing step ST2Av in FIG. 15.

Explanation of symbols

1 intermediate image generation means, 2 intermediate image processing means, 3 addition means, Din input image, Dout output image.

Claims (10)

Intermediate image generating means for generating a first intermediate image obtained by extracting a component in the vicinity of a specific frequency band of the input image;
Intermediate image processing means for generating a second intermediate image based on the first intermediate image;
In an image processing apparatus having a first addition means for adding the input image and the second intermediate image,
The image processing apparatus, wherein the intermediate image processing means includes non-linear processing means for changing processing contents in accordance with pixels of the first intermediate image.   The image processing apparatus according to claim 1, wherein the intermediate image generation unit removes only a predetermined frequency component by removing a low frequency component and a high frequency component of the input image. The intermediate image generating means
First high frequency component image generation means for generating a first high frequency component image obtained by extracting only the high frequency component of the input image;
The image processing apparatus according to claim 1, further comprising a low-frequency component image generation unit that extracts only a low-frequency component of the first high-frequency component image. The nonlinear processing means includes:
Generating a non-linearly processed image obtained by amplifying each pixel value of the first intermediate image with an amplification factor changed according to the pixel,
The intermediate image processing means includes
Second high frequency component image generation means for generating a second high frequency component image obtained by extracting only the high frequency component of the nonlinear processed image;
The image processing apparatus according to claim 1, further comprising second addition means for adding the first intermediate image and the second high-frequency component image. The first high frequency component image generation means includes:
First horizontal high-frequency component image generation means for generating a first horizontal high-frequency component image obtained by extracting a high-frequency component using pixels existing in the horizontal direction of each pixel of the input image;
First vertical high-frequency component image generating means for generating a first vertical high-frequency component image obtained by extracting a high-frequency component using pixels existing in the vertical direction of each pixel of the input image; ,
The low frequency component image generation means includes
Horizontal low-frequency component image generation means for generating a first horizontal intermediate image obtained by extracting only the low-frequency component of the first horizontal high-frequency component image;
The vertical direction low frequency component image generation means which produces | generates the 1st vertical direction intermediate image which took out only the low frequency component of the said 1st vertical direction high frequency component image is characterized by the above-mentioned. Image processing device. The first intermediate image is
Consisting of the first horizontal intermediate image and the first vertical intermediate image,
The nonlinear processing means includes:
Horizontal non-linear processing means for generating a horizontal non-linear processed image obtained by amplifying each pixel value of the first horizontal intermediate image with an amplification factor changed according to the pixel;
Vertical direction nonlinear processing means for generating a vertical direction nonlinear processing image obtained by amplifying each pixel value of the first vertical direction intermediate image with an amplification factor changed according to the pixel,
The second high frequency component image generating means includes:
Second horizontal high frequency component image generation means for generating a second horizontal high frequency component image obtained by extracting only the high frequency component of the horizontal nonlinear processed image;
The second vertical high-frequency component image generation means for generating a second vertical high-frequency component image obtained by extracting only the high-frequency component of the vertical non-linear processing image. Image processing device. The horizontal non-linear processing means includes:
Horizontal zero-cross point determining means for determining a point where the pixel value of the first horizontal intermediate image changes from positive to negative or from negative to positive as a zero-cross point;
Horizontal signal amplification means for determining an amplification factor for each pixel of the first horizontal intermediate image according to a determination result of the horizontal zero-cross point determination means;
The vertical nonlinear processing means includes:
Vertical zero-cross point determination means for determining a point where the pixel value of the first vertical intermediate image changes from positive to negative or from negative to positive as a zero-cross point;
7. The image according to claim 6, further comprising: a vertical direction signal amplification unit that determines an amplification factor for each pixel of the first intermediate image in the first vertical direction according to a determination result of the vertical direction zero cross point determination unit. Processing equipment.   An image display device comprising the image processing device according to claim 1. An intermediate image generation step of generating a first intermediate image obtained by extracting a component in the vicinity of a specific frequency band of the input image;
An intermediate image processing step of generating a second intermediate image based on the first intermediate image;
In an image processing method including an adding step of adding the input image and the second intermediate image,
The intermediate image processing step includes a non-linear processing step of changing a processing content according to a pixel of the first intermediate image.   An image display device for displaying an image processed by the image processing method according to claim 9.

Images