JP2001298621A - Image processor and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor by which the contrast and sharpness of an entire image are enhanced more than heretofore. SOLUTION: An edge of which the change of the pixel value is sharp is kept as it is and input image data S1 is amplified except this edge to emphasize and display the part other than the edge, and thus the contrast and sharpness of the entire image are enhanced mote than heretofore.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and an image processing method, and is suitably applied to, for example, a video camera.

[0002]

2. Description of the Related Art Conventionally, in a video camera, as a method for improving the contrast (difference in brightness and darkness) and sharpness (clarity of a boundary) of an image picked up by a solid-state image pickup device (CCD: Charge Coupled Device), there is known a method for improving the contrast. A contrast enhancement method by tone conversion and a high-frequency component enhancement method for enhancing the contrast of a high-frequency component in an image have been considered.

[0003] Contrast enhancement methods include tone curve adjustment for converting the pixel level of each pixel of an image with a function having a predetermined input / output relationship (hereinafter referred to as a level conversion function), and pixel level adjustment. A method called histogram equalization that adaptively changes the level conversion function according to the frequency distribution of has been proposed, and as a high-frequency component enhancement method, an edge is extracted from an image,
A method called an unsharp mask that performs so-called contour enhancement for enhancing the extracted edge has been proposed.

[0004]

However, in the contrast enhancement method, there is a problem that the contrast can be improved only in a part of the luminance range in the entire dynamic range (difference between the maximum level and the minimum level) of the image. In addition, in the case of tone curve adjustment, there is a problem that the contrast is reduced in the brightest part and the darkest part of the image, and in the case of histogram equalization, in the vicinity of a luminance region where the frequency distribution is small. Further, in the high-frequency component emphasizing method, there is a problem that only the contrast of the high-frequency component of the image is enhanced, whereby the vicinity of the edge of the image is unnaturally enhanced, and the image quality is unavoidably deteriorated.

The present invention has been made in view of the above points, and an object of the present invention is to propose an image processing apparatus and a method thereof capable of further improving the contrast and sharpness of the entire image as compared with the related art.

[0006]

According to the present invention, in order to solve the above-mentioned problem, a portion other than the edge of the input image data is amplified while preserving the edge where the pixel value changes sharply. Can be highlighted.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below in detail with reference to the drawings.

(1) First Embodiment In FIG. 1, reference numeral 1 denotes an overall configuration of a video camera according to a first embodiment, and a solid-state imaging device (CCD: Charge:
The input image data S1 captured by the coupled device 2 is input to the delay circuit 4 and the non-linear smoother 5 of the image processing circuit 3. Here, the input image data S1 is a two-dimensional digital image, and a pixel value corresponding to a position (i, j) on the image is represented as x (i, j).

The nonlinear smoother 5 extracts an edge component having a sharp change in pixel value from the input image data S1, and outputs the edge component as it is without smoothing. By smoothing small amplitude components other than edge components, the input image data S1
The input image data S1 is smoothed while retaining the edge components of.

More specifically, the nonlinear smoother 5 inputs the input image data S1 to the linear low-pass filter 10, as shown in FIG. The linear low-pass filter 10 attenuates the signal level of an extremely high frequency component of the input image data S1, and sends out the resulting image data S2 to the look-up table 11. As described above, the linear low-pass filter 10 attenuates the signal level of an extremely high frequency component of the input image data S1, so that the high-frequency component having a large amplitude is sufficiently smoothed when performing the smoothing process by the ε filter 12 at the subsequent stage. Instead, it is prevented from remaining on the image as point-like noise. Incidentally, the linear low-pass filter 10 is configured by applying a one-dimensional linear low-pass filter in the horizontal and vertical directions of an image, respectively, or is configured by a two-dimensional linear low-pass filter.

The look-up table 11 performs tone conversion such as logarithmic conversion on the image data S2,
The image data S3 obtained as a result is sent to the ε filter 12A. By the way, the ε filter 12 at the subsequent stage performs an adaptive smoothing process according to the amplitude of the image data. In general, the magnitude of the amplitude of the image data is proportional to the intensity of the illumination light illuminating the imaging target. It becomes bigger. Therefore, the look-up table 11 performs the logarithmic transformation on the image data S2 in advance, so that the same smoothing effect can be obtained regardless of the lighting conditions when performing the smoothing processing in the ε filter 12 in the subsequent stage. . At the same time, the look-up table 11 can also control the smoothing effect according to the pixel value, such as increasing the degree of enhancement in the subsequent image enhancement process by increasing the smoothing effect in, for example, dark or bright regions of the image. enable.

Ε filter 1 which is a nonlinear smoothing filter
2A is a digital filter effective for smoothing the pixel value without impairing the sharp change of the pixel value,
Image data S supplied from the lookup table 11
The image data S3 is smoothed while the edge of No. 3 is kept, and the resulting smoothed image data S4A is sent to the ε filter 12B. The smoothing process in the ε filter 12A is performed when the pixel for the filter process is one-dimensional and has 2N + 1 taps.

[0013]

(Equation 1)

Is represented by

That is, the ε filter 12A has a pixel value xn of the central pixel pn and a pixel value xn of the pixel pn-k.
The absolute value | xn−xn−k | of the difference from nk is compared with a predetermined threshold value ε. As a result, the ε filter 12A obtains the absolute value | x
If it is determined that n−xn−k | is smaller than a predetermined threshold ε, the same processing as that of a normal linear low-pass filter in which the pixel value xn−k is substituted for wn−k and ak is each tap coefficient. Is performed, the image is uniformly smoothed about the center pixel pn.

On the other hand, when the ε filter 12A determines that the absolute value | xn−xn−k | is larger than the predetermined threshold value ε, the ε filter 12A substitutes the pixel value xn into wn−k, and the pixel pn− k
The pixel value xn-k is replaced with the pixel value xn of the center pixel pn, and then a low-pass filter process is performed with the center pixel pn as the center, so that the pixel value near the pixel value xn is ignored ignoring the pixel value xn-k. Performs smoothing.

As shown in FIG. 3, for example, as shown in FIG. 3, when the absolute value of the difference between the pixel values before and after the steep edge exceeds a predetermined threshold value ε, the ε filter 12A performs low-pass filter processing centering on the center pixel pn. By performing low-pass filter processing by replacing the pixel value xm of the pixel pm with the pixel value xn of the center pixel pn, smoothing is performed in the vicinity of the pixel value xn, whereas low-pass filter processing is performed around the pixel pm. When performing this, smoothing is performed near the pixel value xm.

At this time, the ε filter 12A outputs the pixel value x of the pixel p at the edge portion as it is, for the pixel p at the edge portion where the pixel p having a pixel value close to the pixel value x does not exist within the range to be filtered. , The sharp change of the edge is preserved as it is. Incidentally, the ε filter 12A
As in the case of the linear low-pass filter 10, there are a case where the one-dimensional ε filter is applied in the horizontal direction and the vertical direction of the image, and a case where the two-dimensional ε filter is formed.

At the subsequent stage of the ε filter 12A, the ε filter 12 having the same configuration as the ε filter 12A is used.
B to 12N are sequentially connected, and the smoothing processing is sequentially performed on the smoothed image data S4A to improve the smoothing effect. In this manner, the smoothed image data S4 in which components other than the edges are sufficiently smoothed
N is obtained and supplied to the lookup table 13 at the subsequent stage.

Here, FIG. 4 shows that the number of pixels for filter processing is one.
The configuration of the ε filter 12A in the case of 7 taps in the dimension is shown, showing the register rows 20A to 20F and the selectors 21A to 21A.
F, amplifiers 23A to 23G, and an adder 24. As shown in FIG. 5, the register row 20A includes registers 22A to 22A each holding a pixel value of one pixel.
E are connected in series, and the register rows 20B to 20F are configured similarly to the register row 20A.

As shown in FIG. 6, the selector 21A adds the pixel value xn of the center pixel pn shown in FIG.
And the selector 31 and the pixel value xm of the pixel pm to the adder 30 and the selector 31. The adder 30 calculates the difference between the pixel value xn and the pixel value xm, and sends the result to the absolute value converter 32. The absolute value converter 32
The absolute value | xn-xm | of the difference between the pixel value xn and the pixel value xm is obtained and sent to the magnitude comparator 33.

The magnitude comparator 33 calculates the absolute value | xn-x
is compared with a predetermined threshold value ε, and the comparison result is sent to the selector 31. The selector 31 calculates the absolute value | xn-x
is smaller than a predetermined threshold ε, the pixel value xm
Is selected and sent to the subsequent amplifier 23A,
If the absolute value | xn-xm | is larger than the predetermined threshold value ε, the pixel value xn is selected and sent to the subsequent amplifier 23A.

Each of the selectors 21B to 21F has the same configuration as the selector 21A, and sends the pixel value x of the selected pixel p to the corresponding subsequent amplifiers 23B and 23C and 23E to 23G. The amplifier 23D is supplied with the pixel value xn of the center pixel pn from the register row 20C.

The amplifiers 23A to 23G are for multiplying the input pixel value x by the tap coefficients ak, respectively, and transmit the operation results to the adder 24, respectively. , And sends this to the ε-filter 12B as smoothed image data S4A.

In this smoothing process, it is desirable to generate an image with as little change as possible in portions other than the edges. For this purpose, it is necessary to use a very large filter in the ε filter 12A. However, if the filter used is increased, the tap coefficients ak
Is multiplied by the selectors 21A to 21F and the amplifier 2 in the subsequent stage.
The number of 3A to 23G must also be increased, which increases the circuit scale. Therefore, in the present embodiment, each register row 20 is composed of registers 22 for several pixels, and the subsequent arithmetic processing is performed only on the first pixel of each register row 20. This realizes an ε filter that covers a wide range spatially without increasing the circuit scale of the subsequent stage. For example, when the number of the registers 22 is 2 for a 7-tap ε filter, 13 pixels × 1 as shown in FIG.
A filter having a size of three pixels can be configured. However, while a filter having a significant coefficient only every few pixels in this way can expect a large smoothing effect, it generally has a tendency to have a large side lobe, and unnecessary high frequency components remain in the smoothed image data. Become responsive. FIG. 14 shows a comparison of the frequency response of a filter configured by changing the number of the registers 22. The horizontal axis is Nyquist frequency 0.5
Where the vertical axis represents the frequency response of the filter. Filter A, Filter B, and Filter
C corresponds to the case where the number of the registers 22 is 1, 2, and 3, respectively.
As shown in FIG. FIG. 14 also shows that the side lobes increase as the number of registers 22 increases.
Therefore, in this embodiment, in order to avoid this problem, smoothing is performed by connecting a plurality of ε filters 12A to 12N having the same configuration in series except for the number of registers 22 in the register rows 20A to 20F. The effect and the suppression of the side lobe are compatible. That is, while a large smoothing effect is obtained by the ε filter having a large number of registers 22,
Unnecessary high frequency components that have passed through the side lobes of other filters are removed by an ε filter with a small number of registers 22. Here, since the ε filter has a property of passing high frequency components around the edge, the register 22
If an ε-filter with a small number is used first, there is a possibility that the frequency components passing through the main lobe may not be sufficiently attenuated by the ε-filter at the subsequent stage near the edge.
Therefore, in order to obtain a good smoothing effect, it is necessary to use the resistor 2
It is desirable to apply them in ascending order of 2.

The look-up table 13 performs an inverse conversion of the logarithmic conversion performed by the look-up table 11 on the smoothed image data S4N supplied from the ε filter 12N, and converts the resulting smoothed image data S10. The signal is sent to the linear low-pass filter 35. The linear low-pass filter 35 generates smoothed image data S11 by slightly dulling the edges of the smoothed image data S10 smoothed while preserving the edges, and outputs the smoothed image data S11 to the adders 40 and 41 (FIG. 1). Send out. As a result, it is possible to maintain the smoothness of the image near the edge in the subsequent image enhancement processing.

Returning to FIG. 1, the delay circuit 4 delays the input image data S1 picked up by the CCD 2 by a predetermined amount and sends it to the adder 40 as delayed image data S15. The adder 40 constitutes an image enhancing means together with the adder 41 and the multiplier 42, and converts each pixel value x (i, j) of the delayed image data S15 to each pixel value s (i, j) of the smoothed image data S11. Is subtracted, and the resulting difference image data S16 is sent to the multiplier 42.

The multiplier 42 multiplies each pixel value (x (i, j) -s (i, j)) of the difference image data S16 by a gain coefficient g (i, j) and amplifies the result. The obtained difference image data S17 is sent to the adder 41. Here, the gain coefficient g (i, j) uses a uniform value for the entire screen or a value set for each pixel (i, j).

The adder 41 calculates each pixel value g (i, j) × (x) of the differential image data S17 supplied from the multiplier 42.
(I, j) -s (i, j)), the smoothed image data S which is the offset subtracted by the adder 40
The respective pixel values s (i, j) of No. 11 are added, and the resulting output image data S18 is sent to the camera signal processing circuit 45. The camera signal processing circuit 45 outputs the output image data S18
Is subjected to predetermined data processing, and the resulting output image data S19 is converted to a VTR (Video Tape Recorder) 46.
And record it.

Each pixel value y (i, j) of the output image data S18 output from the image processing circuit 3 is expressed by the following equation.

[0031]

(Equation 2)

Is represented by In the equation (2), the gain coefficient g (i, j) is replaced with a uniform gain coefficient G for the entire screen, and each pixel value s (i, j) of the smoothed image data S11 is replaced with the input image data S1. When replaced by the median value of the dynamic range or the average value C of all pixel values, this equation (2) can be expressed by the following equation showing a conventional contrast enhancement method by tone curve adjustment.

[0033]

(Equation 3)

It is expressed as follows.

In the above equation (2), the gain coefficient g (i, j) is replaced with a uniform gain coefficient G for the entire screen, and each pixel value s of the smoothed image data S11 is changed.
(I, j) is replaced with each pixel value x (i,
Substituting each pixel value f (i, j) of the image data obtained by performing the linear low-pass filter processing on j), this equation (2) is expressed by the following equation showing a conventional high-frequency component emphasizing method using an unsharp mask.

[0036]

(Equation 4)

It is expressed as follows.

As described above, the conventional contrast enhancement method is a method in which image enhancement processing is performed independently for each pixel of an image, and the conventional high-frequency component enhancement method is a method in which peripheral pixels existing around a central pixel are used. In contrast to the method of performing image enhancement processing based on a relative level difference with pixels, the method according to the present embodiment employs a conventional contrast enhancement method and a high-frequency component enhancement method using a nonlinear smoothing filter. To enable even higher quality image enhancement processing.

In the above configuration, the image processing circuit 3
The input image data S1 is smoothed while the edge component of the input image data S1 is preserved by the non-linear smoother 5, to generate smoothed image data S11. Then, the image processing circuit 3 subtracts the smoothed image data S11 from the delayed image data S15, which is the original image, as an offset,
After multiplying the result of the subtraction by a gain coefficient, smoothed image data S11 as an offset is added.

Accordingly, it is possible to enhance the signal level of the small amplitude component other than the edge component while preserving the edge component of the input image data S1, thereby improving the contrast of the entire image while maintaining the dynamic range of the image. And the sharpness of the image can be improved even near the edges. As a result, the video camera 1 can shoot a hazy distant view or in a fog,
A clear image with a secured dynamic range and contrast can be taken, and high-quality image enhancement processing can be performed.

According to the above configuration, the input image data S1
After the input image data S1 is smoothed while the edge components of the input image data are preserved to generate smoothed image data S11, the smoothed image data S11 is subtracted from the input image data S1 as an offset, and a gain coefficient is added to the subtraction result. , And adding the smoothed image data S11, it is possible to emphasize the signal level of a small amplitude component other than the edge component in the input image data S1 while preserving the edge component. While maintaining the range, the contrast and sharpness of the entire image can be further improved as compared with the related art.

(2) Second Embodiment FIG. 7 in which parts corresponding to those in FIG.
10 shows a video camera 50 according to the third embodiment, and has the same configuration as the video camera 1 according to the first embodiment except for the configuration of an image processing circuit 51.

In the case of this embodiment, the nonlinear smoother 5
Sends the smoothed image data S11 to the adder 40 and the delay circuit 52. The adder 40 outputs the delayed image data S15
, And subtracts the smoothed image data S11 from the data, and sends the resulting difference image data S16 to the noise removal filter 53.

The noise removal filter 53 reduces the noise component of the difference image data S16, and sends the resulting difference image data S30 to the multiplier 42. The noise removal filter 53 has input / output characteristics as shown in FIG. 8, for example.
A gradation conversion called coring is performed to reduce noise components by not outputting small amplitude components in the range indicated by the arrow in the figure. As a result, the S / N ratio (signal-to-noise ratio) of the image data output from the image processing circuit 51 due to the amplification of the noise component in the multiplier 42 at the subsequent stage is prevented from deteriorating.

The multiplier 42 multiplies each pixel value of the difference image data S30 by a gain coefficient and amplifies the result, and sends the resulting difference image data S31 to the adder 41.
The delay circuit 52 delays the smoothed image data S11 by a predetermined amount, and delays the smoothed image data S11 as delayed smoothed image data S32.
Send to 1.

The adder 41 adds each pixel value of the smoothed image data S32, which is an offset, to each pixel value of the difference image data S31 supplied from the multiplier 42,
The resulting output image data S33 is sent to the look-up table 54. The look-up table 54 performs gradation conversion on the output image data S33 to compress, for example, the vicinity of black and the vicinity of white as shown in FIG. 9, and sends the resulting output image data S34 to the camera signal processing circuit 45. I do. As a result, even if the pixel value exceeds the dynamic range on the black or white side due to the addition processing of the adder 41, clipping distortion in which the gradation is lost near black or white is prevented.

In the above configuration, the image processing circuit 51
Generates the smoothed image data S11 by smoothing the input image data S1 while preserving the edge component of the input image data S1 by the nonlinear smoother 5. Then, the image processing circuit 51 subtracts the smoothed image data S11 as an offset from the delayed image data S15 as the original image, multiplies the result of the subtraction by a gain coefficient, and outputs the smoothed image data S11 as the offset. to add.

Therefore, it is possible to enhance the signal level of the small amplitude component other than the edge component while preserving the edge component of the input image data S1, thereby improving the contrast of the entire image while maintaining the dynamic range of the image. And the sharpness of the image can be improved even near the edges.

According to the above configuration, the input image data S1
After the input image data S1 is smoothed while the edge components of the input image data are preserved to generate smoothed image data S11, the smoothed image data S11 is subtracted from the input image data S1 as an offset, and a gain coefficient is added to the subtraction result. , And adding the smoothed image data S11, it is possible to emphasize the signal level of a small amplitude component other than the edge component in the input image data S1 while preserving the edge component. While maintaining the range, the contrast and sharpness of the entire image can be further improved as compared with the related art.

Further, by reducing the noise component of the differential image data S16 in the noise removing filter 53, it is possible to prevent the noise component from being amplified in the multiplier 42 at the subsequent stage and the S / N ratio of the output image data S34 from being degraded. . Further, in the lookup table 54, the output image data S
By performing tone conversion for compressing the vicinity of black and the vicinity of white on 33, clipping distortion can be prevented even when the pixel value exceeds the dynamic range.

(3) Third Embodiment In FIG. 10, reference numeral 60 denotes a video camera 60 according to the third embodiment as a whole, in which a color image is captured by a CCD 61, and the captured color image is converted into an input image. The data is input to the color separation circuit 62 as data S40. The color separation circuit 62 separates the input image data S40 into image data of three primary colors of R (red), G (green), and B (blue), and converts the green image data into green image data S40.
G is input to the image processing circuit 3A of the image processing unit 63,
The red image data is input to the image processing circuit 3B of the image processing unit 63 as red image data S40R, and the blue image data is input to the image processing circuit 3C of the image processing unit 63 as blue image data S40B.

The image processing circuits 3A to 3C correspond to FIG.
Has the same configuration as that of the image processing circuit 3 shown in (1), and performs processing for enhancing the signal level of the small amplitude component other than the edge component on the input image data S40G, S40R, and S40B while preserving the edge component. It has been made like that.

That is, the image processing circuit 3A smoothes the green image data S40G while maintaining the edge components of the green image data S40G to generate smoothed green image data, and then converts the smoothed green image data into an offset component. Is subtracted from the green image data S40G, the subtraction result is multiplied by a gain coefficient, and then the smoothed green image data is added to generate green output image data S41G and send it to the camera signal processing circuit 64.

Similarly to the image processing circuit 3A, the image processing circuits 3B and 3C perform the above-described processing on the red image data S40R and the blue image data S40B, respectively, so that the red output image data S41R and the blue output image data S41B is generated and sent to the camera signal processing circuit 64, respectively.

The camera signal processing circuit 64 performs predetermined data processing on each of the green output image data S41G, the red output image data S41R and the blue output image data S41B, and converts the resulting output image data S42 into a VT
Send to R65 for recording.

In the above configuration, the image processing circuits 3A to 3A
3C stores the image data S40G, S40R, and S40B while maintaining the edge components of the image data S40G, S40B.
40R and S40B are respectively smoothed to generate smoothed image data. Then, the image processing circuits 3A to 3C subtract the smoothed image data from the original image data S40G, S40R, and S40B, respectively, as an offset, and multiply the subtraction result by a gain coefficient. Coded image data are added.

Accordingly, it is possible to enhance the signal levels of the small amplitude components other than the edge components while preserving the edge components of the image data S40G, S40R, and S40B, thereby maintaining the dynamic range of the image and the entire image. Can be improved, and the sharpness of an image can be improved even near the edge.

According to the above configuration, the image data S40
After the image data S40G, S40R, and S40B are respectively smoothed to generate smoothed image data while maintaining the edge components of G, S40R, and S40B, the image data S40 is generated using the smoothed image data as an offset.
G, S40R, and S40B are subtracted from each other, and the subtraction result is multiplied by a gain coefficient, and then the smoothed image data is added.
The signal level of a small amplitude component other than the edge component can be emphasized while the edge component is preserved among the 40R and S40B. Thus, while maintaining the dynamic range of the image, the contrast and sharpness of the entire image are further enhanced as compared with the related art. The degree can be improved.

Further, the color image picked up by the CCD 61 is separated into three primary colors of RGB, and small-amplitude components other than the edge components are retained for the separated image data S40G, S40R, and S40B while retaining the edge components. Even if a color image is input by performing image processing that emphasizes the signal levels of
It is possible to further improve the contrast and sharpness of the entire image as compared with the related art while maintaining the dynamic range of the image.

(4) Other Embodiments In the above-described second embodiment, a case has been described in which the noise component is reduced from the difference image data S16 by the noise removal filter 53 using a technique called coring. However, the present invention is not limited to this, and a noise removal filter may be configured by the ε filter 12A shown in FIG. 4 and only the small amplitude component of the difference image data S16 may be smoothed to reduce the noise component.

In the above-described second embodiment,
Although the case where the noise removal filter 53 is disposed after the adder 40 has been described, the present invention is not limited to this, and the point is that the noise removal filter 53 only needs to be located before the multiplier 42. It may be arranged in the former stage or the latter stage.

In the third embodiment described above,
The input image data S40 is converted to RGB by the color separation circuit 62.
However, the present invention is not limited to this, and the present invention is not limited to this. The luminance signal Y is subtracted from the blue signal B and the color difference signal RY obtained by subtracting the luminance signal Y from the luminance signal Y and the red signal R. May be separated into the color difference signals BY.

In the third embodiment described above,
The ε filter in the image processing circuits 3A to 3C is one-dimensional or two-dimensional.
Although the case has been described in which the filter is constituted by a three-dimensional ε filter, the present invention is not limited to this, and may be constituted by a three-dimensional ε filter.

In this case, as shown in FIG. 11, in which parts corresponding to those in FIG. 10 are assigned the same reference numerals, the image processing unit 71 of the video camera 70 converts the green image data S40G supplied from the color separation circuit 60 into The image data is input to the image processing circuits 72A to 72C, and the red image data S40R is input to the image processing circuits 72A to 72C.
72C, and the blue image data S40B is input to the image processing circuits 72A to 72C.

For example, when the pixel value of the green image data S40G is gn, the pixel value of the red image data S40R is rn, and the pixel value of the blue image data S40B is bn, the image processing circuits 72A to 72C perform the above-mentioned (1). Instead of the absolute value of the difference between the pixel value xn and the pixel value xn-k | xn-xn-k |

[0066]

(Equation 5)

By using each of them, more effective smoothing is performed.

In the third embodiment described above,
The input image data S40 is separated into three primary colors of RGB, and the separated image data S40G, S40R, and S40B are separated.
Although the case where the image enhancement processing is applied to each is described, the present invention is not limited to this, and the image enhancement processing may be performed only on the luminance data of the input image data S40 to reduce the overall calculation amount. .

As shown in FIG. 12, in which parts corresponding to those in FIG. 10 are assigned the same reference numerals, the image processing unit 81 of the video camera 80 includes green image data S40G and red image data supplied from the color separation circuit 62. S40R and the blue image data S40B are input to the matrix circuit 82 of the image processing unit 81. The matrix circuit 82 stores the green image data S40
G, red image data S40R and blue image data S40
B is the color difference data S50 obtained by subtracting the luminance data S50A from the luminance data S50A and the red image data S40R.
B and the luminance data S5 from the blue image data S40B.
0A is converted into color difference data S50C obtained by subtracting 0A, and the luminance data S50A is converted into the image processing circuit 3A and the divider 8
3A and 83B, the color difference data S50B is sent to the divider 83A, and the color difference data S50C is sent to the divider 83B.

The image processing circuit 3A has the same configuration as that of the image processing circuit 3 shown in FIG. 1, performs image enhancement processing on the luminance data S50A, and obtains output luminance data S5 obtained as a result.
1A is sent to the multipliers 84A and 84B and the camera signal processing circuit 64. The dividers 83A and 83B respectively convert the color difference data S50B and S50C into the luminance data S50.
By dividing by A, the color difference data S50B and S5
Normalized data S52A and S52B are generated by normalizing 0C with the luminance data S50A, respectively, and these are sent to delay circuits 85A and 85B, respectively.

The delay circuits 85A and 85B delay the normalized data S52A and S52B by a predetermined amount, respectively, and send them as delayed normalized data S53A and S53B to the multipliers 84A and 84B. Multipliers 84A and 8
4B is obtained by multiplying the delay normalized data S53A and S53B by the luminance data S51A, respectively.
The color difference data S54A and S54B are generated and sent to the camera signal processing circuit 64.

Further, in the above-described embodiment, the case where the present invention is applied to the video cameras 1, 50 and 60 has been described. However, the present invention is not limited to this, and for example, an electronic still camera, a printer, a display, and a computer. The present invention can be widely applied to various other image processing apparatuses. In this case, when correcting the image contrast, the computer can obtain a high-quality contrast-corrected image while maintaining the dynamic range, and when synthesizing images obtained under different lighting conditions, the respective contrasts can be adjusted. Only the difference between the components can be corrected, and a natural synthesized image can be generated.

[0073]

As described above, according to the present invention, the portion other than the edge is enhanced by amplifying the portion other than the edge of the input image data while preserving the edge where the pixel value changes sharply. Thus, the contrast and sharpness of the entire image can be further improved as compared with the related art.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a first embodiment of a video camera according to the present invention.

FIG. 2 is a block diagram illustrating a configuration of a nonlinear smoother.

FIG. 3 is a schematic diagram for explaining the operation of an ε filter.

FIG. 4 is a block diagram illustrating a configuration of an ε filter.

FIG. 5 is a block diagram showing a configuration of a register string.

FIG. 6 is a block diagram illustrating a configuration of a selector.

FIG. 7 is a block diagram illustrating a configuration of a video camera according to a second embodiment.

FIG. 8 is a schematic diagram for explaining input / output characteristics of a noise removal filter.

FIG. 9 is a schematic diagram for explaining a lookup table.

FIG. 10 is a block diagram illustrating a configuration of a video camera according to a third embodiment.

FIG. 11 is a block diagram illustrating a configuration of a video camera according to another embodiment.

FIG. 12 is a block diagram illustrating a configuration of a video camera according to another embodiment.

FIG. 13 is a diagram showing an example of an ε filter using a register 22.

FIG. 14 is a diagram showing a relationship between the number of registers 22 and a frequency response.

FIG. 15 is a diagram illustrating coefficients of a filter corresponding to the frequency response in FIG. 14;

[Explanation of symbols]

60: Video camera, 2, 61: CCD, 3, 5
1, 72 image processing circuit, 5 non-linear smoother, 1
2 ε filter, 20 register row, 21 selector, 22 register, 23 amplifier, 40, 41
... Adders, 42, 84 ... Multipliers.

Continued on the front page (72) Inventor Kazuhiko Ueda 6-35 Kita-Shinagawa, Shinagawa-ku, Tokyo F-term within Sony Corporation (reference) 5B057 BA02 CA01 CA02 CA08 CA12 CA16 CB01 CB02 CB08 CB12 CB16 CE03 CE05 CE06 CE11 CH08 5C021 PA34 PA42 PA52 PA57 PA62 RB03 XA35 XB06 YA01 5C022 AB51 AC69 5C077 LL19 MM03 MP08 PP03 PP32 PP48 PQ12 PQ23 TT09

Claims (10)

[Claims] 1. An image processing apparatus comprising: image processing means for amplifying a portion other than an edge of an input image data with a sharp change in pixel value while preserving an edge of the input image data. A smoothing unit that smoothes the input image data to generate smoothed image data while preserving the edge of the data, subtracts the smoothed image data from the input image data, and amplifies the subtraction result. Image enhancement means for adding smoothed image data to generate output image data, wherein the smoothing means performs nonlinear conversion on each of the input image data to generate nonlinear image data; and Non-linear filtering means for applying a non-linear digital filter to the non-linear image data to generate non-linear smoothed image data; The image processing apparatus characterized by comprising a non-linear inverse conversion means for generating a smoothed image data by performing an inverse transform of the nonlinear transformation means with respect to the non-linear smoother image data. 2. The image processing apparatus according to claim 1, wherein said nonlinear filtering means adaptively changes a degree of smoothing based on a difference value between a central pixel and its neighboring pixels. 3. The non-linear conversion means converts the value of the input image data so that the magnitude of the difference value does not depend on the intensity of illumination light when the image data is captured. The image processing device according to claim 1. 4. The image processing apparatus according to claim 1, wherein said non-linear filtering means comprises a plurality of non-linear filters of different sizes connected in series. 5. The image processing apparatus according to claim 1, wherein said non-linear filtering means comprises a plurality of non-linear filters of different sizes connected in series, and a smaller filter is located at a later stage. 6. An image processing method for performing image processing for amplifying a portion other than the edge of an input image data with a sharp change in pixel value while preserving an edge of the input image data, wherein the edge of the input image data is preserved. Generates smoothed image data by smoothing the input image data, subtracts the smoothed image data from the input image data, amplifies the result of the subtraction, and adds the smoothed image data to generate output image data The steps of
Further, noise components are removed from the input image data to generate noise-removed image data, the smoothed image data is subtracted from the noise-removed image data, and the result of the subtraction is amplified. Summing to generate output image data,
Further, a nonlinear transformation is performed on each of the input image data to generate nonlinear image data, a nonlinear digital filter is performed on the nonlinear image data to generate nonlinear smoothed image data, and a nonlinear smoothed image data is generated. Performing an inverse transformation of the non-linear transformation means to generate smoothed image data. 7. The image processing according to claim 6, wherein the degree of smoothing is adaptively changed based on a difference value between the central pixel and its neighboring pixels to generate the nonlinear smoothed image data. Method. 8. The apparatus according to claim 6, wherein the value of the input image data is converted so that the magnitude of the difference value does not depend on the intensity of illumination light when the image data is captured. Image processing method. 9. The image processing method according to claim 6, wherein a plurality of nonlinear filters having different sizes are serially applied to the nonlinear image data to generate the nonlinear smoothed image data. 10. The non-linear smoothed image data is generated by serially applying a plurality of non-linear filters having different sizes to the non-linear image data in descending order of the size. The image processing method according to 1.