JP2006157453A - Image processing apparatus, image processing method, and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology of obtaining a dynamic range compression image without a sense of incongruity in comparison with a conventional dynamic range compression system in compressing a dynamic range of an image. <P>SOLUTION: A compression ratio processing section 31 calculates a compression ratio gy for compressing the dynamic range of input image signals Rin, Gin, Bin and a saturation correction section 34 applies saturation correction processing for changing the saturation in response to the compression ratio gy to image signals Rdc, Gdc, Bdc whose dynamic range is compressed. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing device, an image processing method, and an imaging device, for example, processing and recording of an imaging result in a video camera, an electronic still camera, etc., image display on a liquid crystal display device, image processing by a personal computer, image synthesis, Furthermore, the present invention is applied to image transmission using these.

  2. Description of the Related Art Conventionally, various image processing circuits such as an imaging apparatus execute various processes such as recording and reproduction by compressing a dynamic range of an image.

  As processing for compressing such a dynamic range, there are a method of correcting the gradation of the entire image and a method of correcting the gradation only for the low frequency components of the image. In the former, gamma correction, knee correction, Furthermore, the dynamic range is compressed by correcting the gradation by so-called histogram equivalence or the like. On the other hand, in the latter, the dynamic range is compressed by gamma correction, knee correction, or the like.

  An image that needs to be compressed in the dynamic range is often a case where two or more illuminations are applied to the subject, and there is an extreme intensity difference between the illuminations. Even in such a case, as a method of effectively compressing the dynamic range of the image, the edge portion of the image is preserved, the portion other than the edge is smoothed, and the compression ratio of the dynamic range is set according to the smoothed level. There is a method of deciding and compressing the dynamic range of the entire image according to the ratio (for example, see Patent Document 1).

  In this method, since a smoothed image is used as a control signal, a fine amplitude component of the image is kept uncompressed, and a dynamic range compressed image with good visibility can be obtained.

JP 2001-275015 A

  However, in the conventional method disclosed in Patent Document 1, although the extreme luminance difference of the image is effectively compressed, when two or more illuminations are shining on the subject, it is usually white between those illuminations. Since the balance is also different, the image with the dynamic range compressed is also the image with the white balance of the high-brightness part shifted.

  Although this is a problem that occurs in the conventional dynamic range compression method such as knee compression, the conventional compression method compresses all the fine amplitude components and saturation, etc., at the same time as the luminance. The compression effect was low, and this white balance shift was rare. However, when effective luminance compression is performed by the conventional method as described above, this white balance shift tends to be noticeable.

  In addition, in the conventional method disclosed in Patent Document 1, the saturation of the high-intensity part is difficult to be compressed as compared with other conventional dynamic range compression methods such as knee compression. Compared with the compressed image, the image may have an impression that the saturation of the high luminance part is too high.

Accordingly, an object of the present invention is to obtain a dynamic range compressed image that is not uncomfortable as compared with the conventional dynamic range compression method in view of the conventional problems as described above. The specific advantages obtained by the above will become more apparent from the description of the embodiments described below.

  An image processing apparatus according to the present invention includes a compression ratio calculation unit that calculates a compression ratio for compressing a dynamic range of an input image signal, and a dynamic ratio obtained by multiplying the input image signal by the compression ratio calculated by the compression ratio calculation unit. Compression processing means for compressing the range; and saturation correction processing for changing the saturation according to the compression ratio calculated by the compression ratio calculation means for the input image signal whose dynamic range has been compressed by the compression processing means. Degree correction means.

  In addition, the image processing method according to the present invention calculates a compression ratio for compressing the dynamic range of the input image signal, multiplies the input image signal by the calculated compression ratio to compress the dynamic range, and compresses the dynamic range. The input image signal is subjected to saturation correction processing for changing the saturation according to the compression ratio.

  Furthermore, an imaging apparatus according to the present invention includes an imaging unit, a compression ratio calculation unit that calculates a compression ratio for compressing a dynamic range for an image signal obtained by imaging a subject by the imaging unit, and the compression ratio. A compression processing unit that multiplies the input image signal by the compression ratio calculated by the calculation unit to compress the dynamic range, and a compression that is calculated by the compression ratio calculation unit to the input image signal that has been compressed by the compression processing unit. And saturation correction means for performing saturation correction processing for changing the saturation according to the ratio.

  In the present invention, when the dynamic range of an image is compressed, the amount of saturation in the high-brightness part is reduced, and a dynamic range compressed image that is not uncomfortable compared to the conventional dynamic range compression method can be obtained.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Needless to say, the present invention is not limited to the following examples, and can be arbitrarily changed without departing from the gist of the present invention.

  The present invention is applied to, for example, an imaging apparatus 100 configured as shown in FIG.

  The imaging apparatus 100 is supplied with an imaging unit 10 that images a subject with a solid-state imaging device such as a C-MOS image sensor or a CCD (Charge Coupled Device) image sensor, and an image signal obtained as an imaging output signal by the imaging unit 10. Correction processing unit 20, a dynamic range compression processing unit 30 to which an image signal subjected to correction processing such as a shading component is supplied by the correction processing unit 20, and a dynamic range is compressed by the dynamic range compression processing unit 30. A camera signal processing unit 40 to which an image signal is supplied, and an image signal subjected to camera signal processing such as gamma correction by the camera signal processing unit 40 is supplied to a recording system such as a VTR, a display system 50 or the like. .

  As shown in the block diagram of FIG. 2, the dynamic range compression processing unit 30 is supplied with the input image signals Rin, Gin, Bin from the correction processing unit 20, as shown in the block diagram of FIG. The white balance correction unit 32, the compression processing unit 33 to which the image signals Rwb, Gwb, and Bwb whose white balance has been corrected by the white balance correction unit 32 are supplied, and the dynamic range is compressed by the compression processing unit 33. The saturation correction unit 34 is supplied with image signals Rdc, Gdc, and Bdc.

  The compression ratio calculation processing unit 31 includes an edge preserving smoothing processing unit 311 and a compression ratio calculation unit 312.

  The edge preserving smoothing processing unit 311 of the compression ratio calculation processing unit 31 merges the input image signals Rin, Gin, Bin supplied from the correction processing unit 20 into one channel, for example, the luminance signal Y, and then the horizontal edge. Pass the preservation smoothing filter, rearrange the pixels in the horizontal direction in the vertical direction, pass the signals rearranged in the vertical direction through the edge preservation smoothing filter in the vertical direction, and again turn the pixels in the vertical direction horizontally By rearranging and passing the horizontal edge preserving smoothing filter again, smoothing is performed while preserving both horizontal and vertical edges.

  For example, as shown in FIG. 3, the edge preserving smoothing processing unit 311 includes a cascade-connected RGB composition unit 311A, horizontal direction smoothing unit 311B, frame memory 311C, vertical direction smoothing unit 311D, and frame memory 311E. The input image signals Rin, Gin, and Bin are nonlinearly smoothed in the horizontal and vertical directions while preserving the edges of the input image signals Rin, Gin, and Bin.

  In the edge preserving smoothing processing unit 311, the input image signals Rin, Gin, and Bin are mixed by the RGB synthesizing unit 311A to form a one-channel luminance signal Y, and the luminance signal Y is edged by a non-linear filter in the horizontal direction smoothing unit 311B. Is stored and stored in the frame memory 311C. The signal Sh stored in the frame memory 311C is read out from the frame memory 311C in the vertical direction, and smoothed in the vertical direction by the vertical direction smoothing unit 311D with the same non-linear filter as that in the horizontal direction with the edges being preserved. The signal Shv stored in the frame memory 311E is read out from the frame memory 311E in the horizontal direction and nonlinearly smoothed in the horizontal and vertical directions as the image signal S. The compression ratio calculation processing unit 31 is supplied.

  The compression ratio calculation unit 312 of the compression ratio calculation processing unit 31 compresses the signal S that has been edge-preserved and smoothed by the edge preserving / smoothing processing unit 311 through a look-up table and does not pass through the look-up table. And the ratio gy of the signal compressed through the look-up table is output.

  The compression ratio gy obtained by the compression ratio calculation processing unit 31 in this way is supplied to the white balance correction unit 32, the compression processing unit 33, and the saturation correction unit 34.

  The white balance correction unit 32 includes three-channel white balance adjustment gain generation units 321R, 321G, and 321B and white balance adjustment multipliers 322R, 322G, and 322B, and the compression ratio calculated by the compression ratio calculation processing unit 31 The white balance adjustment gains gr, gg, and gb are generated from the gy by the white balance adjustment gain generation units 321R, 321G, and 321B, and the generated white balance adjustment gains gr, gg, and gb are multiplied by the input image signals Rin, Gin, and Bin. To do. As a result, the white balance correction unit 32 changes the white balance of the input image signals Rin, Gin, Bin according to the compression ratio gy calculated by the compression ratio calculation processing unit 31.

  The compression processing unit 33 includes three-channel dynamic range compression multipliers 331R, 331G, and 331B. The three-channel dynamic range compression multipliers 331R, 331G, and 331B perform the above-described white balance correction unit 32. A process for compressing the dynamic range is performed by multiplying the input image signals Rwb, Gwb, and Bwb with the white balance corrected by the compression ratio gy calculated by the compression ratio calculation processing unit 31.

  At this time, the compression ratio gy with respect to the luminance is considered to be such that the higher the ratio of compression, that is, the higher the ratio of gy, the higher the ratio of high luminance illumination contributing to the subject. By correcting the white balance of the image according to gy, a more natural dynamic range compressed image can be obtained. That is, when the value of gy is 1.0 (that is, no compression), the white balance is adjusted to match the low-luminance illumination, and the white balance of the high-luminance illumination approaches as the gy value decreases. If controlled, the white balance of low-luminance illumination is applied to the low-luminance area, and the white balance of high-luminance illumination is applied to the high-luminance area.

That is,
gy: Dynamic range compression gain (0 ≦ gy ≦ 1)
rh: white balance gain in the high luminance portion of the R channel gh: white balance gain in the high luminance portion of the G channel bh: white balance gain in the high luminance portion of the B channel rl: white balance gain in the low luminance portion of the R channel gl: White balance gain of the low luminance part of the G channel bl: White balance gain of the low luminance part of the B channel gr: White balance gain of the R channel controlled according to gy gg: White of the G channel controlled according to gy Balance gain gb: Assuming that the B channel white balance gain is controlled according to gy, the gains gr, gg, and gb for adjusting the white balance of the R, G, and B channels are:
gr = (1−gy) × rh + gy × rl (1-1) Equation gg = (1-gy) × gh + gy × gl (1-2) Equation gb = (1−gy) × bh + gy × bl (1-3) Let it be an expression.

  The white balance adjustment gains rl, gl, and bl of the low luminance portion and the white balance adjustment gains rh, gh, and bh of the high luminance portion are threshold values for the R, G, and B channels as shown in FIG. This can be detected by providing the attached integrators 323R, 323G, and 323B, and is provided to the white balance adjustment gain generation units 321R, 321G, and 321B of the respective channels via the system controller 60. As the signal used for the determination with the threshold value, edge-preserving smoothed signal S or Yin = f (Rin, Gin, Bin) before edge preserving smoothing can be used. If it is below, the white balance of the low luminance part can be detected, and if it is above the threshold, the white balance of the high luminance part can be detected. The white balance adjustment gains rl, gl, and bl in the low luminance portion and the white balance adjustment gains rh, gh, and bh in the high luminance portion are set so that these integral values are not biased in each channel.

  The image signals Rin, Gin, Bin of each channel adjust the white balance according to this gain.

here,
Rwb: R channel image signal after white balance adjustment Gwb: G channel image signal after white balance adjustment Bwb: B channel image signal after white balance adjustment Rdc: R channel image signal after dynamic range compression Gdc: After dynamic range compression G channel image signal Bdc: When the B channel image signal after dynamic range compression is used, the image signals Rwb, Gwb, and Bwb of each channel after white balance adjustment are expressed by the following equations (2-1) to (2-3): Indicated by
Rwb = gr.times.Rin (2-1) Equation Gwb = gg.times.Gin (2-2) Equation Bwb = gb.times.Bin (2-3) Equation Dynamic range compression gain gy for the image signals Rwb, Gwb, Bwb of these channels. , Dynamic range compressed image signals Rdc, Gdc, and Bdc in which the white balance is adjusted as shown in the following equations (2-4) to (2-5) are obtained.

Rdc = gy × Rwb (2-4) Gdc = gy × Gwb (2-5) Bdc = gy × Bwb (2-6) In this way, the dynamic range compression processor 30 calculates the compression ratio. The processing unit 31 calculates a compression ratio gy for compressing the dynamic range for the input image signals Rin, Gin, Bin, and the white balance correction unit 32 calculates the input image signals Rin, Gin, Bin according to the compression ratio gy. The white balance is corrected, and the input image signals Rwb, Gwb, and Bwb corrected for white balance are multiplied by the compression ratio gy by the compression processing unit 33 to compress the dynamic range.

  Further, the saturation correction unit 34 performs a saturation correction process for changing the saturation according to the compression ratio gy to the image signals Rdc, Gdc, and Bdc whose dynamic range has been compressed by the compression processing unit 33.

  By adjusting the saturation of an image using gy in this way, it is possible to obtain a dynamic range compressed image with less saturation and less discomfort than conventional dynamic range compression methods such as knee compression. .

That is,
sh: Saturation setting of high luminance part gs: Gain for saturation adjustment Ydc: Luminance signal generated from Rdc, Gdc, Bdc after dynamic range compression processing Rout: After white balance adjustment, dynamic range compression, saturation adjustment R channel video Gout: G channel video after white balance adjustment, dynamic range compression, saturation adjustment Bout: B channel video after white balance adjustment, dynamic range compression, saturation adjustment
gs = (1.0−gy) × sh + gy × 1.0 (3-1) Equation Rout = (Rdc−Ydc) × gs + Ydc (3-2) Equation Gout = (Gdc−Ydc) × gs + Ydc (3-3) By using the formula Bout = (Bdc−Ydc) × gs + Ydc (3-4), the saturation is also compressed in accordance with the compression ratio of the dynamic range, and the dynamic range is less uncomfortable than the conventional dynamic range compression method. A compressed image is obtained.

  As described above, in the dynamic range compression processing unit 30, the compression ratio calculation processing unit 31 calculates the compression ratio gy for compressing the dynamic range for the input image signals Rin, Gin, Bin, and the white balance correction unit 32. The white balance of the input image signals Rin, Gin, Bin is corrected according to the compression ratio gy, and the compression processing unit 33 multiplies the input image signals Rwb, Gwb, Bwb with the corrected white balance by the compression processing unit 33. To compress the dynamic range. Then, the saturation correction unit 34 performs saturation correction processing for changing the saturation according to the compression ratio gy to the image signals Rdc, Gdc, and Bdc compressed in the dynamic range.

  In the imaging apparatus 100, when the dynamic range compression processing unit 30 compresses the dynamic range of the image, by compressing the luminance of the high luminance part and simultaneously correcting the white balance of the high luminance part, It is possible to obtain a dynamic range compressed image that is natural and has no sense of incongruity. Furthermore, by adjusting the saturation accordingly, a more natural and uncomfortable dynamic range compressed image can be obtained.

It is a block diagram which shows the structure of the imaging device to which this invention is applied. It is a block diagram which shows the structure of the dynamic range compression process part in the said imaging device. It is a block diagram which shows the structural example of the edge preservation | save smoothing process part which comprises the said dynamic range compression process part. It is a block diagram which shows the structure of the said dynamic range compression process part.

Explanation of symbols

  10 imaging unit, 20 correction processing unit, 30 dynamic range compression processing unit, 31 compression ratio calculation processing unit, 32 white balance correction unit, 33 compression processing unit, 34 saturation correction unit, 40 camera signal processing unit, 50 display system, 60 system controller, 100 imaging device, 311 edge preserving smoothing processing unit, 311A RGB composition unit, 311B horizontal direction smoothing unit, 311C frame memory, 311D vertical direction smoothing unit, 311E frame memory, 312 compression ratio calculation unit, 321R , 321G, 321B White balance adjustment gain generator, 322R, 322G, 322B White balance adjustment multiplier, 323R, 323G, 323B Threshold integrator, 331R, 331G, 331B Dynamic range compression multiplier

Claims (3)

Compression ratio calculation means for calculating a compression ratio for compressing the dynamic range of the input image signal;
Compression processing means for compressing the dynamic range by multiplying the input image signal by the compression ratio calculated by the compression ratio calculation means;
Saturation correction means for performing saturation correction processing for changing the saturation according to the compression ratio calculated by the compression ratio calculation means on the input image signal whose dynamic range has been compressed by the compression processing means. An image processing apparatus. Calculate the compression ratio to compress the dynamic range for the input image signal,
The dynamic range is compressed by multiplying the input image signal by the calculated compression ratio,
An image processing method, comprising: applying saturation correction processing for changing saturation according to the compression ratio to an input image signal having a compressed dynamic range. Imaging means;
Compression ratio calculating means for calculating a compression ratio for compressing the dynamic range of the image signal obtained by imaging the subject by the imaging means;
Compression processing means for compressing the dynamic range by multiplying the input image signal by the compression ratio calculated by the compression ratio calculation means;
Saturation correction means for performing saturation correction processing for changing saturation according to the compression ratio calculated by the compression ratio calculation means on the input image signal whose dynamic range is compressed by the compression processing means. An imaging device.

Images