JP2013138301A - Image processor, image processing method and imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus including an image processor which can speedily switch an exposure condition when a plurality of images different in the exposure conditions are imaged and can generate a composite image of high image quality.SOLUTION: An imaging apparatus 1 includes: a histogram measuring section 23 which calculates a luminance histogram; an image composition section 31 for combining a high exposure image signal and a low exposure image signal to generate a composite image signal; a gradation compression section 35 for compressing the number of gradations of the composite image signal; and a parameter calculation section 33 for calculating a control parameter Ywhich characterizes brightness of the composite image. An imaging control section 24 refers to an exposure condition table 25T, selects an exposure condition corresponding to the luminance histogram, and sets the selected exposure condition in the imaging section 10. The gradation compression section 35 changes a gradation conversion characteristic so that a change of the brightness of the composite image due to switching of the exposure condition of the imaging section 10 is compensated on the basis of the control parameter Y.

Description

  The present invention relates to an image processing technique, and more particularly to a technique for processing a digital image picked up by a digital camera.

  In general, a digital camera such as a digital still camera or a video camera is equipped with a solid-state imaging device such as a charge-coupled device (CCD) image sensor or a complementary metal-oxide semiconductor (CMOS) image sensor. Since the dynamic range of a solid-state image sensor has physical limitations, if the difference in brightness (illuminance difference) in the subject exceeds the dynamic range capability of the solid-state image sensor, the dynamic range of the captured digital image There is a problem that a phenomenon in which pixel values in a luminance region are saturated (so-called overexposure state) and a phenomenon in which image information in the low-intensity region is lost (so-called blackout phenomenon) occur. Therefore, there exists an image processing technique for generating an image having a wide dynamic range (a width between the highest pixel value and the lowest pixel value) by combining a plurality of images captured under different exposure conditions. This type of technology is disclosed in, for example, Japanese Patent Application Laid-Open No. 2009-71408 (Patent Document 1).

  The image processing apparatus disclosed in Patent Document 1 receives a high exposure value image captured at a high exposure value and a low exposure value image captured at a low exposure value, and inputs the high exposure value image and the low exposure value. The brightness of each image is calculated, and the process of enlarging the gradation of the high exposure value image according to the brightness value is executed. Further, the image processing apparatus adds the luminance of the high exposure value image whose gradation is enlarged and the luminance of the low exposure value image, and based on the addition result, calculates the input image (high exposure value image and low exposure value image). Determine the synthesis ratio. Then, the image processing apparatus combines the high exposure value image and the low exposure value image using the combination ratio, and then executes a process of returning the gradation of the combined image obtained as a result to the original gradation. Thereby, an image with a wide dynamic range can be generated.

JP 2009-71408 A (FIG. 1, paragraphs 0012 to 0016, etc.)

  However, Patent Document 1 does not specifically describe a method of adjusting exposure conditions (exposure time, lens aperture value, etc.) when capturing a high exposure value image and a low exposure value image. The exposure conditions (such as the exposure time and lens aperture value) that are suitable for capturing a high exposure value image and the exposure conditions that are suitable for capturing a low exposure value image are different from each other. There is a problem that it takes a long time to switch from an exposure condition suitable for one imaging to an exposure condition suitable for the other imaging, and the image quality of the composite image is degraded.

  For example, in exposure control using the output of a solid-state image sensor, an index value of a luminance value (for example, an average value of several luminance values) in a main reference region of a captured image is calculated, and this index value is a target value. The exposure condition such as the exposure time and the lens aperture value is adjusted so as to be close to, and the image is taken again after the adjustment. When a high exposure value image and a low exposure value image are continuously captured by the video camera, it is desirable to perform exposure control until the index value substantially matches the target value at the time of capturing each image. However, it may take several to tens of frames of adjustment time to make the exposure condition follow the subject condition (for example, subject illuminance). In particular, the exposure condition is immediately followed for sudden changes in the subject condition. There is a problem that it is difficult to do.

  In view of the above, an object of the present invention is to generate a composite image with good image quality by adjusting exposure conditions in a short time when capturing a plurality of images having different exposure conditions for image composition. An image processing apparatus, an imaging apparatus, and an image processing method that can also be provided.

  An image processing apparatus according to a first aspect of the present invention is an image processing apparatus that processes an image signal representing an image captured by an imaging unit, the exposure control histogram measuring unit measuring a luminance histogram of the image signal; , An exposure condition table in which correspondences between a plurality of exposure condition candidates for the imaging unit and luminance histograms are stored in advance, and a high exposure image that is captured at a first exposure amount by the imaging unit. An exposure image signal and a low exposure image signal representing a low exposure image captured at a second exposure amount lower than the first exposure amount by the imaging unit are combined to produce the high exposure image signal and the low exposure image. An image composition unit that generates a composite image signal having a higher number of gradations than the number of gradations of the image signal, and a gradation conversion characteristic that defines a correspondence relationship between the input gradation level and the output gradation level; Based on conversion characteristics A gradation compression unit that compresses the number of gradations of the input composite image signal, a parameter calculation unit that calculates a control parameter that characterizes the brightness of the composite image represented by the composite image signal, and the imaging unit An imaging control unit that controls the exposure conditions, and the imaging control unit refers to the exposure condition table, and the brightness measured by the exposure control histogram measurement unit from among the plurality of exposure condition candidates The exposure condition corresponding to the histogram is selected, the selected exposure condition is set in the imaging unit, and the tone compression unit is configured to switch the exposure condition of the imaging unit based on the control parameter. The gradation conversion characteristic is changed so as to compensate for a change in image brightness.

  An imaging apparatus according to a second aspect of the present invention includes an imaging unit and the image processing apparatus according to the first aspect.

  An image processing method according to a third aspect of the present invention includes an imaging unit, an exposure control histogram measuring unit that measures a luminance histogram of an image signal representing an image captured by the imaging unit, and a plurality of exposures for the imaging unit. An exposure condition table in which correspondences between candidate conditions and a plurality of frequency distributions of a luminance histogram are stored; a high exposure image signal representing a high exposure image captured at a first exposure amount by the imaging unit; A combination of a low-exposure image signal representing a low-exposure image captured by the imaging unit with a second exposure amount that is lower than the first exposure amount is used to combine the high-exposure image signal and the low-exposure image signal. An image composition unit that generates a composite image signal having a higher number of gradations than the key number, and a gradation conversion characteristic that defines a correspondence relationship between the input gradation level and the output gradation level, based on the gradation conversion characteristic Before the input A gradation compression unit that compresses the number of gradations of the composite image signal, a parameter calculation unit that calculates a control parameter that characterizes the brightness of the composite image represented by the composite image signal, and an exposure condition of the imaging unit is controlled. An imaging control unit, wherein the imaging control unit refers to the exposure condition table, and the exposure condition corresponding to the brightness histogram measured by the exposure control histogram measurement unit from among the plurality of exposure condition candidates And the selected exposure condition is set in the imaging unit, and the gradation compression unit changes the brightness of the composite image by switching the exposure condition of the imaging unit based on the control parameter. The gradation conversion characteristic is changed so as to compensate for this.

  According to the aspect of the present invention, one exposure condition can be selected from a plurality of exposure condition candidates in a short time by referring to the exposure condition table, and the selected exposure condition can be set in the imaging unit. . For this reason, switching of exposure conditions can be performed quickly. As a result, it is possible to follow the exposure conditions of the imaging unit at a high speed with respect to changes in subject conditions. Further, based on the control parameter characterizing the brightness of the composite image, the tone conversion characteristics are changed so as to compensate for the change in the brightness of the composite image due to the switching of the exposure conditions. Therefore, a wide dynamic range image with good image quality can be generated.

1 is a functional block diagram schematically showing a configuration of an imaging apparatus according to a first embodiment of the present invention. 2 is a diagram schematically illustrating a configuration of an imaging unit according to Embodiment 1. FIG. It is a figure which shows roughly the low exposure image and high exposure image which were imaged at the moving image imaging mode. It is a figure which shows an example of the brightness | luminance histogram of a low exposure image. It is a figure which shows an example of the brightness | luminance histogram of a high exposure image. It is a figure which shows the example of the basic pattern used as the candidate of exposure conditions. FIG. 7 is a diagram illustrating an example of a reference table that defines a correspondence relationship between a basic pattern of exposure conditions in FIG. 6 and a luminance histogram. 3 is a functional block diagram schematically showing a configuration example of an image composition unit in the first embodiment. FIG. 6 is a graph illustrating an example of a correspondence relationship between an input value to a composition ratio calculation LUT according to the first embodiment and a composition ratio. It is a graph which illustrates the brightness | luminance histogram of a synthesized image. (A), (B) is a graph which shows the example of the gradation conversion characteristic of the gradation compression part of Embodiment 1. FIG. (A), (B) is a graph which shows the other example of the gradation conversion characteristic of the gradation compression part of Embodiment 1. FIG. It is a functional block diagram which shows roughly the structure of the imaging device of Embodiment 2 which concerns on this invention. It is a graph which shows the example of the brightness | luminance histogram of a low exposure image. It is a graph which shows the example of the brightness | luminance histogram of a high exposure image. (A), (B) is a graph which shows the example of the gradation conversion characteristic of the gradation compression part of Embodiment 2. FIG. (A), (B) is a graph which shows the other example of the gradation conversion characteristic of the gradation compression part of Embodiment 2. FIG. It is a functional block diagram which shows roughly the structure of the imaging device of Embodiment 3 which concerns on this invention. It is a figure which shows the frame image divided | segmented into the several area image. 10 is a flowchart illustrating a procedure of control parameter restriction processing according to Embodiment 3. (A), (B) is a graph which shows the example of the gradation conversion characteristic of the gradation compression part of Embodiment 4. FIG.

  Hereinafter, various embodiments according to the present invention will be described with reference to the drawings.

Embodiment 1 FIG.
FIG. 1 is a functional block diagram schematically showing the configuration of the imaging apparatus 1 according to the first embodiment of the present invention. The imaging device 1 includes an imaging unit 10 and an image processing unit (image processing device) 20.

  FIG. 2 is a diagram schematically illustrating the configuration of the imaging unit 10. As shown in FIG. 2, the imaging unit 10 includes an imaging optical system 12, a solid-state imaging device 13, a front end unit 14, and a drive circuit 15. The imaging optical system 11 includes a front lens 121, a diaphragm mechanism 122, and a rear lens 123. The solid-state imaging device 13 of the present embodiment is configured by a single-plate CCD image sensor having a color filter array such as a Bayer array, but is not limited to this. Instead of the CCD image sensor, another solid-state imaging device such as a CMOS image sensor may be used. The drive circuit 15 generates a drive signal that drives the solid-state imaging device 13 in accordance with the control signal ECs.

  The imaging optical system 12 forms an optical image by forming the incident light 11 on the imaging surface of the solid-state imaging device 13. The solid-state imaging device 13 photoelectrically converts this optical image to generate an imaging signal, and supplies the imaging signal to the front end unit 14. The front end unit 14 generates an analog signal by executing correlated double sampling (CDS) processing and programmable gain amplification (PGA) on the imaging signal. Here, the CDS process is a process of removing unnecessary components such as noise from the imaging signal output from the solid-state imaging device 13. The front end unit 14 further A / D converts the analog signal to generate a digital image signal Rd in RAW format.

  Referring to FIG. 1, the image processing unit 20 includes a signal processing unit 21, a frame buffer 22, a histogram measurement unit 23, an imaging control unit 24, and a data storage unit 25. These components 21 to 24 are connected to each other via a bus (signal transmission path) 28.

  The signal processing unit 21 performs color synchronization processing, noise reduction processing, contour correction processing, white balance adjustment processing, signal amplitude adjustment processing, color correction processing, color space conversion, and the like on the digital image signal Rd input from the imaging unit 10. To generate a color image signal composed of a luminance component and a color difference component. This color image signal is output to the bus 28. In the RAW format digital image, one pixel has only one color component corresponding to one color filter in the color filter array of the solid-state image sensor 13. The color synchronization process is a process of interpolating a color component that is insufficient for each pixel to form a color image. For example, when the color filter array is a primary color Bayer array, each pixel of the RAW format digital image has only one color component of red, green, and green. Are interpolated from the color components of the surrounding pixels.

  The bus 28 transfers the color image signal output from the signal processing unit 21 to the frame buffer 22. The frame buffer 22 has a function of temporarily storing (buffering) a plurality of frames of the color image signal transferred by the bus 28.

  The histogram measuring unit 23 has a function of measuring the luminance histogram of the luminance component of the color image signal read from the frame buffer 22 in units of frame images. The imaging control unit 24 supplies the control signal group EC to the imaging unit 10 by using the luminance histogram measured by the histogram measurement unit 23 and the exposure condition table 25T stored in advance in the data storage unit 25, thereby imaging the imaging unit. Ten exposure conditions can be controlled. As shown in FIG. 2, the control signal group EC includes a control signal ECa supplied to the aperture mechanism 122, a drive signal ECs supplied to the drive circuit 15, and a control signal ECg supplied to the front end unit 14. Consists of. The imaging control unit 24 uses the luminance histogram and the exposure condition table 25T, the aperture value indicating the aperture of the aperture mechanism 122, the exposure time (charge accumulation time) of the solid-state imaging device 13, and the front end unit 14 The gain (amplification factor) of the amplifier can be individually controlled.

  The imaging apparatus 1 according to the present embodiment has two types of operation modes, a still image imaging mode and a moving image imaging mode. In the still image capturing mode, the image capturing control unit 24 controls the operation of the image capturing unit 10 to continuously capture two types of color images with different exposure conditions in units of frame images. Here, the color image is not an image in the RAW format, but means an output image of the signal processing unit 21. Of these two types of color images, one image (hereinafter referred to as a high exposure image) has an exposure amount on the imaging surface of the solid-state imaging device 13 rather than the other image (hereinafter referred to as a low exposure image). There are many images. On the other hand, in the moving image imaging mode, the imaging control unit 24 controls the operation of the imaging unit 10 to alternately capture a high exposure image and a low exposure image in units of frame images. FIG. 3 is a diagram schematically showing low exposure images Fs (1), Fs (2),... And high exposure images Ft (1), Ft (2),.

  The histogram measurement unit 23 reads the data of the low exposure image and the high exposure image from the frame buffer 22 in the order of imaging, measures the luminance histogram of the low exposure image, outputs the measurement data Hsd to the imaging control unit 24, and performs high exposure. The brightness histogram of the image is measured and the measurement data Htd is output to the imaging control unit 24. FIG. 4 is a diagram illustrating an example of a luminance histogram of an 8-bit gradation (256 gradation) low-exposure image, and FIG. 5 is a diagram illustrating an example of a luminance histogram of an 8-bit gradation high-exposure image. . In the graphs of FIGS. 4 and 5, the horizontal axis indicates the gradation level in the range of 0 to 255, and the vertical axis indicates the number of pixels having the luminance value of each gradation level.

  When capturing a low-exposure image, the imaging control unit 24 sets the exposure condition of the imaging unit 10 using the luminance histogram of the most recently captured low-exposure image. On the other hand, when capturing a high-exposure image, the imaging control unit 24 sets the exposure condition of the imaging unit 10 using the luminance histogram of the most recently captured high-exposure image. For example, in the moving image capturing mode, the exposure condition for capturing the low-exposure image Fs (3) in FIG. 3 is performed based on the measurement data of the luminance histogram of the low-exposure image Fs (2) captured two frames before. Is called. Further, the exposure conditions for capturing the high-exposure image Ft (3) in FIG. 3 are performed based on the measurement data of the luminance histogram of the low-exposure image Ft (2) captured two frames before.

  The imaging control unit 24 switches the exposure condition of the imaging unit 10 for each frame image with reference to the exposure condition table 25T based on the measurement result by the histogram measurement unit 23. The exposure condition table 25T is a table in which a correspondence relationship between a plurality of exposure condition candidates and a luminance histogram is stored in advance. The exposure condition table 25T stores in advance values that determine two types of exposure conditions for low-exposure image capturing and high-exposure image capturing.

  FIG. 6 is a diagram showing an example of basic patterns P1, P2, P3, and P4 that are exposure condition candidates. In the example of FIG. 6, the exposure time (long exposure period) for capturing a high exposure image is set to 32 milliseconds (msec.) In all of the basic patterns P1 to P4. Further, as shown in FIG. 6, the exposure time for low-exposure image capturing (short exposure period) is 2 milliseconds for the basic pattern P1, 4 milliseconds for the basic pattern P2, 8 milliseconds for the basic pattern P3, The basic pattern P4 is 16 milliseconds.

FIG. 7 is a diagram showing an example of a reference table that defines the correspondence between the basic patterns P1 to P4 of the exposure conditions in FIG. 6 and the luminance histogram. In the reference table of FIG. 7, basic patterns P1, P2, P3, and P4 (transition source patterns) of exposure conditions selected at the time of the latest low-exposure image capturing, and basic pattern candidates P1, P2, and P3 to be currently selected. , P4 (transition destination pattern) is defined. The conditions for the combination are defined using four types of parameters Yo ₂₀₀ , Yu ₁₀₀ , Yu ₅₀ , and Yu ₂₅ that represent the characteristics of the luminance histogram of the low-exposure image captured most recently.

Here, Yo ₂₀₀ is a parameter indicating a ratio (unit: percent) of pixels having a pixel value (luminance value) of 200 or more to the total number of effective pixels, and Yu ₁₀₀ is a pixel value within a range of 0 to _100. This is a parameter indicating the ratio (unit: percent) of the pixels having (luminance value) to the total number of effective pixels, and Yu ₅₀ is the total number of effective pixels of pixels having a pixel value (luminance value) in the range of 0-50. Is a parameter indicating a ratio (unit: percent) to the total number of effective pixels of pixels having pixel values (luminance values) within a range of 0 to _25. . For example, the combination condition of the transition source pattern P1 and the transition destination pattern P2 is that Yu ₁₀₀ is 90% or more, and the combination condition of the transition source pattern P2 and the transition destination pattern P1 is 50 for Yo _200. % Or more.

The imaging control unit 24 calculates parameters Yo ₂₀₀ , Yu ₁₀₀ , Yu ₅₀ , Yu ₂₅ from the luminance histogram measurement data Hsd of the most recently captured low-exposure image, and these parameters Yo ₂₀₀ , Yu ₁₀₀ , Yu ₅₀ , Yu. A value of ₂₅ is used as an index value for selecting one exposure condition from the exposure condition candidates P1 to P4.

  In the example of FIGS. 6 and 7, the exposure time at the time of capturing a high-exposure image is set to a constant 32 milliseconds, but is not limited to this. In the case of high-exposure image capturing as well, in the same way as in the case of low-exposure image capturing, a reference table that prescribes a combination condition of a transition source pattern and a transition destination pattern of exposure conditions is prepared in advance, Thus, the exposure conditions for capturing a high-exposure image may be set.

  The value that determines the exposure condition is not limited to the exposure time, and the aperture value of the aperture mechanism 122 and the gain of the amplifier in the front end unit 14 can also be set as values that determine the exposure condition.

  Next, image composition processing and gradation compression processing will be described. As illustrated in FIG. 1, the image processing unit 20 includes an image composition unit 31, a parameter calculation unit 33, a histogram measurement unit 34, and a gradation compression unit 35.

  The image synthesizing unit 31 is connected to the bus 28 and synthesizes and combines the temporally continuous low-exposure image signal and high-exposure image signal that are read from the frame buffer 22 and transferred via the bus 28. A function of generating an image signal Fc; The composite image signal Fc has a higher number of gradations than the number of gradations of the low exposure image signal and the high exposure image signal. FIG. 8 is a functional block diagram schematically illustrating a configuration example of the image composition unit 31 according to the first embodiment. As illustrated in FIG. 8, the image composition unit 31 includes a coefficient multiplication unit 311, an addition unit 312, a composition ratio calculation LUT (lookup table) 313, composition ratio multiplication units 314 s and 314 t, and a composition processing unit 315.

As shown in FIG. 8, the coefficient multiplication unit 311 includes a multiplier 311 s that multiplies the luminance component Ys (k) of the low luminance image Fs (k) by a weight coefficient α in units of pixels, and a high luminance image Ft (k). And a multiplier 311t for multiplying the luminance component Yt (k) by a weight coefficient β in units of pixels. The weighting coefficients α and β are given by the imaging control unit 24. In the present embodiment, β = 1 and α = n (n> 1) are set. The value of n is, for example, the ratio (= t _β / t _α ) of the exposure time t _β when capturing the high exposure image Ft (k) to the exposure time t _α when capturing the low exposure image Fs (k). be able to.

  The adder 312 adds the output of the multiplier 311s and the output of the multiplier 311t in units of pixels. The composition ratio calculation LUT 313 has a function of determining composition ratios Ks and Kt to be assigned to the low-exposure image Fs (k) and the high-exposure image Ft (k) in accordance with the output signal of the adder 312. The combination ratio calculation LUT 313 includes a memory in which a plurality of values of the combination ratios Ks and Kt are stored in advance, and the output signal of the adder 312 is used as an address input, and a pair of stored values Ks corresponding to the address input. , Kt can be read from the memory and output.

  The composition ratio multiplying unit 314s multiplies the luminance component Ys (k) of the low-exposure image Fs (k) by the composition ratio Ks × α (= Ks × n) and composes the color difference components Cbs (k) and Crs (k). The ratio Ks is multiplied, and the multiplication results are output to the synthesis processing unit 315. On the other hand, the composition ratio multiplying unit 314t multiplies the luminance component Yt (k) of the high-exposure image Ft (k) by the composition ratio Kt × β (= Kt × 1) and color difference components Cbt (k), Crt (k). Are multiplied by the synthesis ratio Kt, and the multiplication results are output to the synthesis processing unit 315. The synthesis processing unit 315 adds the output of the synthesis ratio multiplication unit 314s and the output of the synthesis ratio multiplication unit 314t to generate a synthesized image signal Fc. That is, the composition processing unit 315 adds the output luminance component Fs (k) × Ks × α of the composition ratio multiplication unit 314s and the output luminance component Ft (k) × Kt × β of the composition ratio multiplication unit 314t to add a composite image. A luminance component of the signal Fc is generated. At the same time, the synthesis processing unit 315 adds the output color difference component Cbs (k) × Ks of the synthesis ratio multiplication unit 314s and the output luminance component Cbt (k) × Kt of the synthesis ratio multiplication unit 314t, and also combines the synthesis ratio multiplication unit 314s. Output color difference component Cbs (k) × Ks and output luminance component Crt (k) × Kt of the combination ratio multiplication unit 314t are added to generate a color difference component of the combined image signal Fc.

  FIG. 9 is a graph showing an example of the correspondence relationship between the input value to the composition ratio calculation LUT 313 and the composition ratios Ks and Kt. In the graph of FIG. 9, the horizontal axis indicates the gradation level of the input signal (luminance component), the right vertical axis indicates the value of the composite ratio Kt, and the left vertical axis indicates the value of the composite ratio Ks. ing. In the graph of FIG. 9, when the luminance value is larger than the threshold value (= 255 × 2), the value of the composition ratio Ks with respect to the low-exposure image Fs (k) is large, and when the luminance value is smaller than the threshold value, the value is high. The value of the composition ratio Kt with respect to the exposure image Ft (k) increases. In the graph of FIG. 9, the value of the composition ratio Ks is switched from 0 to 1 (or from 1 to 0) and the value of the composition ratio Kt is 1 to 0 near the luminance value becomes the threshold value (= 255 × 2). (Or from 0 to 1). Therefore, the synthesis ratios Ks and Kt are complementary to each other with respect to the threshold value. In the example of FIG. 9, the magnification α for the luminance component Ys (k) of the low-exposure image Fs (k) in the coefficient multiplier 311 is set to n times, and the luminance component Yt (k) of the high-exposure image Ft (k). Therefore, the addition result of the luminance components α × Ys (k) and β × Yt (k) has a gradation level in the range of 0 to 255 × (n + 1). Note that when the brightness of the high-exposure image Ft (k) reaches a saturation level (= 255), the brightness value of the high-exposure image is considered to be about 255 / n, so the threshold value is 255 ×. 2.

  Next, the histogram measurement unit 34 measures the luminance histogram of the composite image signal Fc input from the image composition unit 31 in units of frame images, and supplies the measurement data CHd to the gradation compression unit 35. FIG. 10 is a diagram illustrating an example of a luminance histogram of the composite image signal Fc.

  On the other hand, the parameter calculation unit 33 calculates the average luminance value Yavg of the combined image based on the combined image signal Fc input from the image combining unit 31, and supplies this average luminance value Yavg to the gradation compression unit 35. The average luminance value Yavg is a control parameter that characterizes the brightness of the composite image. The average luminance value Yavg can be calculated, for example, by adding the luminance components of all effective pixels for one frame of the composite image and dividing the addition result by the total number of effective pixels.

  The gradation compression unit 35 has gradation conversion characteristics (gradation compression characteristics) for converting the gradation level of the luminance component of the input composite image signal Fc, and the composite image signal Fc according to the gradation conversion characteristics. The gradation level of the luminance component is converted. This gradation conversion characteristic defines the correspondence between the input gradation level and the output gradation level. In this gradation conversion characteristic, since the output gradation level range is narrower than the input gradation level range, the gradation compression unit 35 compresses (normalizes) the number of gradations of the composite image. Can do.

  Also, the gradation compression unit 35 dynamically adjusts the gradation conversion characteristics so as to compensate for changes in the brightness of the composite image due to switching of the exposure conditions of the imaging unit 10 based on the average luminance value Yavg that is a control parameter. It has a function to change. Specifically, the gradation compression unit 35 can increase the output gradation level of the gradation conversion characteristics when the brightness of the composite image is reduced in accordance with the switching of the exposure condition of the imaging unit 10, Conversely, when the brightness of the composite image increases in accordance with the switching of the exposure conditions of the imaging unit 10, the output gradation level of the gradation conversion characteristics can be reduced. Thereby, it is possible to compensate for a change in brightness of the composite image due to switching of the exposure conditions.

  FIGS. 11A and 11B are graphs illustrating the gradation conversion characteristics of the gradation compression unit 35. FIG. FIG. 11A is a graph showing a conversion curve when the value of the control parameter Yavg is relatively high, and FIG. 11B is a graph showing the conversion curve when the value of the control parameter Yavg is relatively low. is there. The conversion curve mainly includes a straight line connecting the point (0, 0) and the point (Yavg, Bt) and a straight line connecting the point (Yavg, Bt) and the point (Ymax, 255). Since the output gradation level value Bt with respect to the value of Yavg is always fixed at a constant value (for example, 100), the conversion curve changes so that the output gradation level generally decreases as the value of the control parameter Yavg increases. However, as the value of the control parameter Yavg decreases, the conversion curve changes so that the output gradation level increases as a whole.

Further, the gradation compression unit 35 detects an effective dynamic range (hereinafter referred to as an effective dynamic range) of the composite image based on the measurement data CHd of the luminance histogram of the composite image, and corresponds to this effective dynamic range. It has a function of dynamically changing the gradation conversion characteristics so that the output gradation level range is expanded. Specifically, the frequency value I (x) (x is the input gradation level) of the luminance histogram of the composite image is integrated from the minimum gradation level (= 0) to the maximum gradation level (= 255 × n). when calculating the cumulative frequency value by, be defined relative to the total effective pixel count N _p is a predetermined percentage of the cumulative frequency value (e.g., 99%) satisfy integrating period equal to or less than [0, Lu] it can. The gradation compression unit 35 can calculate the maximum value Ymax of the upper end value Lu of the integration interval [0, Lu] as the upper limit value of the effective dynamic range. Therefore, when the predetermined ratio is 99%, the following expression (1) is established.

  The upper limit value Ymax of the effective dynamic range of the composite image is, for example, a value as schematically shown in FIG. As shown in FIGS. 11A and 11B, the gradation compression unit 35 sets the input gradation level range of 0 to Ymax as an effective dynamic range, and the input gradation level equal to or higher than the upper limit value Ymax is all 255. Convert the value to the output tone level. Thereby, the output gradation level range corresponding to the effective dynamic range of the composite image can be expanded. Therefore, the dynamic range of the output image can be widened.

The gradation compression unit 35 can also detect the lower limit value Ymin of the effective dynamic range of the composite image. Specifically, the frequency integrated value is calculated by integrating the frequency value I (x) of the luminance histogram of the composite image in the direction from the maximum gradation level (= 255 × n) to the minimum gradation level (= 0). time, it is possible to determine the percentage of total effective pixel count n _p is a predetermined percentage of the cumulative frequency value (e.g., 99%) satisfy integrating period equal to or less than [Lw, 255 × n]. The gradation compression unit 35 can calculate the minimum value Ymin of the lower end value Lw of the integration interval [Lw, 255 × n] as the lower limit value of the effective dynamic range. Therefore, when the predetermined ratio is 99%, the following expression (2) is established.

  The lower limit value Ymin of the effective dynamic range of the composite image is, for example, a value schematically shown in FIG. As shown in FIGS. 12A and 12B, the gradation compression unit 35 sets the input gradation level range of Ymin to Ymax as an effective dynamic range, and the input gradation level equal to or greater than the upper limit value Ymax is 255. The output gradation level range corresponding to the effective dynamic range of the composite image is converted by converting the input gradation level below the lower limit value Ymin to the output gradation level of all 0 values. Can be enlarged. Therefore, it is possible to further widen the dynamic range of the output image.

  As described above, in the image processing unit 20 of the first embodiment, the imaging control unit 24 selects one exposure condition from a plurality of exposure condition candidates in a short time by referring to the exposure condition table 25T. The selected exposure condition can be set in the imaging unit 10. For this reason, since the exposure condition can be quickly switched, the exposure condition of the imaging unit 10 can be followed at a high speed with respect to the change in the subject condition. Further, the gradation compression unit 35 dynamically changes the gradation conversion characteristics so as to compensate for the change in the brightness of the composite image due to the switching of the exposure condition, based on the control parameter Yavg that characterizes the brightness of the composite image. . For this reason, the gradation compression unit 35 can output a wide dynamic range image with good image quality. Therefore, it is possible to achieve both high-speed tracking of exposure conditions with respect to changes in subject conditions and smooth fluctuations in the brightness of the output image between frames.

  Furthermore, since the gradation compression unit 35 dynamically expands the output gradation level range corresponding to the effective dynamic range according to the effective dynamic range of the composite image, the dynamic range of the output image can be widened. .

Embodiment 2. FIG.
Next, a second embodiment according to the present invention will be described. FIG. 13 is a functional block diagram schematically illustrating the configuration of the imaging device 2 according to the second embodiment. The configuration of the imaging apparatus 2 according to the present embodiment is the same as the configuration of the imaging apparatus 1 according to the first embodiment except for a part of the configuration of the image processing unit 20B.

  As shown in FIG. 13, the image processing unit 20 </ b> B of the present embodiment has the same constituent elements as the constituent elements 21 to 25, 31, and 33 of the image processing unit 20 of the first embodiment, but the histogram measuring unit 34. Does not have. The gradation compression unit 35B executes gradation compression processing using the data Hsd and Htd measured by the histogram measurement unit 23. Thereby, the circuit scale can be reduced as compared with the configuration of the first embodiment.

  Similar to the gradation compression unit 35 of the first embodiment, the gradation compression unit 35 </ b> B changes the brightness of the composite image by switching the exposure condition of the imaging unit 10 based on the average luminance value Yavg that is a control parameter. It has a function of dynamically changing the gradation conversion characteristics so as to compensate.

  Also, the gradation compression unit 35B detects the effective dynamic range of the composite image based on the luminance histogram measurement data Hsd and Htd supplied from the histogram measurement unit 23, and the output gradation level range corresponding to the effective dynamic range. Has a function of dynamically changing the gradation conversion characteristics so that the

図１４は、低露光画像の輝度ヒストグラムの例を示すグラフである。低露光画像の実効ダイナミックレンジの上限値ＹＳｍａｘは、たとえば、図１４に示されるような値となる。 Specifically, the frequency value Is (x) (x is the input gradation level) of the luminance histogram of the low-exposure image constituting the composite image is changed from the minimum gradation level (= 0) to the maximum gradation level (= 255). when calculating the frequency integrated value by integrating in the direction, relative to the total effective pixel count N _p is a predetermined percentage of the cumulative frequency value (e.g., 99%) satisfy integrating period equal to or less than [0, LSU] Can be determined. The gradation compression unit 35 can calculate the maximum value YSmax of the upper end value Lsu of the integration interval [0, Lsu] as the upper limit value of the effective dynamic range of the low exposure image. Therefore, when the predetermined ratio is 99%, the following expression (3) is established.

FIG. 14 is a graph illustrating an example of a luminance histogram of a low exposure image. The upper limit value YSmax of the effective dynamic range of the low exposure image is, for example, a value as shown in FIG.

On the other hand, the frequency value It (x) (x is the input gradation level) of the luminance histogram of the high-exposure image constituting the composite image is changed from the minimum gradation level (= 0) to the maximum gradation level (= 255). when calculating the cumulative frequency value by accumulating, to define the percentage of total effective pixel count N _p is a predetermined percentage of the cumulative frequency value (e.g., 99%) satisfy integrating period equal to or less than [0, LTU] Can do. The gradation compression unit 35 can calculate the maximum value YLmax of the upper end value Ltu of the integration interval [0, Ltu] as the upper limit value of the effective dynamic range of the high exposure image. Therefore, when the predetermined ratio is 99%, the following expression (4) is established.

  FIG. 15 is a graph showing an example of a brightness histogram of a high exposure image. The upper limit value YLmax of the effective dynamic range of the high exposure image is, for example, a value as shown in FIG.

The gradation compression unit 35B of the present embodiment can calculate the weighted average value <Ymax> of the upper limit values YSmax and YLmax as the upper limit value of the effective dynamic range of the composite image. That is, the upper limit value <Ymax> of the effective dynamic range of the composite image can be calculated according to the following equation (5).
<Ymax> = (YSmax × α + YLmax × β) / 2 (5)

  As shown in FIGS. 16A and 16B, the gradation compression unit 35B sets the input gradation level range of 0 to <Ymax> as the effective dynamic range of the composite image, and inputs the upper limit value <Ymax> or more. All the gradation levels are converted to an output gradation level having a value of 255. Thereby, the output gradation level range corresponding to the effective dynamic range of the composite image can be expanded. Therefore, the dynamic range of the output image can be widened.

The gradation compression unit 35B can also detect the lower limit value <Ymin> of the effective dynamic range of the composite image. Specifically, when the frequency integrated value is calculated by integrating the frequency value Is (x) of the luminance histogram of the low exposure image from the maximum gradation level (= 255) to the minimum gradation level (= 0). , can be determined relative to the total effective pixel count _{N p} is a predetermined percentage of the cumulative frequency value (e.g., 99%) satisfy integrating period equal to or less than [Lsw, 510]. The gradation compression unit 35B can calculate the minimum value YSmin of the lower end value Lsw of the integration interval [Lsw, 510] as the lower limit value of the effective dynamic range of the low exposure image. Therefore, when the predetermined ratio is 99%, the following expression (6) is established.

  The lower limit value YSmin of the effective dynamic range of the low exposure image is, for example, a value as schematically shown in FIG.

On the other hand, when the frequency integrated value is calculated by integrating the frequency value It (x) of the luminance histogram of the high exposure image from the maximum gradation level (= 255) to the minimum gradation level (= 0), percentage of total effective pixel count _{N p} is a predetermined ratio of integrated values (e.g., 99%) can be determined satisfy integrating period equal to or less than [Ltw, 510]. The gradation compression unit 35B can calculate the minimum value YLmin of the lower end value Ltw of the integration section [Ltw, 510] as the lower limit value of the effective dynamic range of the high exposure image. Therefore, when the predetermined ratio is 99%, the following expression (7) is established.

  The lower limit value YLmin of the effective dynamic range of the high exposure image is, for example, a value schematically shown in FIG.

The gradation compression unit 35B according to the present embodiment can calculate the weighted average value <Ymin> of the lower limit values YSmin and YLmin as the lower limit value of the effective dynamic range of the composite image. That is, the lower limit value <Ymin> of the effective dynamic range of the composite image can be calculated according to the following equation (8).
<Ymin> = (YSmin × α + YLmin × β) / 2 (8)

  As shown in FIGS. 17A and 17B, the gradation compression unit 35B sets the input gradation level range of <Ymin> to <Ymax> as the effective dynamic range of the composite image, and exceeds the upper limit value <Ymax>. The input gradation level is converted to an output gradation level having a value of 255, and the input gradation level below the lower limit value <Ymin> is converted to an output gradation level having a value of all 0, so that the composite image can be effectively processed. The output gradation level range corresponding to the dynamic range can be expanded. Therefore, it is possible to further widen the dynamic range of the output image.

  As described above, the gradation compression unit 35B according to the second embodiment executes the gradation compression processing using the data Hsd and Htd measured by the histogram measurement unit 23, so that it is compared with the configuration according to the first embodiment. Thus, the circuit scale can be reduced. Moreover, since the exposure conditions can be switched quickly as in the first embodiment, the exposure conditions of the imaging unit 10 can follow at high speed with respect to changes in subject conditions such as subject illuminance. Also, the gradation compression unit 35B dynamically changes the gradation conversion characteristics so as to compensate for the change in the brightness of the composite image due to the switching of the exposure condition, based on the control parameter Yavg that characterizes the brightness of the composite image. . For this reason, the gradation compression unit 35 can output a wide dynamic range image with good image quality.

Embodiment 3 FIG.
Next, a third embodiment according to the present invention will be described. FIG. 18 is a functional block diagram schematically illustrating a configuration of the imaging device 3 according to the third embodiment. The configuration of the image processing unit 20C of the present embodiment is the same as the configuration of the image processing unit 20B of the second embodiment except for the histogram measurement unit 23C, the imaging control unit 24C, and the frequency distribution synthesis unit 36.

  As shown in FIG. 19, the histogram measuring unit 23C according to the present embodiment reads N × M pieces (N and M are integers of 2 or more) of one frame of the input image Fd read and transferred from the frame buffer 22. Are divided into area images A (0, 0) to A (N-1, M-1). The histogram measuring unit 23C measures the luminance histogram measurement data Hd (0,0) to Hd (N-1, M-1) for each of the area images A (0,0) to A (N-1, M-1). It has the function to calculate.

The imaging control unit 24C has the characteristics of the measurement data Hd (i, j) (i is an integer from 0 to N-1; j is an integer from 0 to M-1) of each luminance histogram measured by the histogram measurement unit 23C. Representing parameters Yo ₂₀₀ (i, j), Yu ₁₀₀ (i, j), Yu ₅₀ (i, j), and Yu ₂₅ (i, j) are calculated for each region image.

Here, Yo ₂₀₀ (i, j) is a parameter indicating the ratio (unit: percent) of the pixels having the pixel value (luminance value) of 200 or more in the area image A (i, j) to the total number of effective pixels. , Yu ₁₀₀ (i, j) is a parameter indicating the ratio (unit: percent) of the pixels having pixel values (luminance values) in the range of 0 to 100 in the area image A (i, j) to the total number of effective pixels. Yu ₅₀ (i, j) is a ratio (unit: percent) to the total number of effective pixels of pixels having pixel values (luminance values) in the range of 0 to 50 in the area image A (i, j). Yu ₂₅ (i, j) is a ratio (unit: percent) to the total number of effective pixels of pixels having pixel values (luminance values) within the range of 0 to 25 in the area image A (i, j). ) Is another.

Based on the parameter group, the imaging control unit 24C selects index values Yo ₂₀₀ , Yu ₁₀₀ , Yu ₅₀ , Yu ₂₅ for selecting one exposure condition from the exposure condition candidates P1 to P4 in the exposure condition table 25T. Is calculated. Specifically, the index values Yo ₂₀₀ , Yu ₁₀₀ , Yu ₅₀ , and Yu ₂₅ can be calculated according to the following formula.

Yo ₂₀₀ = MAX {Yo ₂₀₀ (i, j)} (i = 0 to N−1, j = 0 to M−1)
Yu ₁₀₀ = MAX {Yu ₁₀₀ (i, j)} (i = 0 to N−1, j = 0 to M−1)
Yu ₅₀ = MAX {Yu ₅₀ (i, j)} (i = 0 to N−1, j = 0 to M−1)
Yu ₂₅ = MAX {Yu ₂₅ (i, j)} (i = 0 to N−1, j = 0 to M−1)

  Here, MAX {X (i, j)} is a function that outputs the maximum value among N × M values X (0,0) to X (N−1, M−1).

These index values Yo ₂₀₀ , Yu ₁₀₀ , Yu ₅₀ , Yu ₂₅ are used in the same manner as the parameters Yo ₂₀₀ , Yu ₁₀₀ , Yu ₅₀ , Yu _{25 of the} first embodiment. Therefore, the gradation compression unit 35B is similar to the gradation compression unit 35 of the first embodiment by referring to the exposure condition table 25T using the index values Yo ₂₀₀ , Yu ₁₀₀ , Yu ₅₀ , Yu ₂₅ . Exposure conditions can be determined.

  The frequency distribution synthesis unit 36 adds all the N × M luminance histogram measurement data Hd (0,0) to Hd (N−1, M−1) supplied from the histogram measurement unit 23 </ b> C to form one frame image. Measurement data of a luminance histogram of Fd is generated. This measurement data is supplied to the gradation compression unit 35B. For this reason, the gradation compression unit 35B can perform gradation compression processing similar to that in the second embodiment.

  As described above, according to the third embodiment, the imaging control unit 24C performs the measurement data Hd () of the luminance histogram of each of the plurality of area images A (0, 0) to A (N−1, M−1). Exposure control can be performed using 0,0) to Hd (N-1, M-1). For this reason, the imaging control unit 24C determines whether the luminance value is saturated when the in-plane distribution of the luminance value is biased as in the case where the luminance value is saturated in a specific local region of the frame image Fd. Focusing on the region with the highest ratio, exposure condition switching control can be performed. Accordingly, it is possible to quickly select an appropriate exposure condition when capturing a high exposure image and a low exposure image.

Embodiment 4 FIG.
Next, a fourth embodiment according to the present invention will be described. The configuration of the imaging apparatus according to the present embodiment is the same as the configuration of the imaging apparatus according to the first embodiment except for a part of the configuration of the gradation compression unit. For this reason, the gradation compression processing of the present embodiment will be described with reference to FIG.

  When the image processing unit 20 operates in the moving image capturing mode, a low exposure image and a high exposure image are alternately captured as shown in FIG. At this time, the gradation compression unit 35 has a function of limiting the current value Yavg (p) of the control parameter within a certain range from the previous value Yavg (p−1). Here, p is an integer representing the number of the composite image Fc.

  FIG. 20 is a flowchart showing the procedure of the control parameter restriction process. As shown in FIG. 20, the gradation compression unit 35 determines that the current value Yavg (p) of the control parameter is an upper limit value UL (= Yavg (p−1) + Δ) within a certain range from the previous value Yavg (p−1). ) Is determined (step S11). When the current value Yavg (p) does not exceed the upper limit value UL (NO in step S11), the gradation compression unit 35 further determines that the current value Yavg (p) is within a certain range from the previous value Yavg (p-1). It is determined whether it is less than the lower limit LL (= Yavg (p−1) −Δ) (step S12). When the current value Yavg (p) is not less than the lower limit value LL (NO in step S12), the gradation compression unit 35 determines the gradation level conversion characteristic curve using the current value Yavg (p) as it is. (Step S13).

  On the other hand, when the current value Yavg (p) exceeds the upper limit value UL (YES in step S11), the gradation compression unit 35 replaces the current value Yavg (p) with the upper limit value UL (step S14), and thereafter Then, a characteristic curve is determined (step S13). On the other hand, when the current value Yavg (p) is less than the lower limit value LL (YES in step S12), the gradation compression unit 35 replaces the current value Yavg (p) with the lower limit value LL (step S15), and then the characteristics. A curve is determined (step S13).

FIG. 21A is a graph illustrating a characteristic curve with respect to the previous value Yavg (p−1) of the control parameter. FIG. 21B shows three types of current values Yavg ⁽⁰⁾ (p) of the control parameter. , Yavg ⁽¹⁾ (p), Yavg ⁽²⁾ is a graph illustrating characteristic curves CL0, CL1, and CL2 with respect to (p). Since the current value Yavg ⁽⁰⁾ (p) is within a certain range centered on the previous value Yavg (p−1), a characteristic curve CL0 as shown in FIG. 21B is generated. Since the current value Yavg ⁽¹⁾ (p) exceeds the upper limit value UL (= Yavg (p−1) + Δ) within a certain range centered on the previous value Yavg (p−1), FIG. A characteristic curve CL1 as shown in FIG. Further, since the current value Yavg ⁽²⁾ (p) is below the lower limit value LL (= Yavg (p−1) −Δ) within a certain range centered on the previous value Yavg (p−1), FIG. A characteristic curve CL2 as shown in (B) is generated.

  As described above, the control parameter restriction process according to the present embodiment can suppress the fluctuation range of the control parameter within a certain range, so that the gradation compression process is sensitive to a sudden change in the subject condition and the exposure condition. The reaction can be suppressed and the treatment can be stabilized.

Modified example of the first to fourth embodiments.
As mentioned above, although embodiment of this invention was described with reference to drawings, these are illustrations of this invention and can also employ | adopt various forms other than the above. For example, although the first to fourth embodiments generate a composite image using two digital images (a high exposure image and a low exposure image), the present invention is not limited to this. It is also possible to appropriately change the configuration so as to generate a composite image from three or more N (N is an integer of 3 or more) digital images.

  Further, there may be a configuration configured by replacing the histogram measurement unit 23 and the imaging control unit 24 of the first embodiment with the histogram measurement unit 23C and the imaging control unit 24C of the third embodiment.

  In addition, all or part of the functions of the image processing units 20, 20B, and 20C may be realized by a hardware configuration, or may be realized by a computer program executed by a microprocessor. When some of the functions of the image processing units 20, 20B, and 20C are realized by a computer program, the microprocessor loads and executes the computer program from a computer-readable recording medium such as a non-volatile memory to execute image processing. The functions of the units 20, 20B, and 20C can be realized.

  In addition, the imaging devices 1 to 3 can be incorporated in an electronic device such as an image monitoring device, an image display device, or an image recording device.

  1 to 3 imaging devices, 10 imaging units, 12 imaging optical systems, 13 solid-state imaging devices, 14 front end units, 20, 20B, 20C image processing units, 21 signal processing units, 22 frame buffers, 23, 23C histogram measurement units, 24, 24C imaging control unit, 25T exposure condition table, 31 image composition unit, 311 coefficient multiplication unit, 311s, 311t multiplier, 312 addition unit, 313 composition ratio calculation LUT (lookup table), 314s, 314t composition ratio multiplication Unit, 315 synthesis processing unit, 33 average value calculation unit, 34 histogram measurement unit, 35, 35B gradation compression unit, 36 frequency distribution synthesis unit.

Claims (15)

An image processing apparatus that processes an image signal representing an image captured by an imaging unit,
An exposure control histogram measurement unit for measuring a luminance histogram of the image signal;
An exposure condition table in which correspondences between a plurality of exposure condition candidates and a luminance histogram for the imaging unit are stored in advance;
A high-exposure image signal representing a high-exposure image captured at a first exposure amount by the imaging unit and a low-exposure image captured at a second exposure amount lower than the first exposure amount by the imaging unit. An image synthesis unit that synthesizes a low-exposure image signal and generates a composite image signal having a higher number of gradations than the number of gradations of the high-exposure image signal and the low-exposure image signal;
A gradation compression unit that has gradation conversion characteristics that define the correspondence between input gradation levels and output gradation levels, and that compresses the number of gradations of the input composite image signal based on the gradation conversion characteristics When,
A parameter calculation unit for calculating a control parameter characterizing the brightness of the composite image represented by the composite image signal;
An imaging control unit that controls an exposure condition of the imaging unit;
The imaging control unit selects an exposure condition corresponding to the luminance histogram measured by the exposure control histogram measurement unit from the plurality of exposure condition candidates with reference to the exposure condition table, and the selection Set the exposed exposure condition in the imaging unit,
The gradation compression unit changes the gradation conversion characteristic so as to compensate for a change in brightness of the composite image due to switching of an exposure condition of the imaging unit based on the control parameter. Processing equipment.   The image processing apparatus according to claim 1, wherein the gradation compression unit outputs the gradation conversion characteristics when the brightness of the composite image is reduced in accordance with switching of an exposure condition of the imaging unit. An image processing characterized by increasing a gradation level and decreasing an output gradation level of the gradation conversion characteristic when the brightness of the composite image increases in accordance with switching of an exposure condition of the imaging unit apparatus.   The image processing apparatus according to claim 2, wherein the parameter calculation unit calculates an average luminance value of the synthesized image signal as the control parameter. The image processing apparatus according to any one of claims 1 to 3,
A gradation conversion histogram measurement unit for measuring a luminance histogram of the composite image signal;
The gradation compression unit detects an effective dynamic range of the composite image based on the luminance histogram measured by the gradation conversion histogram measurement unit, and an output gradation level range corresponding to the effective dynamic range is expanded. An image processing apparatus characterized by changing the gradation conversion characteristics. The image processing apparatus according to any one of claims 1 to 3,
The exposure control histogram measurement unit measures a first luminance histogram of the low exposure image and a second luminance histogram of the high exposure image,
The gradation compression unit detects an effective dynamic range of the composite image based on the first and second luminance histograms, and the gradation is expanded so that an output gradation level range corresponding to the effective dynamic range is expanded. An image processing apparatus characterized by changing conversion characteristics.   6. The image processing apparatus according to claim 5, wherein the gradation compression unit detects an effective dynamic range of the low-exposure image based on the first luminance histogram and based on the second luminance histogram. Detecting an effective dynamic range of the high-exposure image; an upper limit value of a gradation level range corresponding to the effective dynamic range of the low-exposure image; and an upper limit value of a gradation level range corresponding to the effective dynamic range of the high-exposure image; An average value of the image is used as an upper limit value of the effective dynamic range of the composite image. The image processing apparatus according to any one of claims 1 to 6,
The image composition unit
A weighting factor multiplier that multiplies the luminance components of the low-exposure image signal and the high-exposure image signal by weighting factors, respectively, to generate first and second weighted luminance components;
An adder for adding the first and second weighted luminance components;
A synthesis ratio calculation unit that determines first and second synthesis ratios to be assigned to the low-exposure image signal and the high-exposure image signal, respectively, according to the output signal of the addition unit;
A synthesis ratio multiplier for multiplying the low exposure image signal and the high exposure image signal by the first and second synthesis ratios, respectively, to generate first and second multiplication signals;
An image processing apparatus comprising: a synthesis processing unit that adds the first and second multiplication signals to generate the synthesized image signal.   8. The image processing apparatus according to claim 1, wherein a luminance histogram of the image signal is calculated in units of one frame image. The image processing apparatus according to any one of claims 1 to 7,
The exposure control histogram measurement unit divides one frame of the input image represented by the image signal into a plurality of region images, and uses a plurality of luminance histograms of the plurality of region images as luminance histograms of the image signals. Measure and
The imaging control unit calculates a parameter representing the characteristics of the plurality of luminance histograms measured by the exposure control histogram measurement unit for each of the region images, and based on the parameter, the plurality of exposure condition candidate candidates An image processing apparatus that calculates an index value for selecting one of the exposure conditions. The image processing device according to any one of claims 1 to 9,
The imaging control unit causes the imaging unit to alternately capture the high exposure image and the low exposure image,
The image composition unit generates the composite image signal each time a set of the high exposure image and the low exposure image is captured,
The image processing apparatus, wherein the gradation compression unit limits a current value of the control parameter within a certain range from a previous value of the control parameter. An image processing apparatus according to any one of claims 1 to 10,
The imaging unit
An imaging optical system having a lens and a diaphragm mechanism;
A solid-state imaging device that photoelectrically converts an optical image formed by the imaging optical system;
An amplifier that amplifies the output of the solid-state imaging device,
The image processing apparatus controls the exposure condition by controlling at least one of an exposure time of the solid-state image sensor, a gain of the amplifier, and an aperture value of the aperture mechanism. . An imaging unit;
An exposure control histogram measurement unit that measures a luminance histogram of an image signal representing an image captured by the imaging unit;
An exposure condition table in which correspondence relationships between a plurality of exposure condition candidates for the imaging unit and a plurality of frequency distributions of a luminance histogram are stored;
A high-exposure image signal representing a high-exposure image captured at a first exposure amount by the imaging unit and a low-exposure image captured at a second exposure amount lower than the first exposure amount by the imaging unit. An image synthesis unit that synthesizes a low-exposure image signal and generates a composite image signal having a higher number of gradations than the number of gradations of the high-exposure image signal and the low-exposure image signal;
A gradation compression unit that has gradation conversion characteristics that define the correspondence between input gradation levels and output gradation levels, and that compresses the number of gradations of the input composite image signal based on the gradation conversion characteristics When,
A parameter calculation unit for calculating a control parameter characterizing the brightness of the composite image represented by the composite image signal;
An imaging control unit that controls an exposure condition of the imaging unit;
The imaging control unit selects an exposure condition corresponding to the luminance histogram measured by the exposure control histogram measurement unit from the plurality of exposure condition candidates with reference to the exposure condition table, and the selection Set the exposed exposure condition in the imaging unit,
The gradation compression unit changes the gradation conversion characteristic so as to compensate for a change in brightness of the composite image due to switching of an exposure condition of the imaging unit based on the control parameter. apparatus.   13. The imaging apparatus according to claim 12, wherein when the brightness of the composite image is reduced according to switching of exposure conditions of the imaging unit, the gradation compression unit outputs an output floor of the gradation conversion characteristics. An image pickup apparatus that increases a tone level and reduces an output gradation level of the gradation conversion characteristic when the brightness of the composite image increases in accordance with switching of an exposure condition of the image pickup unit. An image processing method for processing an image signal representing an image captured by an imaging unit,
Measuring a luminance histogram of the image signal;
A high-exposure image signal representing a high-exposure image captured at a first exposure amount by the imaging unit and a low-exposure image captured at a second exposure amount lower than the first exposure amount by the imaging unit. Combining a low-exposure image signal to generate a composite image signal having a higher number of gradations than the number of gradations of the high-exposure image signal and the low-exposure image signal;
Compressing the number of gradations of the composite image signal based on a gradation conversion characteristic that takes the composite image signal as input and defines a correspondence relationship between an input gradation level and an output gradation level;
Calculating a control parameter characterizing the brightness of the composite image represented by the composite image signal;
With reference to an exposure condition table in which correspondences between a plurality of exposure condition candidates for the imaging unit and a plurality of frequency distributions of the brightness histogram are stored, the brightness histogram is selected from the plurality of exposure condition candidates. Selecting an exposure condition corresponding to
Setting the selected exposure condition in the imaging unit;
And a step of changing the gradation conversion characteristic so as to compensate for a change in brightness of the composite image due to switching of an exposure condition of the imaging unit based on the control parameter. The image processing method according to claim 14, comprising:
The step of changing the gradation conversion characteristic includes:
Increasing the output gradation level of the gradation conversion characteristics when the brightness of the composite image is reduced in response to switching of the exposure conditions of the imaging unit;
And a step of reducing an output gradation level of the gradation conversion characteristic when the brightness of the composite image increases in response to switching of an exposure condition of the imaging unit.

Images