JP2023175437A - Imaging apparatus and control method of the same - Google Patents

Abstract

To provide an imaging apparatus which can generate an image with a wide dynamic range, capable of suppressing an image quality deterioration in a synthesis location of a plurality of images.SOLUTION: An imaging apparatus comprises: imaging means capable of performing a photographing by a first photographing method for acquiring an image in a predetermined exposure and a second photographing system for acquiring a plurality of imaging having a different exposure; adjustment means of adjusting a signal value of each image in the image acquired by the imaging means; and generation means of generating a high-dynamic range image by synthesizing a plurality of images acquired by the second photographing system. The adjustment means adjusts the signal value of each image in a different exposure level in each of the first and second photographing systems.SELECTED DRAWING: Figure 13

Description

The present invention relates to an imaging device and a control method thereof.

2. Description of the Related Art Conventionally, in an imaging device that captures an image using an image sensor such as a CMOS image sensor, it is common to adjust the sensitivity to light so that the exposure of the image does not differ due to individual differences in the image sensor.

In particular, in imaging devices that can adjust the sensitivity for each color such as red pixels (R pixels), blue pixels (B pixels), and green pixels (G pixels), the desired color ratio can be achieved using the desired light source. The output is often adjusted as follows.

By the way, in the above-mentioned imaging device, a technique has been proposed to expand the dynamic range regarding gradation.

Patent Document 1 discloses a technique for expanding the dynamic range by applying a plurality of different amplification gains to signals of the same pixel, reading the signals simultaneously, and combining the signals through signal processing.

Additionally, Patent Document 2 discloses a technology that expands the dynamic range by taking multiple shots with different exposures depending on the state of the detected subject, the shooting mode, etc., and compositing them through signal processing. There is.

Considering that shooting to expand the dynamic range as described above requires processing time and is not suitable for continuous shooting, etc., we have set it as a different mode from normal shooting, and users can selectively use it. I often try to make it possible to do so.

JP 2015-128253 Publication Japanese Patent Application Publication No. 2014-171146

When photographing is performed by enlarging the difference in amplification gain using the methods disclosed in Patent Documents 1 and 2 to expand the dynamic range, the image signal may cause blackout of pixels below a predetermined output level or exceed the predetermined output level. Pixel whiteout is less likely to occur.

Specifically, the low-brightness part of the image with high amplification gain is used, and the high-brightness part of the image with low amplification gain where gradation remains is multiplied by the gain of the difference from the image with high amplification gain. can be used to combine these images to create an image with a wide dynamic range.

However, due to the characteristics of the image sensor, when the amplification gains are different, the linearity is not necessarily constant. Therefore, if you try to combine a low-gain image with an "output region higher than the proper exposure of the high-gain image", the "output region of the low-gain image is lower than the proper exposure" will be amplified and used. There is a concern that the synthesis may be performed without matching linearity. In this case, there is a possibility that the image quality of the composite portion will deteriorate.

The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to suppress deterioration in image quality at a point where multiple images are combined in an imaging device that can generate images with a wide dynamic range.

An imaging device according to the present invention includes an imaging means capable of performing imaging using a first imaging method that acquires an image with a predetermined exposure and a second imaging method that acquires a plurality of images with different exposures; an adjusting means for adjusting the signal value of each image in the image acquired by the imaging means; and a generating means for generating a high dynamic range image by combining the plurality of images acquired by the second photographing method. The adjusting means adjusts the signal value of each image at different exposure levels in the first photographing method and the second photographing method.

According to the present invention, in an imaging device that can generate images with a wide dynamic range, it is possible to suppress image quality deterioration at a point where multiple images are combined.

1 is a diagram showing a schematic configuration of an imaging device according to an embodiment of the present invention. FIG. 1 is a diagram showing a schematic configuration of an image sensor. FIG. 2 is a diagram showing the configuration of one column of pixels, column output lines, and column amplification sections in an image sensor. The figure which shows the structure of the analog amplifier circuit in an image sensor. A timing chart of the image sensor in normal shooting mode. 6 is a diagram showing the connection state between the column output line and the connection switch at each timing in FIG. 5. FIG. FIG. 3 is a diagram showing the electrical connection state of the column amplification circuit in normal imaging mode. A timing chart in high dynamic range shooting mode in the image sensor. 9 is a diagram showing the connection state between the column output line and the connection switch at each timing in FIG. 8; FIG. FIG. 3 is a diagram showing electrical connections of a column amplification circuit in a high dynamic range imaging mode. FIG. 7 is a diagram showing a specific example of dynamic range expansion processing in a high dynamic range shooting mode. The figure which shows an example of the output adjustment with respect to the exposure amount in normal photography mode. The figure which shows an example of the output adjustment with respect to the exposure amount in high dynamic range photography mode.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the claimed invention. Although a plurality of features are described in the embodiments, not all of these features are essential to the invention, and the plurality of features may be arbitrarily combined. Furthermore, in the accompanying drawings, the same or similar components are designated by the same reference numerals, and redundant description will be omitted.

FIG. 1 is a diagram showing a schematic configuration of an imaging device 100 according to an embodiment of the present invention.

In FIG. 1, a control unit 101 controls each part of the imaging apparatus 100 while reading a program stored in advance in a ROM 105, which will be described later. For example, the control unit 101 instructs the imaging unit 104, which will be described later, to start and end an imaging operation. Alternatively, an image processing unit 107, which will be described later, is instructed to perform image processing based on a program stored in the ROM 105. Instructions by the user are input into the imaging apparatus 100 by an operation unit 110, which will be described later, and are transmitted to each block of the imaging apparatus 100 via the control unit 101.

The drive unit 102 is constituted by a motor or the like, and drives an optical system 103, which is a photographic lens to be described later, according to instructions from the control unit 101. Further, the optical system 103 is also provided with a diaphragm for restricting the light beam incident on the imaging unit 104, which will be described later. The imaging unit 104 includes an imaging device 104a such as a CMOS sensor, and performs photoelectric conversion to convert an incident optical signal into an electrical signal.

The ROM 105 is a read-only nonvolatile memory that serves as a recording medium, and in addition to operation programs for each block included in the imaging device 100, the ROM 105 stores parameters necessary for the operation of each block, correction data necessary for image correction, adjustment data, etc. I remember. Adjustment data used for sensitivity correction, which will be described later, is also stored in the ROM 105.

The RAM 106 is a rewritable volatile memory, and is used as a temporary storage area for output data during the operation of each block included in the imaging device 100. Further, the RAM 106 is also provided with an area for storing information necessary for photographing, and for example, information on the driving range of the optical system 103 during a depth compositing operation is also stored in a predetermined area of the RAM 106.

The output correction unit 111 receives the output of the imaging unit 104 and corrects the output value based on adjustment data stored in the ROM 105. Correction is mainly performed by digitally applying a gain to the output value and correcting the difference from the desired output based on data stored in the ROM 105.

The image processing unit 107 performs various image processing such as white balance adjustment processing, color interpolation processing, and filtering processing on image data output from the imaging unit 104 or image data recorded in a built-in memory 109 (described later). I do. A part of the sensitivity, which will be described later, is also adjusted by the image processing unit 107. Furthermore, the image processing unit 107 performs compression processing on the image data captured by the imaging unit 104 according to a standard such as JPEG.

The display unit 108 includes a liquid crystal display for displaying image data temporarily stored in the RAM 106, image data stored in a built-in memory 109 (described later), or a setting screen of the imaging device 100. Note that the display section 108 also displays settings for selecting the first shooting method, ``normal shooting mode,'' or the second shooting method, ``high dynamic range shooting mode,'' which will be described later. The user can make settings from the settings screen.

The built-in memory 109 records images captured by the imaging unit 104 and image data processed by the image processing unit 107.

The operation unit 110 includes, for example, operation members such as buttons, switches, keys, and mode dials for inputting instructions to the imaging device 100, a touch panel arranged on the display unit 108, and the like. Instructions from the user are transmitted to the control unit 101 via the operation unit 110.

FIG. 2 is a diagram showing a schematic configuration of the image sensor 104a. The image sensor 104a includes a pixel section 20, a vertical scanning circuit 202, a column output line 203, column amplification sections 204a, 204b, column memory sections 205a, 205b, horizontal scanning circuits 206a, 206b, horizontal signal lines 207a, 207b, and an output circuit 208a. , 208b.
The pixel unit 20 includes a plurality of pixels 201 arranged two-dimensionally in a matrix of m rows and n columns. Although not shown, the pixel 201 includes a photodiode that is a photoelectric conversion element, a pixel output section that outputs a signal based on the charge accumulated in the photodiode, and a pixel selection section that selects a pixel.

The vertical scanning circuit 202 is electrically connected to the pixel selection section of the pixel 201. When a pixel 201 is selected by the vertical scanning circuit 202, the pixel signal photoelectrically converted by the pixel 201 is outputted to the selected column output line 203 by the vertical scanning circuit 202.

A plurality of column output lines 203 are provided in each column of pixels, and each can be electrically connected to column amplification sections 204a and 204b. Pixel signals are input to column amplification sections 204a and 204b via column output lines 203. The column amplification sections 204a and 204b include a plurality of amplification circuits provided for each column output line.

The AD converters 209a and 209b are composed of a ramp signal (reference signal) generation circuit and a comparator, and convert the signals from the column amplifiers 204a and 204b into digital signals. Note that in the AD converters 209a and 209b, it is possible to correct the output value (signal value) by adjusting the ramp signal generation circuit or the like. The adjustment of the output value by the ramp signal generation circuits of the AD converters 209a and 209b will be referred to as "first output adjustment." Note that although the configuration of the above-mentioned AD conversion section, column amplification section, etc. is divided into two systems, a configuration may be adopted in which a uniform amplification gain is applied as a common circuit.

Column memory sections 205a and 205b hold digital signals output from AD conversion sections 209a and 209b. Horizontal scanning circuits 206a, 206b electrically connect column memory sections 205a, 205b to horizontal signal lines 207a, 207b, and output signals from column memory sections 205a, 205b to output circuits 208a, 208b. The signals output from the output circuits 208a and 208b are output to the image processing unit 107 via the output correction unit 111.

FIG. 3 is a diagram showing the configuration of one column of each of the pixels 201, column output line 203, and column amplification section 204 in the image sensor 104a.

One column of pixel array consists of pixels 201(1) to 201(2n) (n is an integer of 1 or more), and two lines of column output lines, 203(odd) and 203(even), are used as the column output line 203. is located.

The column output line 203 (odd) is electrically connected to the odd-numbered row pixels 201 (2n-1) via the row selection switch SEL[2n-1], and the column output line 203 (even) is connected to the row selection switch SEL[2n-1]. It is electrically connected to the even-numbered row pixels 201 (2n) via SEL[2n]. The column amplification section 204a includes a plurality of column amplification circuits 302a, and similarly the column amplification section 204b includes a plurality of column amplification circuits 302b.

One end of the column output line 203(odd) is electrically connected to the column amplifier circuit 302a via a connection switch 301a(odd), and the other end of the column output line 203(odd) is electrically connected via a connection switch 301b(odd). It is electrically connected to column amplification circuit 302b.

Similarly, one end of the column output line 203 (even) is electrically connected to the column amplifier circuit 302a via the connection switch 301a (even), and the other end of the column output line 203 (even) is connected to the connection switch 301b (even). The column amplifier circuit 302b is electrically connected to the column amplifier circuit 302b via the column amplifier circuit 302b.

Further, the row selection switches SEL[1] to [2n] electrically connect the pixels 201 in odd rows and the column output line 203 (odd), and the pixels 201 in even rows and the column output line 203 (even), respectively. .

FIG. 4 is a diagram showing an example of the configuration of column amplifier circuits 302a and 302b. Note that the column amplification circuits 302a and 302b have similar configurations and are also electrically connected to the column output line 203, so the column amplification circuit 302a will be described as a representative of the column amplification circuits.

The column amplification circuit 302a includes a differential amplifier 401, an input capacitor C0, feedback capacitors C1 and C2, and switches S1, S2, and Sr. The reference voltage Vref is input to the non-inverting input node of the differential amplifier 401, and one end of the input capacitor C0 is electrically connected to the inverting input node. The other end of the input capacitor C0 is electrically connected to column output lines 203 (odd) and 203 (even) via connection switches 301a (odd) and 301a (even), respectively.

The inverting input node and output node of differential amplifier 401 can be electrically connected via switch Sr, switch S1 and feedback capacitor C1, and switch S2 and feedback capacitor C2. The gain of the column amplifier circuit 302a can be switched by controlling the switch S1 and the switch S2.

For example, when switches S1 and S2 are on, the gain of column amplification circuit 302a is C0/(C1+C2), and when only switch S1 is on, the gain of column amplification circuit 302a is C0/C1. Become. Furthermore, by turning on all the switches S1, S2, and Sr, the output node and the inverting input node of the differential amplifier 401 are short-circuited, and the feedback capacitances C1 and C2 are reset.
In this embodiment, the electrical connections between the column output line 203 and the column amplification circuits 302a and 302b are switched by connection switches 301a (odd), 301b (odd), 301a (even), and 301b (even). Furthermore, by switching and driving the gains of the column amplifier circuits 302a and 302b, the normal photography mode and the high dynamic range photography mode are switched.

Hereinafter, driving methods in each operation mode will be explained using timing charts.
FIG. 5 shows a timing chart in the normal photographing mode, and FIG. 6 shows the electrical connection state between the column output line and the connection switch in each timing period. Further, FIG. 7 shows the electrical connection state of each switch for setting the gain of the column amplifier circuits 302a and 302b.

5 to 7, the vertical pulse PV is supplied to the vertical scanning circuit 202, and the row selection switches SEL[1] to [n] are turned on every two rows in synchronization with the vertical pulse PV. When the shooting mode is the normal shooting mode, the row selection switches SEL[1] and SEL[2] are turned on from time t0 to t1, the pixels 201 in the first and second rows are selected, and the pixel signal is is read out. Similarly, row selection switches SEL[3] and SEL[4] are turned ON between times t1 and t2, and row selection switches SEL[5] and SEL[6] are turned ON between times t2 and t3, and each selection The pixel signals are read out. Although not shown, such an operation is repeated for the number of 2n rows arranged in the pixel section 20 (that is, n times).

Further, when the mode for reading out the photographed signal is the high-speed reading mode, the switches 301a (odd) and 301b (even) are always ON. Therefore, the pixel signals of the odd rows electrically connected to the column output line 203 (odd) are amplified by the column amplification circuits 302a, and the pixel signals of the even rows electrically connected to the column output line 203 (even) are amplified by the column amplification circuits 302a. Each pixel signal is amplified by a column amplification circuit 302b.

In the normal photographing mode, the switches S1a, S2a, S1b, and S2b of the column amplification circuits 302a and 302b are ON, and the gains of the column amplification circuits 302a and 302b are both the same gain Co/(C1+C2).

As described above, in the normal photographing mode, every two rows are selected at the same time, and odd-numbered rows and even-numbered rows are read out by different column amplification circuits 302a and 302b, respectively. In this way, by simultaneously reading out two rows of pixel signal data, it is possible to achieve high-speed reading.

FIG. 8 shows a timing chart in the high dynamic range imaging mode, and FIG. 9 shows the electrical connection state of the column output line connection switch in each timing period. Further, FIG. 10 shows the electrical connection state of each switch for setting the gain of the column amplifier circuits 302a and 302b.

The high dynamic range imaging mode here is a driving method in which the output signal of the same pixel obtained by one imaging is read out using a plurality of different amplification gains. In this embodiment, in one image sensor 104a, it is possible to select a normal imaging mode in which signals from pixels in multiple rows are read out simultaneously, and a high dynamic range imaging mode in which signals from the same pixel are read out by multiple column amplifier circuits.

In the high dynamic range shooting mode, in synchronization with the vertical scanning pulse PV, the row selection switch SEL[1] is selected from time t0 to t1, the row selection switch SEL[2] is selected from time t1 to t2, and the row selection switch SEL[2] is selected from time t1 to t2. SEL[3] is selected from t2 to t3. In this way, pixels are selected row by row.

From time t0 to t1, the connection switches 301a (odd) and 301b (odd) are turned on, and the connection switches 301a (even) and 301b (even) are turned off. The selected pixel signal of the first row (odd row) is amplified by two column amplification circuits 302a and 302b.

At the next time t1 to t2, the connection switches 301a (odd) and 301b (odd) are turned off, and the connection switches 301a (even) and 301b (even) are turned on. Thereby, the pixel signal of the selected second row (even row) is amplified by the two column amplification circuits 302a and 302b.

Subsequently, from time t2 to t3, the pixel signal of the selected third row (odd number row) is amplified by the two column amplification circuits 302a and 302b, similar to the electrical connection state from time t0 to t1.

In this way, pixel reading is repeated for the number of 2n rows arranged in the pixel section 20. At this time, as shown in FIG. 10, in the column amplifier circuit 302a, the switches S1a and S2a are turned on, and the gain of the column amplifier circuit 302a becomes C0/(C1+C2). On the other hand, in the column amplifier circuit 302b, only the switch S1b is turned on, and the gain of the column amplifier circuit 302b becomes C0/C1.

For example, if the values of each capacitance are C0 = 800 [fF], C1 = 50 [fF], and C2 = 750 [fF], the gain of the column amplifier circuit 302a is 1 times, and the gain of the column amplifier circuit 302b is 16 times ( = 3 steps higher state as output).

As a result, when sending charges of pixels that have been exposed to the same light to the subsequent image processing unit 107, the output of the low gain column amplification circuit 302a is at the upper limit of the circuit, compared to the high gain column amplification circuit 302b. It can create difficult situations. Note that the above-mentioned gain is not limited to 16 times, and any gain G can be used.

The signals output from the column amplifier circuits 302a, 302b as described above are output from the output circuit 208 via the AD conversion sections 209a, 209b, the column memory sections 205a, 205b, and the like. After that, the output correction unit 111 corrects the output value, and then the image processing unit 107 performs dynamic range expansion processing.

In dynamic range expansion processing, for example, for pixels whose signal is lower than a predetermined level, a high-gain signal is used as is, and for pixels whose signal is higher than a predetermined level, a low-gain signal is used to generate image data. .

A specific example of dynamic range expansion processing in the high dynamic range shooting mode will be described using FIG. 11. FIG. 11 is a diagram showing output values with respect to exposure amount in the high dynamic range shooting mode. Note that the output value indicates a value obtained by adjusting the sensitivity based on the appropriate exposure in the normal shooting mode.

FIG. 11A shows the state of the output signal via the high gain side using the column amplification circuit 302b and the output signal via the low gain side using the column amplification circuit 302a before input to the image processing unit 107. It shows.

As the imaging device acquires images at the same time, when a photograph is taken under exposure conditions (photographing conditions) in which the high gain side provides proper exposure, the exposure on the low gain side is only the gain difference between the column amplification circuit 302b and the column amplification circuit 302a. Get dark.

FIG. 11(b) shows an output signal that has passed through the low gain side using the column amplification circuit 302a, and the number of stages has been corrected (gain multiplied) by the image processing unit 107 to synthesize the high dynamic range image. Showing a signal.

By multiplying the gain ratio between the low gain and the high gain, the signal level can be increased to a signal output close to the high gain image obtained at the same time.

FIG. 11(c) shows that, as a dynamic range expansion process, the high gain side signal in FIG. 11(a) and the low gain side signal in FIG. This shows a state in which the amplified) signals are combined at a desired combination exposure area.

Since the signal on the low gain side, which has a small amount of charge, has a low exposure (brightness) linearity as an image sensor, the linearity tends to shift in the composite area, and shifts occur for each color, resulting in poor color ratio. A deviation may also occur.

Therefore, the sensitivity adjustment target for each mode in this embodiment will be explained using FIGS. 12 and 13.

FIG. 12 is a diagram showing an example of output correction for the exposure amount in the normal shooting mode of this embodiment.

The output signals of column amplifier circuits 302a and 302b are shown by broken lines. In addition, gain correction is applied by the AD conversion units 209a and 209b, which are the "first output adjustment means" at the subsequent stage, and the output correction unit 111, which is the "second output adjustment means" at the rear stage of the imaging unit 104. A signal sent to the image processing unit 107 is shown by a solid line. The column amplification circuits 302a and 302b and the output correction section 111 are provided before the image processing section 107.

For comparison with the high dynamic range photographing mode described later, signals of each color (G, R, B) with low gain (for example, ISO 100) and high gain (ISO 800) are shown. Note that each correction is performed by applying a uniform gain to each sensitivity of each imaging device, regardless of the exposure amount.

In the normal shooting mode, the exposure and the balance of each color signal are adjusted and output by the AD converters 209a and 209b so that R, G, and B are output properly under the proper exposure conditions of the desired light source that the subject is most likely to encounter. Adjustment is performed by applying an adjustment gain in the correction unit 111.

In the normal shooting mode, regardless of whether the sensitivity set by the imaging device is low gain or high gain, each color has a similar balance of color signals with proper exposure. The target exposure condition to be adjusted in the normal shooting mode is referred to as a "first predetermined exposure level."

FIG. 13 is a diagram showing an example of output correction for the exposure amount in the high dynamic range shooting mode.

Similarly to FIG. 12, the output signals of the column amplification circuits 302a and 302b are shown by broken lines, and the image processing unit 107 is subjected to gain correction by the AD conversion units 209a and 209b at the subsequent stage, and the output correction unit 111 at the downstream stage of the imaging unit 104. The solid line indicates the signal sent to. Each color (G, R, B) with low gain (for example, ISO 100) and high gain (ISO 800) is shown.

In the high dynamic range shooting mode, in the dynamic range expansion process in the image processing unit 107, the desired output for each color is achieved in the composite exposure area of the high gain image and the image with the low gain corrected by the number of steps. Output correction is performed at each exposure level at low gain and high gain so that the ratio is the same. Specifically, the output correction is performed before the image processing unit 107 amplifies the difference in the number of stages between the high gain and the low gain on the low gain image.

For example, if the composite exposure region (composite exposure range) is "appropriate exposure at high gain + around 0.5 stops", output correction on the high gain side is performed for the output value of "appropriate exposure + 0.5 stops". Furthermore, on the low gain side, a luminance target (color ratio target) for each color is set for an output value of "appropriate -2.5 stages" as a state before stage number correction during composition. That is, in this example, the output of each color is adjusted at the exposure level of "appropriate -2.5 steps" to balance the color ratio. Here, the exposure condition (here, "appropriate -2.5 stops") that is the adjustment target for the low gain image in the high dynamic range shooting mode is referred to as the "second predetermined exposure level." By performing the above-described correction, it is possible to reduce exposure deviation and color ratio deviation in the composite exposure area.

Let me explain this point in a little more detail.

The synthesis region in which the low gain image is corrected by the number of steps and synthesized with the high gain image is a region where the output is originally low, and the linearity is about to collapse. Therefore, simply amplifying the low gain image and correcting the number of stages may result in a shift in color ratio in the region where it is combined with the high gain image.

Therefore, in this embodiment, before correcting the number of stages of the low gain image, the color ratio of the output level points to be combined after correcting the number of stages of the low gain image is adjusted to the color ratio of the high gain to be combined. Thereby, it is possible to reduce the color ratio deviation at the compositing point, and it is possible to improve the linearity of the color ratio after compositing.

(Modified example)
Note that although the high dynamic range shooting mode of the above embodiment is explained using an example in which multiple exposure images are simultaneously acquired with different gains (low gain, high gain), the present invention is not limited to this. It's not something you can do.

For example, even in high dynamic range shooting mode, where multiple exposure images are acquired by acquiring images with different exposure times, images can be obtained by changing the exposure adjustment value during proper exposure shooting, underexposure, and overexposure shooting. , it is possible to obtain similar effects.

The disclosure of this specification includes the following imaging device and control method.

(Item 1)
an imaging means capable of photographing using a first photographing method that acquires an image with a predetermined exposure and a second photographing method that acquires a plurality of images with different exposures;
Adjustment means for adjusting the signal value of each image in the image acquired by the imaging means;
generating means for generating a high dynamic range image by combining the plurality of images acquired by the second imaging method,
The image pickup apparatus is characterized in that the adjustment means adjusts the signal value of each image at different exposure levels in the first photography method and the second photography method.

(Item 2)
The imaging device according to item 1, wherein the adjustment of the signal value of each image is an adjustment of the balance of each color signal.

(Item 3)
In the first photographing method, the adjustment means adjusts the signal value of the image at a first predetermined exposure level, and in the second photographing method, the adjusting means adjusts the signal value of the image at a first predetermined exposure level, and in the second photographing method, adjusts the signal value of the image under a plurality of photographing conditions for photographing a plurality of images with different exposures. 3. The imaging device according to item 1 or 2, wherein the image pickup apparatus corrects the signal value of the image at a second predetermined exposure level different from the first predetermined exposure level for at least one of the image pickup apparatuses.

(Item 4)
Item 1, in the second photographing method, the adjustment means performs adjustment so that the balance of each color signal falls within a predetermined range in a predetermined exposure range after combining the plurality of images. The imaging device according to any one of items 3 to 3.

(Item 5)
According to any one of items 1 to 4, in the second imaging method, the imaging means acquires the plurality of images by amplifying signals of the same pixel with different gains. Imaging device.

(Item 6)
The adjustment means includes a first output adjustment means that performs a common correction on the plurality of images, and is arranged after the first output adjustment means and performs different corrections on each of the plurality of images. 6. The imaging device according to any one of items 1 to 5, further comprising a second output adjustment means capable of adjusting the color signal, and wherein the second output adjustment means adjusts the balance of each color signal.

(Item 7)
7. The imaging device according to item 6, wherein the first output adjustment means corrects the signal values of the plurality of images by adjusting a reference signal used for AD conversion.

(Item 8)
The generating means combines the plurality of images by multiplying an image with a lower exposure level among the plurality of images by an amplification factor corresponding to a difference in exposure level between the plurality of images. The imaging device according to any one of items 1 to 7.

(Item 9)
9. The imaging device according to any one of items 1 to 8, wherein the adjusting means is arranged before the generating means.

(Item 10)
A method for controlling an imaging device including an imaging means capable of performing imaging using a first imaging method that acquires an image with a predetermined exposure and a second imaging method that acquires a plurality of images with different exposures. hand,
an adjustment step of adjusting the signal value of each image in the image acquired by the imaging means;
a generation step of generating a high dynamic range image by combining the plurality of images acquired by the second imaging method,
A method for controlling an imaging apparatus, wherein in the adjustment step, signal values of each image are adjusted at different exposure levels in the first imaging method and the second imaging method.

(Other embodiments)
The present invention also provides a system or device with a program that implements one or more functions of the above-described embodiments via a network or a storage medium, and one or more processors in a computer of the system or device reads the program. This can also be achieved by executing a process. It can also be implemented by a circuit (eg, ASIC) that implements one or more functions.

The invention is not limited to the embodiments described above, and various changes and modifications can be made without departing from the spirit and scope of the invention. Therefore, the following claims are hereby appended to disclose the scope of the invention.

100: Imaging device, 101: Control unit, 102: Drive unit, 103: Optical system, 104: Imaging unit, 104a: Imaging element, 105: ROM, 106: RAM, 107: Image processing unit, 108: Display unit, 109 : built-in memory, 110: operation section, 111: output correction section, 20: pixel section, 201: pixel, 202: vertical scanning circuit, 203: column output line, 204: column amplification section, 205: column memory section, 206: Horizontal scanning circuit, 207: Horizontal signal line, 208: Inverting amplifier, 209: AD conversion section

Claims (10)

an imaging means capable of photographing using a first photographing method that acquires an image with a predetermined exposure and a second photographing method that acquires a plurality of images with different exposures;
Adjustment means for adjusting the signal value of each image in the image acquired by the imaging means;
generating means for generating a high dynamic range image by combining the plurality of images acquired by the second imaging method,
The image pickup apparatus is characterized in that the adjustment means adjusts the signal value of each image at different exposure levels in the first photography method and the second photography method. The imaging apparatus according to claim 1, wherein the adjustment of the signal value of each image is an adjustment of the balance of each color signal. In the first photographing method, the adjustment means adjusts the signal value of the image at a first predetermined exposure level, and in the second photographing method, the adjusting means adjusts the signal value of the image at a first predetermined exposure level, and in the second photographing method, adjusts the signal value of the image under a plurality of photographing conditions for photographing a plurality of images with different exposures. 2. The imaging apparatus according to claim 1, wherein for at least one signal value of the image is corrected at a second predetermined exposure level different from the first predetermined exposure level. 2. In the second photographing method, the adjustment means performs adjustment so that the balance of each color signal falls within a predetermined range in a predetermined exposure range after combining the plurality of images. 1. The imaging device according to 1. 2. The imaging apparatus according to claim 1, wherein in the second imaging method, the imaging means acquires the plurality of images by amplifying signals of the same pixel with different gains. The adjustment means includes a first output adjustment means that performs a common correction on the plurality of images, and is arranged after the first output adjustment means and performs different corrections on each of the plurality of images. 2. The imaging apparatus according to claim 1, further comprising a second output adjustment means capable of adjusting the color signal, and wherein the second output adjustment means adjusts the balance of each color signal. The imaging device according to claim 6, wherein the first output adjustment means corrects the signal values of the plurality of images by adjusting a reference signal used for AD conversion. The generating means combines the plurality of images by multiplying an image with a lower exposure level among the plurality of images by an amplification factor corresponding to a difference in exposure level between the plurality of images. The imaging device according to claim 1. The imaging device according to claim 1, wherein the adjustment means is arranged before the generation means. A method for controlling an imaging device including an imaging means capable of performing imaging using a first imaging method that acquires an image with a predetermined exposure and a second imaging method that acquires a plurality of images with different exposures. hand,
an adjustment step of adjusting the signal value of each image in the image acquired by the imaging means;
a generation step of generating a high dynamic range image by combining the plurality of images acquired by the second imaging method,
A method for controlling an imaging apparatus, wherein in the adjustment step, signal values of each image are adjusted at different exposure levels in the first imaging method and the second imaging method.

Images