JP2014007437A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To image a subject such that the cumulative frequency of pixels imaged by high-sensitivity photoelectric conversion characteristic becomes a fixed ratio.SOLUTION: A luminance distribution detection part 602 detects the luminance distribution of image data outputted from a characteristic conversion part 601 and outputs it as evaluation data to a control part 102. The control part 102 determines whether the cumulative frequency has a value in the predetermined frequency acceptable range with a lower limit value Th1 and an upper limit value Th2. If the cumulative frequency is not larger than the lower limit value Th1, the control part 102 shortens an exposure period. If the cumulative frequency is not smaller than the upper limit value Th2, the control part 102 extends an exposure period.

Description

  The present invention relates to an imaging apparatus that images a subject with a plurality of photoelectric conversion characteristics having different sensitivities.

  2. Description of the Related Art In recent years, an imaging apparatus that captures a subject with a plurality of photoelectric conversion characteristics having different sensitivities in order to acquire a high dynamic range and high sensitivity image is known.

  For example, Patent Document 1 discloses a plurality of regions divided according to the amount of incident light for the purpose of obtaining a high-contrast image without saturating the camera output, even for a subject with a large contrast. It is disclosed that the image pickup means having different input / output characteristics is provided, the screen light quantity K is obtained from the average value of the video luminance signal, and the image pickup means is adjusted so that the screen light quantity K matches the light quantity target value R. .

  Further, in Patent Document 1, the light quantity target value R is set so that the light quantity Q and the light quantity target value R at the switching point on the lowest illuminance side in the input / output characteristics region of the imaging unit satisfy R <Q. Is disclosed.

  Further, in Patent Document 1, blackout is generated when the total number of pixels in which the video luminance signal is 0 exceeds a predetermined value (for example, 1% of the light amount detection area) in the video luminance signal output from the imaging unit. It is disclosed to determine that there is.

JP 2006-237851 A

  However, in Patent Document 1, since the switching point is controlled based on the average value of the video luminance signal, depending on the luminance distribution of the subject, the number of pixels captured in the linear region is greatly reduced, and the image is captured with high sensitivity. There is a problem that the number of pixels to be reduced decreases. That is, in Patent Document 1, there is a problem that the number of pixels picked up with high sensitivity varies according to the luminance distribution of the subject, and a high-sensitive image cannot be acquired stably.

  Moreover, in patent document 1, since the image of a high dynamic range will be output as it is, there exists a subject that it will display as an image with a low contrast, if it displays with the display of a low dynamic range.

  An object of the present invention is to provide an image pickup apparatus that picks up an image of a subject so that the cumulative frequency of pixels picked up with high-sensitivity photoelectric conversion characteristics becomes a constant ratio.

  (1) An imaging device according to the present invention is an imaging unit that images a subject with a first photoelectric conversion characteristic and a second photoelectric conversion characteristic that is less sensitive than the first photoelectric conversion characteristic, and is captured by the imaging unit. A luminance distribution detecting unit that detects a luminance distribution of the image; and a control unit that controls an exposure time of the imaging unit, wherein the control unit is imaged based on the luminance distribution by the first photoelectric conversion characteristic. The cumulative frequency of the pixel is detected, and the exposure time is controlled so that the cumulative frequency satisfies a predetermined frequency allowable range.

  According to this configuration, the subject is imaged by the first and second photoelectric conversion characteristics (however, the sensitivity is the first photoelectric conversion characteristic> the second photoelectric conversion characteristic) having different sensitivities, and the luminance distribution of the obtained image is detected. The Then, the cumulative frequency of the pixels imaged in the first photoelectric conversion characteristic having high sensitivity is detected from the detected luminance distribution, and the exposure time is adjusted so that the cumulative frequency satisfies a predetermined frequency allowable range.

  Here, when the exposure time is lengthened, the sensitivity is improved, but there is a characteristic that the cumulative frequency of pixels imaged by the first photoelectric conversion characteristic is reduced. On the other hand, when the exposure time is shortened, the sensitivity decreases, but there is a characteristic that the cumulative frequency of highly sensitive pixels imaged by the first photoelectric conversion characteristic increases.

  Therefore, by using this characteristic, it is possible to adjust the ratio of the pixels imaged with the first photoelectric conversion characteristic in the entire image within a predetermined range, and it is possible to obtain a stable and highly sensitive image. .

  (2) The first photoelectric conversion characteristic is a linear characteristic, the second photoelectric conversion characteristic is a logarithmic characteristic, and further includes a characteristic conversion unit that converts the linear characteristic and the logarithmic characteristic into a reference photoelectric conversion characteristic, Preferably, the distribution detection unit detects a luminance distribution of the image converted into the reference photoelectric conversion characteristic.

  According to this configuration, since the exposure time is adjusted so that the cumulative frequency of the pixels in the linear region falls within the allowable frequency range, a highly sensitive image can be obtained in an image having linear logarithmic characteristics.

  (3) It is preferable that the control unit adjusts the pixel value of the inflection point so that the pixel value of the inflection point between the linear characteristic and the logarithmic characteristic is constant before and after the control of the exposure time. .

  According to this configuration, since the pixel value of the inflection point is made constant before and after the adjustment of the exposure time, the logarithmic region can be maintained before and after the adjustment of the exposure time.

  (4) The imaging unit captures the subject by multiple exposure to acquire the first and second images of the first and second photoelectric conversion characteristics so that an image having a predetermined light amount or more becomes the second image. It is preferable that a synthesis unit for synthesizing the first and second images is further included, and the luminance distribution detection unit detects a luminance distribution of the image synthesized by the synthesis unit.

  According to this configuration, a high-sensitivity image can be obtained when multiple exposure is employed.

  (5) It is preferable that the control unit controls the exposure time of the second photoelectric conversion characteristic in conjunction with the control of the exposure time of the first photoelectric conversion characteristic.

  According to this configuration, since the exposure time of the second photoelectric conversion characteristic is controlled in conjunction with the control of the exposure time of the first photoelectric conversion characteristic, a highly sensitive image can be obtained while maintaining the dynamic range. .

  (6) The control unit calculates a sensitivity amplification factor for making the brightness of the image converted into the reference photoelectric conversion characteristic constant, based on the luminance distribution detected by the luminance distribution detection unit. It is preferable to further include a sensitivity amplifying unit that amplifies the sensitivity of the image with the sensitivity amplification factor.

  According to this configuration, since the sensitivity of the image is amplified so that the brightness of the entire image is constant, it is possible to prevent the overall image from becoming a dark image or an overall image that is too bright.

  (7) The control unit obtains an average value of pixel values of pixels equal to or less than the pixel value of the inflection point based on the luminance distribution, and the sensitivity amplification factor so that the obtained average value falls within a predetermined target luminance range. Is preferably set.

  According to this configuration, since the sensitivity amplification factor is set so that the average value of the pixel values of the pixels below the pixel value of the inflection point falls within a predetermined target luminance range, it is not too bright while being highly sensitive. An image that is not too dark can be obtained.

  (8) The control unit detects a predetermined control point in the luminance distribution, uses the image amplified by the sensitivity control unit with the sensitivity amplification factor as an input image, and has a pixel value at the control point of the input image. It is preferable to further include an input / output characteristic conversion unit that obtains a conversion characteristic for converting the input / output characteristic of the input image so as to become a specified value and converts the input / output characteristic of the input image by the conversion characteristic.

  According to this configuration, since the conversion characteristic such that the pixel value of the control point becomes the specified value is obtained and the input / output characteristic of the input image is converted using the conversion characteristic, the input image is converted based on the luminance distribution. Input / output characteristics can be converted. Here, as the conversion characteristic, for example, a compression characteristic applied to dynamic range compression processing or a gradation characteristic applied to gamma correction can be employed.

  (9) It is preferable that the control point includes at least an average pixel value of pixels equal to or less than a pixel value of an inflection point and a maximum pixel value.

  According to this configuration, the input / output characteristics of the input image can be converted so that the average value and the maximum pixel value of the pixels below the inflection point maintain the prescribed values.

  (10) Preferably, the control unit determines an imaging scene based on the luminance distribution, and when the imaging scene is at night, the frequency allowable range is set higher than when the imaging scene is in daytime.

  In the case of nighttime, there is a tendency that many pixels are distributed on both the dark side and the bright side in the luminance distribution. On the other hand, in the daytime, pixels tend to be distributed in the intermediate area. Therefore, when the exposure time of an image at night is controlled using a frequency allowable range set based on daytime, the maximum pixel value may exceed the saturation level of the imaging unit and the image may be saturated.

  In this configuration, in the case of nighttime, the frequency tolerance is set to a higher side than in the case of daytime. This can prevent image saturation at night.

  According to the present invention, it is possible to adjust the ratio of the pixels captured with the first photoelectric conversion characteristic to the entire image within a predetermined range, and it is possible to obtain a stable and highly sensitive image.

1 is an overall configuration diagram of an imaging apparatus according to Embodiment 1 of the present invention. (A) is a circuit diagram of the pixel with which an imaging part is provided, (b) is the graph which showed the photoelectric conversion characteristic which the pixel shown to (a) has. It is the graph which showed the photoelectric conversion characteristic which the pixel in Embodiment 2 of this invention has. It is the graph which showed the relationship between the exposure time of an imaging part, and a photoelectric conversion characteristic. This is a photoelectric conversion characteristic of a pixel when only the exposure time is adjusted and the inflection point pixel value is not adjusted. FIG. 2 is a block diagram illustrating a detailed configuration of an image processing unit illustrated in FIG. 1. It is the block diagram which showed the detailed structure of the characteristic conversion part. It is a graph for demonstrating the process of a characteristic conversion part. (A) is a block diagram which shows the detailed structure of a luminance distribution detection part, (b) is the graph which showed an example of the luminance distribution which a luminance distribution detection part detects. (A) is a block diagram which shows the detailed structure of a sensitivity amplification part, (b) is a graph explaining the process by a sensitivity amplification part. It is a block diagram which shows the detailed structure of DR compression part. (A) shows an input image input to the illumination component extraction unit, (b) an illumination component image output from the illumination component extraction unit and a compressed image output from the compression unit, and (c) a reflectance component extraction. (D) is a diagram showing the characteristics of the output image output from the synthesis unit. It is a graph which shows the relationship between exposure time and a photoelectric conversion characteristic, a vertical axis | shaft shows a pixel value and the horizontal axis has shown the light quantity in logarithm. It is explanatory drawing of the relationship between exposure time and luminance distribution. It is a flowchart which shows the process at the time of the imaging device by Embodiment 1 of this invention controlling exposure time. It is the graph which showed the photoelectric conversion characteristic when not adjusting an inflection point pixel value. (A) shows the photoelectric conversion characteristics of each image data when two exposure times are set during one frame period and two image data are acquired, and (b) shows different photoelectric conversion characteristics of (a). The photoelectric conversion characteristics when two pieces of image data are combined into one piece of image data are shown. It is a graph explaining a mode that the exposure time of 1st, 2nd image data is controlled. It is a flowchart which shows the process at the time of the imaging device by Embodiment 2 of this invention controlling exposure time. It is a figure explaining the difference in the light quantity distribution of nighttime and daytime, (a) shows the light quantity distribution in nighttime, (b) has shown the light quantity distribution in daytime. It is the graph which showed the photoelectric conversion characteristic of image data. It is explanatory drawing of the imaging scene discrimination | determination process, (a) is a luminance distribution of the image data in one imaging scene, (b) is a luminance distribution of the image data in another imaging scene. It is a flowchart of an imaging scene discrimination process. (A) shows an example of the luminance distribution at night, and (b) shows an example of the luminance distribution at daytime. 7 is a flowchart illustrating processing when controlling the exposure time when the imaging apparatus according to Embodiment 1 of the present invention adopts the first strategy. It is a graph which shows the photoelectric conversion characteristic of the image data converted into the logarithmic characteristic. (A) is a luminance distribution of the photoelectric conversion characteristic 2601 shown in FIG. 26, and (b) is a luminance distribution of the photoelectric conversion characteristic 2602 shown in FIG. It is a flowchart of sensitivity control. It is a graph which shows a compression characteristic. The luminance distribution detected by the luminance distribution detector is shown. FIG. 10 is a block diagram illustrating a modification of the image processing unit in the first and second embodiments. It is the graph which showed the gradation characteristic which a gamma correction part uses for gamma correction. It is a block diagram of the image processing part by Embodiment 2 of this invention.

(Embodiment 1)
FIG. 1 is an overall configuration diagram of an imaging apparatus according to Embodiment 1 of the present invention. The imaging device includes an imaging unit 101, a control unit 102, and an image processing unit 103. The imaging unit 101 images a subject with a plurality of photoelectric conversion characteristics having different sensitivities. In the first embodiment, linear characteristics and logarithmic characteristics are employed as the plurality of photoelectric conversion characteristics, and the imaging unit 101 includes a plurality of pixels having linear logarithmic characteristics in which the linear characteristics and logarithmic characteristics are switched at inflection points. .

  The control unit 102 includes, for example, a CPU and a calculation memory, and controls the image pickup device and the image processing unit 103 according to a program stored in a storage unit (not shown). Here, the control unit 102 outputs control data for adjusting the inflection point to the imaging unit 101. Further, the control unit 102 generates control data for requesting the image processing unit 103 to perform various image processing in accordance with the evaluation data output from the image processing unit 103, and outputs the control data to the image processing unit 103.

  The image processing unit 103 receives the image data captured by the imaging unit 101 and executes predetermined image processing according to the control data output from the control unit 102.

  2A is a circuit diagram of the pixel 31a included in the imaging unit 101, and FIG. 2B is a graph showing the photoelectric conversion characteristics of the pixel 31a shown in FIG.

  The pixel 31a includes a photodiode PD, a transfer transistor Q1, a reset transistor Q2, a floating diffusion FD, an output transistor Q3, and a selection transistor Q4.

  PVDD and PVSS are power supply potentials of the pixel 31a, PVDD indicates a positive power supply potential, PVSS indicates a negative power supply potential, and are generated by, for example, a power supply circuit (not shown).

  The readout sequence of the pixel 31a is as follows. In the exposure time, φRST = Hi is set from a vertical scanning circuit (not shown), φTX = Lo, the reset transistor Q2 is turned on, the transfer transistor Q1 is turned off, and the floating diffusion FD is reset. When the exposure time ends, φRST = Lo, and the reset transistor Q2 is turned off.

  Next, φVSEN = Hi, and the selection transistor Q4 is turned on. As a result, the potential in the reset state of the floating diffusion FD is output as a noise signal 31n to the common vertical signal line 31b for each vertical column of the pixels 31a via the output transistor Q3.

  Next, φTX = Hi is set and the transfer transistor Q1 is turned on. As a result, the photocurrent Ip flows through the PD, and the charge accumulated in the PD is transferred to the floating diffusion FD and accumulated. Here, during the exposure time, φTX is set to an intermediate potential between Lo and Hi. Therefore, when the amount of light is low, the charge accumulated in the floating diffusion FD has a linear characteristic. On the other hand, when the amount of light is high, the charge accumulated in the floating diffusion FD has a logarithmic characteristic due to the subthreshold characteristic of the transfer transistor Q1. The switching point between the linear characteristic and the logarithmic characteristic, that is, the light amount Lpt at the inflection point Pt can be arbitrarily set by setting the transfer signal φTX of the transfer transistor Q1.

  Next, φVSEN = Hi is set, the selection transistor Q4 is turned on, and the potential of the floating diffusion FD is output as the pixel signal 31p to the vertical signal line 31b via the output transistor Q3.

  The output pixel signal 31p is converted between the pixel signal 31p and the noise signal 31n by using a correlated double sampling (hereinafter referred to as CDS) method by a subsequent circuit such as an A / D converter (not shown). An imaging signal 31s (31s = 31p−31n) in which the difference is taken and noise is canceled is obtained.

  Then, again, φRST = Hi is set, the exposure time is started, and the next frame is imaged.

  In the graph of FIG. 2B, the horizontal axis indicates the amount of light in logarithm, and the vertical axis indicates the pixel value (corresponding to the imaging signal 31s) linearly. In a state of lower brightness than the light amount Lpt at the inflection point Pt (A1 region), the pixel value changes linearly with respect to the light amount and has a linear characteristic. On the other hand, in a state of higher luminance than the light amount Lpt at the inflection point Pt (A2 region), the pixel value changes logarithmically with respect to the light amount and has logarithmic characteristics. Therefore, it can be seen that the pixel 31a has a linear logarithmic photoelectric conversion characteristic having a linear characteristic on the low luminance side and a logarithmic characteristic on the high luminance side with the inflection point Pt as a boundary.

  FIG. 4 is a graph showing the relationship between the exposure time of the imaging unit 101 and the photoelectric conversion characteristics. The vertical axis indicates the pixel value (imaging signal 31s) linearly, and the horizontal axis indicates the light quantity in logarithm.

  By controlling the exposure time of the imaging unit 101, the sensitivity of the linear region can be changed in the photoelectric conversion characteristics of the imaging unit 101. Specifically, as the exposure time becomes shorter, the linear logarithmic characteristic, which is a photoelectric conversion characteristic, changes from 401 → 402 → 403. In the linear logarithmic characteristics 401 to 403, when the linear region is 401_1 to 403_1, the slope of the linear region becomes gentle in the order of 401_1 → 402_1 → 403_1, and the sensitivity decreases. The light quantity range increases compared to the logarithmic area light quantity range, and the number of pixels in the linear region increases. In the linear logarithmic characteristics 401 to 403, when the logarithmic areas are 401_2 to 403_2, the slopes of the logarithmic areas 401_2 to 403_2 are all constant.

  From this, it can be seen that when the exposure time is shortened, the sensitivity of the linear region decreases, but the light amount range of the linear region increases and the number of pixels in the linear region increases. On the other hand, when the exposure time is increased, the sensitivity in the linear region increases, but the light amount range in the linear region decreases, and the number of pixels in the linear region decreases. Therefore, the number of pixels in the linear region having a linear logarithmic characteristic can be adjusted according to the exposure time.

  When the readout time is started as described above, φRST = Lo, and when the readout time ends, φRST = Hi, and the exposure time is started. That is, φRST = Hi in the exposure time. Therefore, for example, the exposure time can be adjusted by adjusting the time of φRST = Hi.

  The pixel value (inflection point pixel value A) of the inflection point Pt shown in FIG. 2B can be controlled by setting the transfer signal φTX in the exposure time. That is, as the transfer signal φTX approaches Hi, the opening degree of the gate of the transfer transistor Q1 increases, so that the inflection point pixel value A decreases, the area A1 decreases, and the dynamic range increases. On the other hand, when the transfer signal φTX approaches Lo, the opening degree of the gate of the transfer transistor Q1 decreases, so that the inflection point pixel value A increases, the area A1 increases, and the dynamic range decreases. Therefore, the inflection point pixel value A can be adjusted by adjusting the voltage level of the transfer signal φTX.

  In order to realize this, for example, the control unit 102 has in advance a function or LUT that prescribes the relationship between the exposure time and the voltage level of the transfer signal φTX for making the level of the inflection point pixel value A constant. In other words, the voltage level of the transfer signal may be determined using this function or LUT.

  FIG. 5 shows the photoelectric conversion characteristics of the pixel 31a when only the exposure time is adjusted and the inflection point pixel value is not adjusted, and the vertical and horizontal axes are the same as those in FIG. As shown in FIG. 5, as the exposure time becomes shorter, the linear logarithmic characteristic changes from 401 → 402 → 403. However, in FIG. 5, since the inflection point pixel value is not adjusted, the inflection point pixel value increases from A1 to A2 to A3 as the linear logarithmic characteristic changes from 401 to 402 to 403. From this, it is understood that when the inflection point pixel value is not adjusted, the inflection point pixel value increases as the exposure time becomes shorter. Therefore, when the inflection point pixel value is not adjusted, the logarithmic area decreases as the exposure time becomes shorter.

  On the other hand, in the example of FIG. 4, the inflection point Pt is controlled so that the inflection point pixel value A is constant regardless of the exposure time. Thereby, the logarithmic region is maintained even if the exposure time is shortened. However, when the control of FIG. 5 is employed, it is not necessary to control the inflection point pixel value, so that the control can be simplified. Therefore, in the present embodiment, as shown in FIG. 4, it is possible to adopt a control for making the inflection point constant regardless of the exposure time, or as shown in FIG. You may employ | adopt control which does not adjust a music point.

  FIG. 6 is a block diagram showing a detailed configuration of the image processing unit 103 shown in FIG. The image processing unit 103 includes a characteristic conversion unit 601, a luminance distribution detection unit 602, a sensitivity amplification unit 603, and a DR (dynamic range) compression unit 604 (an example of an input / output characteristic conversion unit).

  The characteristic conversion unit 601 receives image data from the imaging unit 101, converts the photoelectric conversion characteristic of the input image data from a linear logarithmic characteristic to a logarithmic characteristic (an example of a reference photoelectric conversion characteristic), and sets an inflection point Pt as a boundary. The photoelectric conversion characteristics to be switched to are unified into one photoelectric conversion characteristic. Note that the characteristic conversion unit 601 may convert the photoelectric conversion characteristic of the input image data from a linear logarithmic characteristic to a linear characteristic to unify the linear characteristic.

  The luminance distribution detection unit 602 detects the luminance distribution of the image data output from the characteristic conversion unit 601 and outputs the luminance distribution to the control unit 102 as evaluation data. The sensitivity amplification unit 603 amplifies the sensitivity of the image data output from the characteristic conversion unit 601 using the control data DA indicating the sensitivity amplification factor output from the control unit 102. Here, as the sensitivity amplification factor, a value for making the brightness of the image data constant is calculated.

  The DR compression unit 604 executes dynamic range compression processing on the image data output from the sensitivity amplification unit 603 based on the control data DB from the control unit 102.

  FIG. 7 is a block diagram showing a detailed configuration of the characteristic conversion unit 601. The characteristic conversion unit 601 includes an LUT 701. FIGS. 8A and 8B are graphs for explaining the processing of the characteristic conversion unit 601. FIG. 8A shows a case where a linear logarithmic characteristic is converted into a logarithmic characteristic, and FIG. 8B shows a case where the linear logarithmic characteristic is converted into a linear characteristic. Is shown. 8A and 8B, the vertical axis indicates the pixel value input to the characteristic conversion unit 601, and the horizontal axis indicates the amount of light in logarithm.

  The characteristic conversion unit 601 receives linear logarithmic characteristic image data. The LUT 701 stores in advance the logarithmic characteristic pixel value after conversion corresponding to the pixel value of the linear logarithmic characteristic before conversion. Therefore, as shown in FIG. 8A, the linear logarithmic characteristic image data input to the characteristic converting unit 601 is converted into image data having a logarithmic characteristic in the entire light amount range by the LUT 701.

  When the characteristic conversion unit 601 converts linear logarithmic characteristic image data into linear characteristic image data, the LUT 701 stores the converted linear characteristic pixel value in advance according to the linear logarithmic characteristic pixel value before conversion. Just remember. As a result, as shown in FIG. 8B, the linear logarithmic characteristic image data input to the characteristic conversion unit 601 is converted into image data having a linear characteristic in the entire light amount range by the LUT 701.

  FIG. 9A is a block diagram illustrating a detailed configuration of the luminance distribution detection unit 602, and FIG. 9B is a graph illustrating an example of the luminance distribution detected by the luminance distribution detection unit 602.

  The luminance distribution detection unit 602 includes a histogram detection unit 901. The histogram detection unit 901 receives image data from the characteristic conversion unit 601, detects a histogram of the image data, and calculates evaluation data. The histogram detection unit 901 obtains the frequency of the pixel value (luminance value) of each pixel in the input image data, and obtains a luminance distribution as shown in FIG. Here, the frequency represents the number of pixels having the same pixel value or the number of pixels having the same pixel value. Therefore, the histogram detection unit 901 counts the number of pixels having the same pixel value in the input image data, or classifies each pixel into a plurality of classes according to the pixel value, and the number of pixels in each class. The histogram is detected by counting. Thereby, as shown in FIG. 9B, a luminance distribution indicating the relationship between the pixel value (luminance value) and the frequency is obtained. Then, the histogram detection unit 901 outputs the obtained luminance distribution to the control unit 102 as evaluation data.

  FIG. 10A is a block diagram illustrating a detailed configuration of the sensitivity amplifying unit 603, and FIG. 10B is a graph illustrating processing by the sensitivity amplifying unit 603. In FIG. 10B, the vertical axis indicates the pixel value of the image data input to the sensitivity amplifier 603, and the horizontal axis indicates the amount of light in logarithm.

  The sensitivity amplification unit 603 includes an addition / subtraction unit 1001. The addition / subtraction unit 1001 receives the sensitivity amplification factor of the image data from the control unit 102 as the control data DA, and adds the sensitivity amplification factor to the input image data or subtracts it from the image data, thereby the sensitivity of the image data. Adjust.

  As shown in FIG. 10B, the sensitivity of the image data increases when the sensitivity amplification factor is added to the image data, and the sensitivity of the image data decreases when the sensitivity amplification factor is subtracted from the image data.

  Here, the sensitivity of the image data is adjusted by adding or subtracting the sensitivity amplification factor because the image data input to the sensitivity amplification unit 603 is logarithmic. When linear characteristic image data is input to the sensitivity amplification unit 603, the sensitivity amplification unit 603 adjusts the sensitivity by multiplying the image data by the sensitivity amplification factor.

  FIG. 11 is a block diagram illustrating a detailed configuration of the DR compression unit 604. The DR compression unit 604 includes an illumination component extraction unit 1201, a compression unit 1202, a reflectance component extraction unit 1203, and a synthesis unit 1204. 12A shows an input image D1201 input to the illumination component extraction unit 1201, and FIG. 12B shows an illumination component image D1202 output from the illumination component extraction unit 1201 and a compressed image output from the compression unit 1202. D1204, FIG. 12C shows the characteristics of the reflectance component image D1203 output from the reflectance component extraction unit 1203, and FIG. 12D shows the characteristics of the output image D1205 output from the synthesis unit 1204. In FIGS. 12A to 12D, the vertical axis indicates the pixel value, and the horizontal axis indicates the pixel position, that is, the pixel address.

  The illumination component extraction unit 1201 extracts an illumination component image D1202 that is a low-frequency image from the input image D1201 using a low-frequency extraction filter. As a result, as shown in FIG. 12B, an illumination component image D1202 from which high-frequency components have been removed from the input image D1201 is extracted.

  The compression unit 1202 compresses the illumination component image D1202 based on the compression characteristics input as the control data DB, and generates a compressed image D1204. Thereby, as shown in FIG. 12B, a compressed image D1204 in which the dynamic range of the illumination component image D1202 is compressed is obtained.

  The reflectance component extraction unit 1203 extracts the reflectance component image D1203 based on the illumination component image D1202 and the input image D1201.

  Here, when the input image D1201 has logarithmic characteristics, the reflectance component image D1203 is calculated by the following equation.

  Reflectance component image D1203 = input image D1201-illumination component image D1202

  On the other hand, when the input image D1201 has a linear characteristic, the reflectance component image D1203 is calculated by the following equation.

  Reflectance component image D1203 = input image D1201 / illumination component image D1202

  Thereby, as shown in FIG.12 (c), the reflectance component image D1203 which is a high frequency component contained in the input image D1201 is extracted.

  The combining unit 1204 combines the compressed image D1204 and the reflectance component image D1203 to generate an output image D1205. As a result, the image data subjected to the DR compression process is output from the DR compression unit 604.

  When the input image D1201 has a linear characteristic, the output image D1205 is calculated by the following equation.

  Output image D1205 × compressed image D1204 = reflectance component image D1203

  When the input image D1201 has logarithmic characteristics, the output image D1205 is calculated by the following equation.

  Output image D1205 + compressed image D1204 = reflectance component image D1203

  As a result, as shown in FIG. 12D, an output image D1205 in which the reflectance component image D1203 is superimposed on the compressed image D1204 is generated.

  FIG. 13 is a graph showing the relationship between the exposure time and the photoelectric conversion characteristics, where the vertical axis indicates the pixel value and the horizontal axis indicates the amount of light in logarithm.

  14A and 14B are explanatory diagrams of the relationship between the exposure time and the luminance distribution. FIG. 14A is the luminance distribution of the photoelectric conversion characteristic 1301 shown in FIG. 13, and FIG. 14B is the luminance of the photoelectric conversion characteristic 1302 shown in FIG. Distribution.

  As shown in FIG. 14A, the luminance distribution in the photoelectric conversion characteristic 1301 when the exposure time is long is applied to the entire image data of the pixel on the low luminance side with respect to the inflection point pixel value A, that is, the pixel having the linear characteristic. The proportion occupied is reduced compared to the case of the photoelectric conversion characteristics 1302.

  On the other hand, as shown in FIG. 14B, the luminance distribution in the photoelectric conversion characteristic 1302 when the exposure time is short is, as shown in FIG. 14B, image data of pixels on the low luminance side with respect to the inflection point pixel value A, that is, pixels with linear characteristics. The proportion of the total is increased compared to the case of the photoelectric conversion characteristic 1301.

  Therefore, when the exposure time is decreased, the ratio of the pixels having the inflection point pixel value A or less to the entire image data can be increased. On the other hand, when the exposure time is increased, it is possible to reduce the ratio of pixels having the inflection point pixel value A or less to the entire image data.

  Therefore, by controlling the exposure time, the ratio of pixels having linear characteristics to the entire image data can be made constant, and high-sensitivity image data can be obtained.

  FIG. 15 is a flowchart showing processing when the imaging apparatus according to Embodiment 1 of the present invention controls the exposure time. First, the control unit 102 acquires the luminance distribution of the image data output as the evaluation data from the luminance distribution detection unit 602, and counts the cumulative frequency of pixels having an inflection point pixel value A or less in this luminance distribution (S1501). .

  Next, the control unit 102 determines whether the cumulative frequency has a value within a predetermined frequency allowable range of the lower limit value Th1 and the upper limit value Th2 (S1502). Here, as the frequency allowable range, a range that makes the sensitivity of the image data high can be adopted. For example, as the lower limit Th1, a number of pixels that is 85% of the number of pixels of the entire image data is adopted, As the upper limit value Th2, the number of pixels that is 95% of the total number of pixels of the image data can be employed. However, the lower limit Th1 and the upper limit Th2 are not limited to these values, and other values may be adopted as appropriate.

  Next, when the cumulative frequency is within the frequency allowable range (YES in S1502), that is, when the cumulative frequency has a value that is greater than 85% and less than 95% of the total number of pixels of the image data, Since the sensitivity of the image data to be processed is within the allowable range, the processing is terminated without adjusting the exposure time.

  If the cumulative frequency is less than or equal to the lower limit Th1 (NO in S1502 and YES in S1503), that is, if the cumulative frequency is 85% or less of the total number of pixels of the image data, the control unit 102 shortens the exposure time (S1506). As a result, as shown in FIG. 14B, the ratio of pixels with the inflection point pixel value A or less to the entire image data increases, and the ratio of pixels with linear characteristics increases.

  On the other hand, when the cumulative frequency is equal to or greater than the upper limit value Th2 (NO in S1502 and NO in S1503), the control unit 102 increases the exposure time (S1505). As a result, as shown in FIG. 14A, the ratio of pixels with the inflection point pixel value A or less to the entire image data decreases, and the ratio of pixels with linear characteristics decreases.

  Here, when the imaging unit 101 captures a moving image, the control unit 102 gradually increases or decreases the exposure time with a predetermined resolution every time one frame of image data is captured, and the inflection occurs after one or several frames. A cumulative frequency equal to or less than the point pixel value A may be within the frequency allowable range. On the other hand, when the imaging unit 101 captures a still image, the control unit 102 obtains a cumulative frequency equal to or less than the inflection point pixel value A from the luminance distribution obtained by pre-exposure, and determines the difference between the cumulative frequency and the frequency allowable range. Accordingly, the current exposure time may be increased / decreased by a predetermined increase / decrease value, and imaging of the main exposure may be performed.

  Next, the control unit 102 adjusts the inflection point pixel value A in order to return the inflection point pixel value A changed by adjusting the exposure time to the inflection point pixel value A before adjustment (S1507). In this case, in order to make the inflection point pixel value A constant, the control unit 102 sets a voltage level of the transfer signal φTX determined in advance according to the exposure time, and supplies it to the gate of the transfer transistor Q1. Thus, the inflection point pixel value A can be prevented from changing before and after the exposure time adjustment, and the logarithmic region can be kept constant.

  In FIG. 15, the inflection point pixel value A is adjusted, but it is not necessary to adjust it. FIG. 16 is a graph showing photoelectric conversion characteristics when the inflection point pixel value is not adjusted, and the vertical and horizontal axes are the same as those in FIG.

  It is assumed that the exposure time is adjusted to be short and the photoelectric conversion characteristic 1601 is shifted to the photoelectric conversion characteristic 1602. When an aspect in which the inflection point pixel value A is not adjusted is employed, the inflection point pixel value varies depending on the exposure time. In this case, the control unit 102 notifies the image processing unit 103 of the inflection point pixel value AA after the change, causes the image processing unit 103 to recognize the inflection point pixel value AA, and executes various image processing. Good.

<Imaging control according to imaging scene>
Note that the control unit 102 may determine an imaging scene such as nighttime or daytime from the luminance distribution, and change the control of the imaging unit 101 according to the imaging scene.

  FIG. 20 is a diagram for explaining the difference in light quantity distribution between nighttime and daytime. (A) shows the light quantity distribution at night, and (b) shows the light quantity distribution during daytime. FIG. 21 is a graph showing the photoelectric conversion characteristics of image data, and the vertical and horizontal axes are the same as those in FIG.

  In outdoor shooting, power tends to concentrate at two locations on the dark side and the bright side near the light source at night. On the other hand, in the daytime scene, the frequency in the intermediate range is relatively large, and the change in the frequency tends to be small from the dark part to the bright part.

  Therefore, the same problems may occur in the daytime scene and the nighttime scene, that is, when the inflection point light amount a is set at the same cumulative frequency from the light amount 0 and the following inconveniences occur. There is.

  That is, there occurs a case where the light amount range Δd from the light amount a at the inflection point to the maximum light amounts d1 and d2 is greatly different between nighttime and daytime. That is, at night, the power is concentrated on the dark side, so that the light amount a at the inflection point is set lower than in the daytime. As a result, at night, the light amount range Δd from the light amount a at the inflection point to the maximum light amount d1 increases compared to the daytime.

  On the other hand, since the frequency in the middle region is relatively large during the daytime, the light amount a at the inflection point is set higher than at night. As a result, the light amount range Δd from the light amount a at the inflection point to the maximum light amount d2 is reduced in the daytime compared to that at night. Therefore, if the frequency allowable range is set based on the daytime light amount distribution, the maximum light amount d1 may exceed the saturation light amount m of the imaging unit 101 at night.

  In order to avoid the above problems, the following two measures are adopted in the present embodiment.

  First strategy: The exposure time is set by changing the frequency tolerance between daytime and nighttime.

  When the first measure is adopted, the control is complicated, but at the same time as capturing an image with a high S / N ratio, the image is not saturated and the occurrence of overexposure can be prevented.

  Second Measure: A frequency allowable range is set so that the maximum light amount d1 is equal to or less than the saturated light amount m on the basis of the nighttime light amount distribution, and the frequency allowable range is not changed between nighttime and daytime using the determined frequency allowable range. The exposure time is controlled. In the second measure, since the frequency allowable range is set based on the light distribution at night, the subject is imaged in the daytime with a wider dynamic range than the imaging scene. As a result, image saturation during both daytime and nighttime can be prevented.

  When the second measure is adopted, although control is simple, an image with a relatively high SN can be taken without always being saturated.

  When the first measure is adopted, for example, the imaging unit 101 is made to capture typical daytime and nighttime imaging scenes in advance, and the light quantity distribution at daytime and nighttime is measured. In each daytime and nighttime light amount distribution, the light amount a is set so that the maximum light amounts d2 and d1 are equal to or less than the saturated light amount m and as close as possible to the maximum light amounts d2 and d1, and the accumulated light amount a or less. The lower limit value and the upper limit value of the frequency allowable range may be set based on the frequency. Further, when the second policy is adopted, the frequency allowable range set in the night-time imaging scene in the first policy may be adopted.

<Imaging scene discrimination processing>
FIG. 22 is an explanatory diagram of the imaging scene discrimination process, where (a) shows the luminance distribution of the image data in one imaging scene, and (b) shows the luminance distribution of the image data in another imaging scene. FIG. 23 is a flowchart of an imaging scene discrimination process.

  First, the control unit 102 acquires the luminance distribution (evaluation data) detected by the luminance distribution detection unit 602, and detects a peak from the acquired luminance distribution (S2301). Here, as a method for detecting a peak, a method may be employed in which a luminance distribution is differentiated and a portion that is convex upward and has a differential value of 0 is detected as a peak. In FIG. 22A, a peak PK1 having a pixel value P1 on the dark side and a peak PK2 having a pixel value P2 on the bright side are detected. On the other hand, in FIG. 22B, a peak PK1 having a pixel value P1 on the dark side, a peak PK3 having a pixel value P3 on the bright side, and a peak PK2 having a pixel value P2 on the intermediate area are detected.

  Next, when two or more peaks are detected (YES in S2302), the control unit 102 detects a peak interval that is a pixel value interval between peaks. In this case, the peak interval is detected for each of all sets of the plurality of peaks of the control unit 102. Since two peaks are detected in FIG. 22A, the peak interval between the peak PK1 and the peak PK2 is detected. On the other hand, in FIG. 22B, since three peaks are detected, three peak intervals of peak intervals S12, S23, and S13, which are peak intervals of all combinations of the peaks PK1 to PK3, are detected.

  Next, when the peak interval is equal to or greater than the predetermined threshold Ths (YES in S2304), the control unit 102 determines that the imaging scene is night (S2305), and when the peak interval is less than the threshold Ths (NO in S2304), the control unit 102 determines that the imaging scene is daytime (S2306).

  In the case of FIG. 22A, since one peak interval S is detected, if the peak interval S is equal to or greater than the threshold Ths, it is determined that the night. In the case of FIG. 22B, since three peak intervals S12, S13, and S23 are detected, it is determined whether or not each peak interval is equal to or greater than the threshold Ths, and one, two, or all If the peak interval is greater than or equal to the threshold Ths, it is determined that the night. When a plurality of peak intervals are detected, the control unit 102 may extract the maximum peak interval, and if the maximum peak interval is equal to or greater than the threshold Ths, the control unit 102 may determine the night and simplify the processing.

  On the other hand, when two or more peaks are not detected (NO in S2302), the control unit 102 determines that the imaging scene is daytime (S2306).

  Thus, at night, there is a tendency for a plurality of peaks to appear on the dark side and on the bright side, while in the daytime, one peak tends to appear in the intermediate region. By detecting, nighttime and daytime can be accurately determined.

<First policy>
Next, the first policy described above will be described. FIG. 24A shows an example of the luminance distribution at night, and FIG. 24B shows an example of the luminance distribution at daytime. FIG. 25 is a flowchart showing processing when controlling the exposure time when the imaging apparatus according to Embodiment 1 of the present invention adopts the first strategy.

  S2501 is the same as S1501 in FIG. In step S2502, the control unit 102 determines an imaging scene from the luminance distribution using the imaging scene determination process described above. When determining that the imaging scene is nighttime, the control unit 102 sets predetermined nighttime values as the lower limit value Th1 and the upper limit value Th2 (S2503), and performs the processing after S2504. Here, the processing after S2504 is the same as the processing after S2502 in FIG.

  On the other hand, when determining that the imaging scene is daytime, the control unit 102 sets predetermined daytime values as the lower limit value Th1 and the upper limit value Th2 (S2503), and performs the processing after S2504.

  Here, the lower limit value Th1 and the upper limit value Th2 of the nighttime and daytime are set as follows.

Nighttime: Th1 = N1, Th2 = N2
Daytime: Th1 = D1, Th2 = D2

  Then, N1, N2, D1, and D2 have the following relationship.

  D1 <N1, D2 <N2

  That is, the night frequency tolerance range is set to be larger than the daytime frequency tolerance range. Accordingly, as shown in FIGS. 24A and 24B, the inflection point pixel value A1 at night is higher than the inflection point pixel value A2 at daytime, and saturation on the high luminance side, which is a concern at night, is saturated. Can be suppressed.

<Calculation of sensitivity gain>
Next, the sensitivity amplification factor calculation process will be described. FIG. 26 is a graph showing the photoelectric conversion characteristics of image data converted into logarithmic characteristics, where the vertical axis indicates pixel values and the horizontal axis indicates the amount of light in logarithm. FIG. 27A shows the luminance distribution of the photoelectric conversion characteristic 2601 shown in FIG. 26, and FIG. 27B shows the luminance distribution of the photoelectric conversion characteristic 2602 shown in FIG.

  In the present embodiment, as described above, the exposure time is controlled so that the ratio of the linear characteristic pixels to the entire image data is within a certain range in the luminance distribution of the image data. However, if priority is given only to this, the overall brightness of the image data may change significantly. Therefore, in the present embodiment, the sensitivity of the image data is amplified so that the overall brightness of the image data is constant.

  As shown in FIG. 26, when the sensitivity amplification factor increases, the photoelectric conversion characteristics change from 2602 to 2601. Therefore, by increasing the sensitivity amplification factor, the average value B of the pixel values of the inflection point pixel value A or less can be increased as shown in FIG.

  On the other hand, when the sensitivity amplification factor decreases, the photoelectric conversion characteristics change from 2601 to 2602. Therefore, by reducing the sensitivity amplification factor, the average value B of the pixel values of the inflection point pixel value A or less can be increased as shown in FIG.

  FIG. 28 is a flowchart of sensitivity control. First, the control unit 102 obtains an average value of luminances of pixels having an inflection point pixel value A or less from the luminance distribution detected by the luminance distribution detection unit 602 (S2801). Next, the control unit 102 determines whether the average value is larger than the lower limit value Th3 of the target luminance range and less than the upper limit value Th4 of the target luminance range (S2802). If the average value is equal to or lower than the lower limit Th3 (NO in S2802 and YES in S2803), the control unit 102 increases the sensitivity amplification factor (S2805). On the other hand, if the average value is equal to or greater than the upper limit value Th4 (NO in S2802 and NO in S2803), the sensitivity amplification factor is decreased (S2804). The control unit 102 supplies the sensitivity amplification factor calculated in S2804 and S2805 to the sensitivity amplification unit 603 as control data DA, and the sensitivity amplification unit 603 increases the sensitivity of the image data with the sensitivity amplification factor that is the control data DA. Amplify.

  Here, when the imaging unit 101 captures a moving image, the control unit 102 gradually increases or decreases the sensitivity amplification factor with a predetermined resolution every time one frame of image data is captured, and averages after one or several frames. The value may be set to the target luminance range. On the other hand, when the imaging unit 101 captures a still image, the control unit 102 increases or decreases the current sensitivity gain by a predetermined increase / decrease value according to the difference between the average value obtained by the pre-exposure and the target luminance range. The sensitivity of the image data for the main exposure may be amplified.

<DR compression processing>
Next, details of the DR compression processing will be described. FIG. 29 is a graph illustrating compression characteristics, where the vertical axis represents the image data of the illumination component image D1202 input to the compression unit 1202, and the horizontal axis represents the image data of the compressed image D1204 output from the compression unit 1202. ing.

  FIG. 30 shows the luminance distribution detected by the luminance distribution detection unit 602. First, the control unit 102 detects the average value B and the maximum pixel value C from the luminance distribution detected by the luminance distribution detection unit 602. Here, the average value B is the average value of the luminance of the pixels having the inflection point pixel value A or less, and the maximum pixel value C is the maximum pixel value of the image data. Further, the inflection point pixel value A is known.

  Next, the control unit 102 changes the detected inflection point pixel value A, the average value B, and the maximum pixel value C with the sensitivity amplification factor g, so that the values that become predetermined targets are changed. The compression characteristic f is obtained so as to obtain the curve pixel value A ′, the average value B ′, and the maximum pixel value C ′. Here, for example, on the graph shown in FIG. 29, the control unit 102 amplifies the inflection point pixel value A, the average value B, and the maximum pixel value C with the sensitivity amplification factor g, and the inflection point pixel value. The compression characteristics f may be obtained by plotting the intersection points CR1 to CR3 with A ′, the average value B ′, and the maximum pixel value C ′ and linearly interpolating the intersection points CR1 to CR3 with a straight line.

  Then, the control unit 102 supplies the calculated compression characteristic f to the DR compression unit 604 as a control data DB. Note that the sensitivity amplification factor g is a sensitivity amplification factor calculated as control data DA by the control unit 102. Here, the reason why the sensitivity-amplified value g is used is that the image data input to the DR compression unit 604 is image data after sensitivity amplification processing.

  When the characteristic conversion unit 601 converts to a logarithmic characteristic, the compression characteristic f satisfies the following condition.

A '= f (A + g)
B ′ = f (B + g)
C ′ = f (C + g)

  When the characteristic conversion unit 601 converts to a linear characteristic, the compression characteristic f satisfies the following condition.

A ′ = f (A × g)
B ′ = f (B × g)
C ′ = f (C × g)

  In this way, by obtaining the compression characteristic f using the average value B and the maximum pixel value C as control points, the average value and the maximum pixel value of the image data can always be at the same level. Further, by adding the inflection point pixel value A as a control point to obtain the compression characteristic f, it is possible to output the linear region having a high S / N ratio and the logarithmic region having a high dynamic range with different optimum characteristics. It becomes possible. In the above description, the compression characteristic f is obtained in consideration of the inflection point pixel value A as the control point. However, the compression characteristic f may be obtained without considering the inflection point pixel value A.

  As described above, according to the imaging apparatus according to the present embodiment, the exposure time and the inflection point pixel value are controlled according to the luminance distribution of the photographic scene. Ratio image data can be obtained.

  In addition, since the sensitivity amplification factor is calculated so that the average value of the luminance of pixels below the inflection point pixel value falls within the target luminance range and the sensitivity of the image data is amplified, the output level is stable regardless of the imaging scene. Image data can be obtained.

  Further, since the compression characteristic is calculated based on a predetermined control point of the luminance distribution and the sensitivity amplification factor, it is possible to obtain image data with a high saturation ratio, a stable output level, and low saturation. .

(Embodiment 2)
Next, an imaging apparatus according to Embodiment 2 of the present invention will be described. In this embodiment, the pixel configuration is the same as that of the first embodiment, and a subject is imaged a plurality of times with different exposure times during one frame period, and image data at each exposure time is output. The subject is imaged with the photoelectric conversion characteristics. In addition, in this Embodiment, the same thing as Embodiment 1 attaches | subjects the same code | symbol, and abbreviate | omits description.

  FIG. 3 is a graph showing the photoelectric conversion characteristics of the pixel 31a according to the second embodiment of the present invention. The vertical axis indicates the pixel value linearly, and the horizontal axis indicates the light quantity linearly.

  In the example of FIG. 3, three exposure times are set during one frame period, and three types of image data are captured. Note that the photoelectric conversion characteristics of these three image data are all linear characteristics. The photoelectric conversion characteristics of these three pieces of image data have a gentle slope in the order of 301 → 302 → 303, and the sensitivity is reduced. This is because the exposure time is set shorter in the order of 301 → 302 → 303.

  FIG. 17A shows the photoelectric conversion characteristics of the respective image data when the first and second exposure times are set during one frame period and the first and second image data are acquired. FIG. 17B shows the photoelectric conversion characteristics of the combined image data obtained by combining the first and second image data shown in FIG. 17A and 17B, the vertical axis and the horizontal axis are the same as those in FIG.

  In multiple exposure, imaging is performed at two or more different exposure times in the imaging unit 101, and image data with a high dynamic range is captured by combining the captured image data.

  In the example of FIG. 17, imaging is performed twice in the first and second exposure times, and the second exposure time is set shorter than the first exposure time.

  The first image data and the second image data are combined, for example, in the area below a predetermined inflection point pixel value A17, the first image data is directly used as the composite image data, and in the area larger than the inflection point pixel value A17. The image data obtained by adding the offset level SS to the two image data is used as the composite image data. Here, as the offset level SS, a difference between the first image data at the light amount a17 and the second image data at the light amount a17 can be adopted.

  FIG. 33 is a block diagram of the image processing unit 103 according to the second embodiment of the present invention. In this embodiment, the image processing unit 103 includes a synthesis unit 3301 instead of the characteristic conversion unit 601. The synthesizing unit 3301 performs the above synthesizing process. The processing after the synthesis processing is the same as that in the first embodiment, and the inflection point pixel value A17 corresponds to the inflection point pixel value A in the first embodiment. That is, the luminance distribution detection unit 602 detects the luminance distribution in the composite image data and supplies it to the control unit 102 as evaluation data. The control unit 102 detects the cumulative frequency of pixels having an inflection point pixel value A17 or less, and controls the exposure time so that the cumulative frequency falls within a predetermined frequency allowable range.

  FIG. 19 is a flowchart showing processing when the imaging apparatus according to Embodiment 2 of the present invention controls the exposure time. S1901 to S1903 are the same as S1501 to S1503 in FIG.

  When the cumulative frequency of the pixels having the inflection point pixel value A17 or less is equal to or less than the lower limit value Th1 (YES in S1903), the control unit 102 shortens the first exposure time (S1905). As a result, the sensitivity of the first image data is reduced, and the cumulative frequency of pixels having an inflection point pixel value A17 or less is increased.

  On the other hand, when the cumulative frequency of pixels having the inflection point pixel value A17 or less is equal to or greater than the upper limit value Th2 (NO in S1903), the control unit 102 increases the first exposure time (S1904). As a result, the sensitivity of the first image data is increased, and the cumulative frequency of pixels having an inflection point pixel value A17 or less is decreased.

  Next, the control unit 102 controls the second exposure time in conjunction with the control of the first exposure time (S1906). In this case, for example, when the first exposure time is set to 1.5 times, the control unit 102 may set the second exposure time to 1.5 times.

  FIG. 18 is a graph illustrating how the exposure times of the first and second image data are controlled, and the vertical and horizontal axes are the same as those in FIG. For example, it is assumed that the first exposure time is increased α times and the photoelectric conversion characteristic of the first image data changes from 1801 to 1801 ′. Then, the second exposure time is also multiplied by α, and the photoelectric conversion characteristic of the second image data changes from 1802 to 1802 ′.

  On the other hand, it is assumed that the first exposure time is shortened to 1 / α times, and the photoelectric conversion characteristic of the first image data changes from 1801 ′ to 1801. Then, the second exposure time is also shortened to 1 / α times, and the photoelectric conversion characteristic of the second image data changes from 1802 ′ to 1802. Thereby, it is possible to obtain image data with a high S / N ratio while maintaining the dynamic range.

  As described above, according to the imaging apparatus of the second embodiment, even when the multiple exposure method is adopted, the same effect as that of the first embodiment can be obtained.

  In the above description, the case where two image data are captured as multiple exposure has been described as an example. However, the present invention is not limited to this, and three or more image data may be captured. In this case, the image data having the longest exposure time is set as the first image data, and a plurality of other image data is set as the second image data, and these image data may be combined into one piece of image data.

(Modification 1)
FIG. 31 is a block diagram illustrating a modification of the image processing unit 103 according to the first and second embodiments. In the first and second embodiments, the luminance distribution detection unit 602 is provided between the characteristic conversion unit 601 (synthesis unit 3301) and the sensitivity amplification unit 603. However, as shown in FIG. You may provide between the amplification part 603 and DR compression part 604. FIG. In this case, the sensitivity amplification unit 603 amplifies the sensitivity of the image data of the current frame using the sensitivity amplification factor (control data DA) calculated from the luminance distribution of the image data of the previous frame. Then, the luminance distribution detection unit 602 detects the luminance distribution from the image data whose sensitivity has been amplified by the sensitivity amplification unit 603.

  Further, as shown in FIG. 31B, a luminance distribution detection unit 6021 is provided between the characteristic conversion unit 601 (synthesis unit 3301) and the sensitivity amplification unit 603, and the sensitivity amplification unit 603 and the DR compression unit 604 are provided. A luminance distribution detector 6022 may be provided between the two. In this case, the control unit 102 calculates the exposure time and the sensitivity amplification factor (control data DA) using the luminance distribution detected by the luminance distribution detection unit 6021, and sends the sensitivity of the image data of the current frame to the sensitivity amplification unit 603. May be amplified at the calculated sensitivity amplification factor.

  Further, the control unit 102 obtains a compression characteristic f (control data DB) using the luminance distribution detected by the luminance distribution detection unit 6022, and causes the DR compression unit 604 to perform compression processing on the image data of the current frame. That's fine. In this case, as described in FIG. 29, the control unit 102 detects the average value B and the maximum pixel value C from the luminance distribution supplied from the luminance distribution detection unit 6022 without using the sensitivity amplification factor g. The detected average value B, maximum pixel value C, inflection point pixel value A, target average value B ′, and maximum pixel value C ′ are plotted on a graph shown in FIG. 29 and compressed by linear interpolation. The characteristic f may be calculated.

(Modification 2)
In the first and second embodiments, the DR compression unit 604 is provided in the image processing unit 103 as an example of the input / output characteristic conversion unit. The modification 2 is characterized in that a gamma correction unit is employed instead of the DR compression unit 604 as the input / output characteristic unit.

  FIG. 32 is a graph showing the gradation characteristic h used by the gamma correction unit for gamma correction, and the details are the same as FIG.

  The gamma correction unit adjusts the input / output characteristics of the input image data so as to match the display characteristics of the display, but the basic concept is the same as that of the DR compression unit 604. That is, the gamma correction unit is supplied with the gradation characteristic h from the control unit 102 as the control data DB, and converts the input / output characteristics of the input image data according to the gradation characteristic h.

  Here, similarly to the DR compression unit 604, the control unit 102 targets the inflection point pixel value A, the average value B of the pixel values below the inflection point pixel value A, and the maximum pixel value C in the luminance distribution. The gradation characteristics are obtained so that the inflection point pixel value A ′, the average value B ′, and the maximum pixel value C ′ are obtained. Here, when the control unit 102 obtains the gradation characteristic h, the target inflection point pixel value A ′, average value B ′, and maximum pixel value C ′ are values suitable for obtaining the gradation characteristic h. What should be adopted.

DESCRIPTION OF SYMBOLS 101 Image pick-up part 102 Control part 103 Image processing part 601 Characteristic conversion part 602 Luminance distribution detection part 603 Sensitivity amplification part 604 DR compression part 901 Histogram detection part 1001 Addition / subtraction part 1201 Illumination component extraction part 1202 Compression part 1203 Reflectance component extraction part 1204 Composition Part

Claims (10)

An imaging unit that images a subject with a first photoelectric conversion characteristic and a second photoelectric conversion characteristic having a lower sensitivity than the first photoelectric conversion characteristic;
A luminance distribution detector that detects a luminance distribution of an image captured by the imaging unit;
A control unit for controlling the exposure time of the imaging unit,
The control unit detects an accumulated frequency of pixels imaged by the first photoelectric conversion characteristic based on the luminance distribution, and controls the exposure time so that the accumulated frequency satisfies a predetermined frequency allowable range. apparatus. The first photoelectric conversion characteristic is a linear characteristic,
The second photoelectric conversion characteristic is a logarithmic characteristic,
A characteristic converter that converts the linear characteristic and logarithmic characteristic into a reference photoelectric conversion characteristic;
The imaging apparatus according to claim 1, wherein the luminance distribution detection unit detects a luminance distribution of an image converted into the reference photoelectric conversion characteristic.   The imaging unit according to claim 2, wherein the control unit adjusts the pixel value of the inflection point so that the pixel value of the inflection point between the linear characteristic and the logarithmic characteristic is constant before and after the control of the exposure time. apparatus. The imaging unit captures the subject by multiple exposure to obtain first and second images of the first and second photoelectric conversion characteristics;
A synthesis unit that synthesizes the first and second images so that an image having a predetermined light quantity or more becomes the second image;
The imaging apparatus according to claim 1, wherein the luminance distribution detection unit detects a luminance distribution of the image synthesized by the synthesis unit.   The imaging apparatus according to claim 4, wherein the control unit controls the exposure time of the second photoelectric conversion characteristic in conjunction with the control of the exposure time of the first photoelectric conversion characteristic. The control unit calculates a sensitivity amplification factor for making the brightness of the image converted into the reference photoelectric conversion characteristic constant based on the luminance distribution detected by the luminance distribution detection unit,
The imaging apparatus according to claim 1, further comprising a sensitivity amplification unit that amplifies the sensitivity of the image with the sensitivity amplification factor.   The control unit obtains an average value of pixel values of pixels equal to or less than the pixel value of the inflection point based on the luminance distribution, and sets the sensitivity amplification factor so that the obtained average value falls within a predetermined target luminance range. The imaging device according to claim 6. The control unit detects a predetermined control point in the luminance distribution, uses the image amplified by the sensitivity control unit with the sensitivity amplification factor as an input image, and sets a pixel value at the control point of the input image to a specified value. To obtain a conversion characteristic for converting the input / output characteristics of the input image,
The imaging apparatus according to claim 1, further comprising an input / output characteristic conversion unit that converts an input / output characteristic of the input image with the conversion characteristic.   The imaging device according to claim 8, wherein the control point includes at least an average value of pixel values of pixels equal to or less than a pixel value of an inflection point and a maximum pixel value.   The said control part determines an imaging scene based on the said brightness distribution, and when the said imaging scene is nighttime, the said frequency tolerance range is set higher than the case where an imaging scene is daytime. The imaging device described in 1.

Images