JP4720130B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus having a solid-state imaging device capable of switching a plurality of photoelectric conversion characteristics.

  Conventionally, in a solid-state imaging device that performs a linear conversion operation that linearly converts the amount of incident light, its dynamic range is as narrow as two orders of magnitude, so when imaging a subject that constitutes a luminance distribution in a wide luminance range, other than the dynamic range The luminance information in the range is not output. Further, as a conventional solid-state imaging device, there is one that performs a logarithmic conversion operation for logarithmically converting the amount of incident light (see Patent Document 1). In this solid-state imaging device, the dynamic range is as wide as 5 to 6 digits. Therefore, even if an image of a subject constituting a luminance distribution in a slightly wide luminance range is captured, all luminance information in the luminance distribution is converted into an electrical signal. Can be output. However, since the imageable area becomes wider with respect to the luminance distribution of the subject, an area without luminance data is formed in the low luminance area or the high luminance area in the imageable area. On the other hand, the present applicant has proposed what can switch between the above-described linear conversion operation and logarithmic conversion operation (see Patent Document 2).

In addition, when the linear conversion operation and the logarithmic conversion operation of the solid-state imaging device can be switched, the applicant of the present invention performs signal processing on the image signal obtained by the linear conversion operation and the image signal obtained by the logarithmic conversion operation. In addition, there is proposed an imaging apparatus that can process an image signal having one photoelectric conversion characteristic to the other photoelectric conversion characteristic so that it can be processed by a common circuit (see Patent Document 3). In such an imaging apparatus, for example, even when a solid-state imaging device that switches to an optimum photoelectric conversion characteristic for each frame is provided, all output image signals can be converted into one of the photoelectric conversion characteristics. Therefore, it is not necessary to provide a signal processing circuit corresponding to each photoelectric conversion characteristic as a subsequent signal processing circuit.
JP-A-11-313257 JP 2002-77733 A JP 2002-33961 A

  However, in the imaging apparatus disclosed in Patent Document 3, after image signals having different photoelectric conversion characteristics are always converted to image signals having the same photoelectric conversion characteristics, various signal processes such as edge enhancement and gamma correction are performed by a common circuit. Therefore, depending on each photoelectric conversion characteristic, the color tone is deteriorated or the white balance is shifted. For this reason, when the photoelectric conversion characteristics of the solid-state image sensor are switched and the dynamic range is switched, the gradation characteristics and the like are different in each photoelectric conversion characteristic. May result in a bad image.

  In view of such a problem, an object of the present invention is to provide an imaging apparatus that performs signal processing using parameters according to the photoelectric conversion characteristics when the photoelectric conversion characteristics of the solid-state imaging device are switched.

  In order to achieve the above object, an imaging apparatus of the present invention includes a solid-state imaging device having a plurality of pixels, and a signal processing circuit that processes an image signal output from the solid-state imaging device, and the solid-state imaging In an imaging apparatus that selects and switches a photoelectric conversion characteristic of an element from a plurality of photoelectric conversion characteristics, the luminance distribution of a subject is detected from an image signal output from the solid-state imaging element, and the luminance distribution of the subject is detected A photoelectric conversion characteristic switching unit that selects a photoelectric conversion characteristic from the plurality of photoelectric conversion characteristics and sets the photoelectric conversion characteristic as a photoelectric conversion characteristic of the solid-state imaging device, and a parameter for the signal processing circuit to process the image signal A plurality of types of parameters corresponding to each of the plurality of photoelectric conversion characteristics, and parameters corresponding to the selected photoelectric conversion characteristics Select from serial plurality of types of parameters and sets the parameter of the processing in the signal processing circuit.

  In such an imaging apparatus, the photoelectric conversion characteristic switching unit detects the luminance distribution of the subject from the signal level of the image signal, and switches to the photoelectric conversion characteristic according to the luminance distribution of the subject. Then, a parameter corresponding to the selected photoelectric conversion characteristic is set for the signal processing circuit. Therefore, the signal processing circuit can optimally perform signal processing on the input image signal by performing signal processing using a parameter corresponding to the photoelectric conversion characteristic.

  The signal processing circuit includes a black reference correction circuit that performs a reference value correction for converting a signal level corresponding to a minimum luminance value in an image signal output from the solid-state imaging device into a predetermined reference value, and the black reference correction circuit. However, by providing a plurality of types of parameters corresponding to the photoelectric conversion characteristics as parameters for performing the reference value correction, even if the photoelectric conversion characteristics are switched, the image signal to be supplied to the signal processing circuit The reference level can be kept constant.

  Further, at this time, the black reference correction circuit sets a signal level for the minimum luminance value that is a parameter for performing the reference value correction according to the selected photoelectric conversion characteristic, and solid-state imaging A subtractor for performing a reference value correction by subtracting a signal level for the lowest luminance value set by the parameter setting unit from the image signal output from an element, and the black reference correction circuit. However, a plurality of types of signal levels corresponding to the photoelectric conversion characteristics are provided as signal levels for the minimum luminance value for performing reference value correction.

  The signal processing circuit includes a gradation conversion circuit for converting the gradation of the image signal, and the gradation conversion circuit has a plurality of parameters corresponding to the photoelectric conversion characteristics as parameters for gradation conversion. It does not matter even if it has a kind of parameter. A parameter for increasing the gradation property may be set for the photoelectric conversion characteristics with a wide dynamic range, and a parameter for decreasing the gradation property for the photoelectric conversion characteristics with a narrow dynamic range.

  Further, the gradation conversion circuit is input in accordance with a parameter setting unit for setting a parameter for performing the gradation conversion according to the selected photoelectric conversion characteristic, and the parameter set by the parameter setting unit. And a gradation conversion unit that performs gradation conversion of the image signal.

  In addition, the signal processing circuit includes a coring circuit that removes noise components of the image signal and performs edge enhancement, and the coring circuit uses each of the photoelectric conversion characteristics as a parameter for performing noise component removal. A plurality of types of parameters may be provided, and a parameter for performing edge enhancement may be provided with a plurality of types of parameters according to the photoelectric conversion characteristics, and noise component removal may be performed. In addition, as a parameter for performing edge enhancement, a plurality of types of parameters corresponding to the respective photoelectric conversion characteristics may be provided.

  At this time, the coring circuit extracts the noise component by subtracting the low-pass filter that removes the noise component from the input image signal and the image signal that has passed through the low-pass filter from the input image signal. A coring unit that corals the noise component obtained by the subtractor to remove noise components other than the edge portion, and the edge component obtained by the coring portion has a predetermined amount. An edge-enhanced image obtained by adding the edge component multiplied by the edge enhancement amount by the multiplier to the image signal from which the noise component has been removed by the multiplier and a multiplier for multiplying the edge enhancement amount. An adder that outputs a signal, and a plurality of types of parameters that are set according to the photoelectric conversion characteristics are noise components in the low-pass filter. Matrix of the low-pass filter coefficients to remove, and coring coefficient for eliminating the noise components other than the edge portion by the coring circuit, and is one of the edge enhancement amount of said multiplier. At this time, the low-pass filter coefficient is an n × n matrix coefficient, for example, a 3 × 3 or 5 × 5 matrix coefficient is used.

  The image signal output from the solid-state imaging device is a plurality of types of color signals, and the plurality of types of color signals are generated for one pixel and applied to the signal processing circuit. However, for each type of the color signal of each pixel, a color correction circuit that performs a color correction by giving a correlation with another color signal, the color correction circuit as a parameter for performing the color correction, A plurality of types of parameters corresponding to the photoelectric conversion characteristics may be provided. This color correction is a process for emphasizing a certain hue. The solid-state imaging device may be a single-plate solid-state imaging device, and may include a color interpolation circuit that generates the plurality of types of color signals for each pixel. The solid-state image sensor may be a multi-plate solid-state image sensor.

  The color correction circuit is detected by a parameter setting unit that sets a parameter for performing the color correction according to the selected photoelectric conversion characteristic for each type of the color signal, and the luminance range detection unit. A parameter setting unit for setting a parameter for performing the color correction according to the luminance range; and the parameter set by the parameter setting unit for each of all types of color signals of the same pixel as the input color signal. And an adder that adds the color signals of all the same pixels multiplied by the parameter by the multiplier and corrects the color of the input color signal.

  In the photoelectric conversion characteristic switching unit, when it is confirmed that an image signal having a saturation level that maximizes the signal level is output from the solid-state imaging device by a number equal to or more than a first predetermined number of pixels, the dynamic range of the solid-state imaging device is expanded. The photoelectric conversion characteristics may be selected.

  Further, in the photoelectric conversion characteristic switching unit, when the maximum signal level of the image signal output from the solid-state image sensor is a value lower by a predetermined signal level than the saturation level at which the signal level is maximum, the solid-state image signal Photoelectric conversion characteristics that narrow the dynamic range of the image sensor may be selected. Furthermore, the maximum signal level of the image signal output from the solid-state image sensor may be a signal level output from a pixel having a second predetermined number of pixels or more. At this time, by setting the first predetermined number of pixels to a value smaller than the second predetermined number of pixels, it is possible to provide a hysteresis when the dynamic range is expanded and when the dynamic range is decreased.

  The time for switching the photoelectric conversion characteristics so as to expand the dynamic range may be set to be shorter than the time for switching the photoelectric conversion characteristics so as to narrow the dynamic range.

  Further, in each of the above-described solid-state imaging devices, each photoelectric conversion characteristic includes a logarithmic conversion characteristic in which the electric signal logarithmically converted with respect to the incident light amount is the image signal, and an electric signal linearly converted with respect to the incident light amount. The linear conversion characteristics used as the image signal may be switched according to the luminance. Moreover, it does not matter as what has a different photoelectric conversion characteristic for every luminance range. At this time, the photoelectric conversion characteristic of the solid-state imaging device is switched by changing the luminance value at which the characteristic is switched.

  According to the present invention, since the signal processing parameters are set according to the photoelectric conversion characteristics set for the solid-state imaging device, it is possible to perform signal processing that is optimal for the photoelectric conversion characteristics on the image signal. . In addition, since the parameter for performing the reference value correction can be set according to the set photoelectric conversion characteristic, an offset generated due to the difference in the photoelectric conversion characteristic can be removed. Further, since the parameter for the gradation characteristic can be set according to each photoelectric conversion characteristic, gradation conversion according to the dynamic range of the subject can be performed. In addition, since parameters for noise component removal or edge enhancement can be set according to each photoelectric conversion characteristic, noise component removal or edge enhancement can be performed according to the dynamic range of the subject, and the image signal has good contrast. It can be converted into a highly accurate image signal. In addition, since parameters for color correction can be set according to each photoelectric conversion characteristic, color enhancement according to the dynamic range of the subject can be performed, and the image signal can be converted into a good color. As described above, since it is possible to perform optimum signal processing for each photoelectric conversion characteristic, it is possible to obtain a highly accurate image with good contrast and high SN ratio without being affected by the dynamic range of the subject. .

<Configuration of imaging device>
A configuration of an imaging apparatus according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing the internal configuration of the imaging apparatus of the present invention.

  1 is output from an optical system 1 composed of a plurality of lenses, a solid-state image sensor 2 that converts an incident light amount of light incident through the optical system 1 into an electrical signal, and the solid-state image sensor 2. An amplifier 3 for amplifying the electric signal, an AD conversion circuit 4 for converting the electric signal amplified by the amplifier 3 into a digital signal, a black reference correction circuit 5 for setting a minimum level of the digital signal from the AD conversion circuit 4, An FPN correction circuit 6 that removes FPN (Fixed Pattern Noise) due to the sensitivity of each pixel of the solid-state imaging device 2 from the digital signal corrected by the black reference value in the black reference correction circuit 5, and the FPN correction circuit 6 An AE / WB evaluation value detection circuit 7 for detecting respective evaluation values for performing automatic exposure control (AE) and white balance (WB) from the digital signal from which FPN has been removed at Based on the WB control circuit 8 that performs correction for each color signal so that the color balance of the digital signal from which FPN is removed by the FPN correction circuit 6 is obtained, and the color signals of a plurality of adjacent pixels output from the WB control circuit 8. The color interpolation circuit 9 that interpolates each color signal, the color correction circuit 10 that corrects the hue of each color signal output from the color interpolation circuit 9 for each pixel by another color signal, and the color correction circuit 10 output the color signal. A gradation conversion circuit 11 that performs gradation conversion of a digital signal, a coring circuit 12 that performs processing such as edge enhancement on the digital signal output from the gradation conversion circuit 11, and an overall control unit 13 that controls each block An aperture control unit 14 for controlling the exposure amount by the aperture 1 a provided in the optical system 1, and a clock for supplying a clock for operation timing to the solid-state imaging device 2 and the AD conversion circuit 4. It includes a timing generating circuit 15, a.

  In the imaging apparatus configured as described above, when light is incident on the solid-state imaging device 2 having a different color filter for each pixel via the optical system 1, a photoelectric conversion operation is performed in each pixel, and each pixel An analog signal that is a different color signal is output every time. That is, as shown in FIG. 2, when a color filter having a Bayer array with RGB is provided in the solid-state imaging device 2, an R signal representing red is a color that becomes G from a pixel provided with an R color filter A G signal representing green is outputted from the pixel provided with the filter, and a B signal representing blue is outputted from the pixel provided with the color filter B. As will be described later, in the solid-state imaging device 2, the luminance position at which the linear conversion operation and the logarithmic conversion operation are switched is changed by changing the driving condition by the overall control unit 13. As a result, the dynamic range of the solid-state imaging device 2 changes. The dynamic range becomes wider as the switching point becomes the higher luminance side.

  The R signal, the G signal, and the B signal that are serially output from the solid-state imaging device 2 are amplified by the amplifier 3 and then converted into a digital signal by the AD conversion circuit 4. When the R signal, the G signal, and the B signal thus converted into digital signals are supplied to the black reference correction circuit 5, the black level that becomes the minimum luminance value based on the dynamic range control signal supplied from the overall control unit 13 is obtained. Is corrected to the reference value (0). That is, since the black level varies depending on the dynamic range of the solid-state imaging device 2, the signal level that becomes the black level is subtracted from the signal levels of the R signal, the G signal, and the B signal output from the AD conversion circuit 4. Thus, the reference value correction is performed. The operation of the black reference correction circuit 5 will be described later.

  The FPN component is removed from the R signal, the G signal, and the B signal subjected to the black reference correction by subtracting the FPN component stored in the FPN correction circuit 6. This FPN component is an offset variation caused by a threshold variation of MOS transistors constituting each pixel in the solid-state imaging device 2. When extracting the FPN component, for each of the RGB signals, an offset based on the difference in transmittance of the color filter is subtracted from the image signal for each pixel output from the solid-state imaging device 2 during uniform light irradiation. At this time, the offset based on the transmittance of each color filter is determined by the average value of each RGB signal at the time of uniform light irradiation, and each average value is subtracted from the RGB signal at the time of uniform light irradiation. The FPN component of the pixel may be extracted. The R signal, G signal, and B signal from which the FPN component has been removed in this way are supplied to the AE / WB evaluation value detection circuit 7 and the WB control circuit 8.

  The AE / WB evaluation value detection circuit 7 calculates the average value distribution range of the luminance representing the luminance range of the subject by confirming the luminance value of the image signal composed of the given R signal, G signal, and B signal, It is sent to the overall control unit 13 as an AE evaluation value for setting the exposure amount. Based on this AE evaluation value, the overall control unit 13 controls the aperture of the diaphragm 1a, whereby the exposure amount is controlled. The AE / WB evaluation value detection circuit 7 confirms the respective luminance ratios and luminance differences from the given R, G, and B signals, and calculates a WB evaluation value that is a reference value for white balance. And sent to the overall control unit 13. The WB control circuit 8 performs white balance processing based on the WB evaluation value and the dynamic range control signal given from the overall control unit 13 so that the R signal, the G signal, and the B signal have the same photoelectric conversion characteristics. Applied.

  The AE / WB evaluation value detection circuit 7 detects the optimum dynamic range for the subject based on the RGB signals from the FPN correction circuit 6. When the detection result is sent to the overall control unit 13, the overall control unit 13 generates a dynamic range control signal based on the detected dynamic range. The dynamic range detection operation in the AE / WB evaluation value detection circuit 7 will be described later.

  The R signal, G signal, and B signal that have been subjected to white balance processing by the WB control circuit 8 are subjected to color interpolation processing by the color interpolation circuit 9. When a color filter having an RGB Bayer array as shown in FIG. 2 is provided in the solid-state imaging device 2, the color signal output from each pixel is only the color signal from the color filter provided in the pixel. Therefore, the color interpolation circuit 9 performs color interpolation processing by generating other color signals from the color signals of adjacent pixels.

  When the RGB color filters are arranged in the pixels G11 to G44 as shown in FIG. 2, the R signals r11, r31, r13, r33 from the pixels G11, G31, G13, G33 are converted into the pixels G21, G41, G signals G21, g41, g12, g32, g23, g43, g14, g34 from G12, G32, G23, G43, G14, G34, and B signals b22, b42, b24, b44 from pixels G22, G42, G24, G44. Is output. At this time, the RGB signals of the pixels G22, G23, G32, and G33 are expressed by the following equations.

R signal r22, G signal g22, B signal b22 of pixel G22
r22 = (r11 + r31 + r13 + r33) / 4
g22 = (g21 + g12 + g32 + g23) / 4
b22 = b22
R signal r32, G signal g32, B signal b32 of the pixel G32
r32 = (r31 + r33) / 2
g32 = g32
b32 = (b22 + b42) / 2
R signal r23, G signal g23, B signal b23 of pixel G23
r23 = (r13 + r33) / 2
g23 = g23
b23 = (b22 + b24) / 2
R signal r33, G signal g33, B signal b33 of pixel G33
r33 = r33
g33 = (g32 + g23 + g43 + g34) / 4
b33 = (b22 + b42 + b24 + b44) / 4.

When the RGB signal is obtained for each pixel by performing the color interpolation processing in this way, the RGB signal of each pixel is given to the color correction circuit 10 and the color correction processing for emphasizing the hue of each pixel is performed. Is done. At this time, each of the RGB signals is subjected to color correction by the values of other color signals. That is, the RGB signals rxkl, gxkl, bxkl of the pixel Gkl subjected to the hue correction are generated by substituting the RGB signals rkl, gkl, bkl of the pixel Gkl into the following expression. At this time, the color correction coefficients based on a1 to a3, b1 to b3, and c1 to c3 are switched based on the photoelectric conversion characteristics of the input RGB signal, and the color tone of the RGB signal of each pixel is emphasized. Details of the switching operation of the color correction coefficients a1 to a3, b1 to b3, and c1 to c3 in the color correction circuit 10 will be described later.
rxkl = a1 * rkl + a2 * gkl + a3 * bkl
gxkl = b1 * rkl + b2 * gkl + b3 * bkl
bxkl = c1 * rkl + c2 * gkl + c3 * bkl.

When the color correction process is not performed in the color correction circuit 10, the following color correction coefficients a1 to a3, b1 to b3, and c1 to c3 are given from the overall control unit 13.
(A1, a2, a3) = (1.0, 0.0, 0.0)
(B1, b2, b3) = (0.0, 1.0, 0.0)
(C1, c2, c3) = (0.0, 0.0, 1.0).

  The RGB signal that has been subjected to color correction by the color correction circuit 10 is given to the gradation conversion circuit 11 so that the dynamic range control signal and the AE evaluation value are input from the overall control unit 13 so as to obtain an appropriate output level. Based on the above, the gradation characteristics are changed by a change based on the γ curve or a change in digital gain. Then, in the coring circuit 12 having the level conversion characteristic having the relationship as shown in FIG. 3 with respect to the edge component, the noise component superimposed on each of the RGB signals is removed, and the edge component is extracted and subjected to edge enhancement processing. Is done. The operations of the gradation conversion circuit 11 and the coring circuit 12 will be described later.

<Configuration example of solid-state imaging device>
The configuration of the solid-state imaging device 2 in the imaging apparatus configured as shown in FIG. 1 will be described with reference to the drawings. FIG. 4 is a block diagram schematically showing a part of the configuration of the solid-state imaging device of this example, and FIG. 5 is a circuit diagram showing the configuration of each pixel.

  As shown in FIG. 4, in the solid-state imaging device 2, G11 to Gmn represent pixels arranged in a matrix (matrix arrangement). Reference numeral 21 denotes a vertical scanning circuit, which sequentially scans rows (lines) 23-1, 23-2,..., 23-n that apply a signal φV to each pixel and lines 24-1 and 24-2. ,..., 24-n, and a signal φVD is applied to each pixel via lines 25-1, 25-2,. A horizontal scanning circuit 22 sequentially reads out photoelectric conversion signals derived from the pixels to the output signal lines 26-1, 26-2, ..., 26-m in the horizontal direction for each pixel. Reference numeral 20 denotes a power supply line. For each pixel, not only the lines 23-1 to 23-n, 24-1 to 24-n, 25-1 to 25-n, the output signal lines 26-1 to 26-m, and the power supply line 20, but also These lines (for example, a clock line and a bias supply line) are also connected, but these are omitted in FIG.

  Also, constant current sources 27-1 to 27-m are connected to the output signal lines 26-1 to 26-m, and pixels are provided via the signal lines 26-1 to 26-m, respectively. Selection circuits 28-1 to 28-m for sample-holding image signals and noise signals supplied from G11 to Gmn are provided. Then, when the image signal and the noise signal are sequentially transmitted from the selection circuits 28-1 to 28-m to the correction circuit 29, the correction circuit 29 performs correction processing and outputs the image signal from which noise has been removed to the outside. Is done. The DC voltage VPS is applied to one end of the constant current sources 27-1 to 27-m.

  In such a solid-state imaging device, an image signal and a noise signal output from the pixel Gab (a natural number of a: 1 ≦ a ≦ m, b: 1 ≦ b ≦ n) are respectively output signal lines 26- The signal is output via a and amplified by a constant current source 27-a connected to the output signal line 26-a. The image signal and noise signal output from the pixel Gab are sequentially sent to the selection circuit 28-a, and the sent image signal and noise signal are sampled and held in the selection circuit 28-a. Thereafter, after the sampled and held image signal is sent to the correction circuit 29 from the selection circuit 28-a, the same sampled and held noise signal is sent to the correction circuit 29. The correction circuit 29 corrects the image signal given from the selection circuit 28-a based on the noise signal similarly given from the selection circuit 28-a, and outputs the noise-removed image signal to the amplifier 3.

  In the solid-state imaging device 2 having such a configuration, as shown in FIG. 5, the pixels G11 to Gmn have the drain of the MOS transistor T1 connected to the anode of the photodiode PD to which the DC voltage VPD is applied to the cathode. The gate and drain of the MOS transistor T2 and the gate of the MOS transistor T3 are connected to the source of T1. The gate of the MOS transistor T4 and the drain of the MOS transistor T5 are connected to the source of the MOS transistor T3, and the drain of the MOS transistor T6 is connected to the source of the MOS transistor T4. The drain of the MOS transistor T6 is connected to the output signal line 26 (corresponding to the output signal lines 26-1 to 26-m in FIG. 4). The MOS transistors T1 to T6 are P-channel MOS transistors.

  The signal φVPS is input to the source of the MOS transistor T2 via the line 25 (corresponding to the lines 25-1 to 25-n in FIG. 4), and the DC voltage VPD is applied to the drains of the MOS transistors T3 and T4. Further, the other end of the capacitor C to which the signal φVD is applied via the line 24 (corresponding to the lines 24-1 to 24-n in FIG. 4) is connected to the source of the MOS transistor T3. The DC voltage VRG is input to the source of the MOS transistor T5, and the signal φRS is input to the gate thereof. Further, signals φS and φV are input to the gates of the MOS transistors T1 and T6, respectively.

  The signal φVPS is a binary voltage signal. When the amount of incident light exceeds a predetermined value, the voltage for operating the MOS transistor T2 in the subthreshold region is VL, and the voltage higher than this voltage is used for the MOS transistor T2. The voltage for making the conductive state is VH. The signal φVD is a ternary voltage signal, the voltage value when integrating the capacitor C is set to the highest Vh, the voltage value when reading the image signal is set to Vm lower than Vh, and the noise signal is read The voltage value is set to Vl lower than Vm.

  The operation of the pixels G11 to Gmn in the solid-state imaging device 2 configured as described above will be described with reference to the time chart of FIG. First, when a pulse signal φVD having a voltage value Vm and a pulse signal φV are given and an image signal is output, the signal φVD is set to Vh, then the signal φS is set high to turn off the MOS transistor T1, and the reset operation is performed. Begins. Next, the signal φVPS applied to the source of the MOS transistor T2 is set to VH, and the source voltage of the MOS transistor T2 is increased, so that negative charges accumulated in the gate and drain of the MOS transistor T2 and the gate of the MOS transistor T3 are accumulated. Are quickly recombined. At this time, the signal φRS is set low, the MOS transistor T5 is turned on, and the voltage at the connection node between the capacitor C and the gate of the MOS transistor T4 is initialized.

  Then, the signal φVPS applied to the source of the MOS transistor T2 is set to VL, the potential state of the MOS transistor T2 is returned to the original state, and then the signal φRS is set high to turn off the MOS transistor T5. Thereafter, the capacitor C performs an integration operation, and the voltage at the connection node between the capacitor C and the gate of the MOS transistor T4 becomes in accordance with the reset gate voltage of the MOS transistor T2. Then, the pulse signal φV is applied to the gate of the MOS transistor T6 to turn on the MOS transistor T6 and set the voltage value of the signal φVD to Vl. At this time, since the MOS transistor T4 operates as a source follower type MOS transistor, a noise signal appears on the output signal line 26 as a voltage signal. After that, the pulse signal φRS is again applied to the MOS transistor T5 to reset the voltage at the connection node between the capacitor C and the gate of the MOS transistor T4. Make it ready.

  After the noise signal is output in this way, the imaging operation is started when the MOS transistor T1 is turned on. At this time, the signal φRS is set high and the MOS transistor T5 is turned off. Further, the signal φVPS applied to the source of the MOS transistor T2 is set to VL, and the voltage value of the signal φVD applied to the capacitor C is set to Vh to perform the integration operation. When photocharge corresponding to the amount of incident light flows from the photodiode PD into the MOS transistor T2, the MOS transistor T2 is in a cut-off state, so that the photocharge is accumulated at the gate of the MOS transistor T2.

  Therefore, when the luminance of the subject to be imaged is low and the amount of incident light incident on the photodiode PD is small, a voltage corresponding to the amount of photocharge accumulated at the gate of the MOS transistor T2 appears at the gate of the MOS transistor T2, A voltage linearly proportional to the integrated value of the light quantity appears at the gate of the MOS transistor T3. Also, when the luminance of the object to be imaged is high and the amount of incident light entering the photodiode PD is large, and the voltage according to the amount of photocharge accumulated at the gate of the MOS transistor T2 increases, the MOS transistor T2 operates in the subthreshold region. Therefore, a voltage that changes logarithmically with respect to the amount of incident light appears at the gate of the MOS transistor T3.

  Since a drain current that is linearly or naturally logarithmically changed with respect to the incident light quantity flows from the capacitor C through the current amplified by the MOS transistor T3, the gate voltage of the MOS transistor T4 is equal to the integral value of the incident light quantity. Thus, the voltage changes linearly or in a natural logarithm. Then, by setting the voltage value of the signal φVD to Vm and supplying the pulse signal φV to the MOS transistor T6, a source current corresponding to the gate voltage of the MOS transistor T4 flows to the output signal line 6 via the MOS transistor T6. . At this time, since the MOS transistor T4 operates as a source follower type MOS transistor, an image signal appears as a voltage signal on the output signal line 6. Thereafter, the signal φV is set high to turn off the MOS transistor T6, and the voltage value of the signal φVD is set to Vh.

  When operating in this way, the voltage value VL of the signal φVPS at the time of imaging decreases, and the potential difference between the gate and the source of the MOS transistor T2 increases as the difference from the voltage value VH of the signal φVPS at the time of reset increases. The ratio of subject luminance at which the MOS transistor T2 operates in the cutoff state increases. Therefore, as shown in FIG. 7, the lower the voltage value VL, the larger the ratio of subject luminance to be linearly converted. Therefore, for example, the luminance range of the subject is detected, and when the luminance range of the subject is narrow, the voltage value VL is lowered to widen the luminance range for linear conversion, and when the luminance range of the subject is wide, the voltage value VL is increased. By widening the luminance range for logarithmic conversion, photoelectric conversion characteristics that match the characteristics of the subject can be obtained. It should be noted that when the voltage value VL is minimized, it is possible to always perform linear conversion, and when the voltage value VL is maximized, it is possible to always perform logarithmic conversion.

  The solid-state image sensor that can switch the dynamic range according to the luminance range of the subject, etc., by the overall controller 13 switching the voltage value VL of the signal φVPS given to the pixels G11 to Gmn of the solid-state image sensor 2 operating in this way. 2 can be used. That is, the inflection point (luminance value) for switching from the linear conversion operation to the logarithmic conversion operation in the pixels G11 to Gmn of the solid-state imaging device 2 can be set by switching the voltage value VL of the signal φVPS. it can. It should be noted that the amount of photocharge flowing into the MOS transistor T2 until reaching the gate voltage of the MOS transistor T2 when changing to the logarithmic conversion operation at the time of imaging is the same in all the pixels.

  In this configuration example, the solid-state imaging device having pixels configured as shown in FIG. 5 is used. However, the present invention is not limited to this configuration, and linear conversion operation and logarithmic conversion operation are automatically performed in each pixel. As long as it can be switched to the above, it may be composed of pixels having other configurations such as a pixel having a configuration as shown in Patent Document 2. In addition, the inflection point between the linear conversion operation and the logarithmic conversion operation is changed by changing the voltage value VL of the signal φVPS at the time of imaging. The inflection point between the conversion operation and the logarithmic conversion operation may be changed. The inflection point may be changed by changing the reset time. Furthermore, each pixel is provided with an RGB filter, but other color filters such as cyan, magenta, and yellow may also be provided.

<First Example of Dynamic Range Detection Operation>
A first example of the dynamic range detection operation by the AE / WB evaluation value detection circuit 7 in the imaging apparatus configured as shown in FIG. 1 will be described below with reference to the drawings. FIG. 8 is a flowchart showing the dynamic range detection operation of this example.

  As shown in the flowchart of FIG. 8, the AE / WB evaluation value detection circuit 7 in this example first checks the signal level of the RGB signal from which the FPN component has been removed by the FPN correction circuit 6, thereby outputting the output level of each pixel. (STEP 1). The photoelectric conversion characteristics of the RGB signals input from the FPN correction circuit 6 are as shown in FIG. As can be confirmed from FIG. 9, since the FPN component is removed by the FPN correction circuit 6, the signal level Vth at which the photoelectric conversion characteristic changes from the linear conversion characteristic to the logarithmic conversion characteristic, and the pixel is saturated. The maximum signal level (saturation level) Vmax is equal for each of the RGB signals.

  When the signal level corresponding to the output level of each pixel is confirmed, the number of pixels at that signal level is counted by one (STEP 2). That is, when the signal level Vx is confirmed in STEP1, if the number of pixels corresponding to the signal level Vx is already x, the number of pixels corresponding to the signal level Vx is counted as one and is set to x + 1. Thereafter, it is confirmed whether or not the signal levels for all the pixels G11 to Gmn of the solid-state imaging device 2 have been detected (STEP 3). If all the pixels have not been confirmed (No), the process proceeds to STEP 1. If the brightness values of all the pixels are confirmed in STEP 3 (Yes), a subject histogram is generated from the number of pixels for each signal level (STEP 4). Note that the signal levels for all the pixels are detected in STEP 3, but the histogram may be generated by detecting the signal levels of some pixels in the imaging region instead of for all the pixels.

  When the histogram of the subject is confirmed in this way, the maximum signal level is detected based on this histogram (STEP 5). At this time, when a histogram as shown in FIG. 10 is obtained, the maximum signal level Vh is detected from the signal level at which the number of pixels is equal to or greater than the predetermined number Gth1. It is confirmed whether or not the maximum signal level Vh detected in this way is the saturation level Vmax (STEP 6). At this time, when the maximum signal level Vh is not the saturation level Vmax (No), the maximum signal level Vh is compared with a predetermined threshold level Va corresponding to the dynamic range currently set for the solid-state imaging device 2 (STEP 7). ). The predetermined threshold level Va is a value set for each dynamic range set for the solid-state imaging device 2, and is a value serving as a reference for changing the dynamic range.

  If it is confirmed in STEP 6 that the maximum signal level Vh is the saturation level Vmax (Yes), the dynamic range of the solid-state imaging device 2 is detected as a dynamic range that is wider than the current dynamic range, and the entire control is performed. This is notified to the unit 13 (STEP 8). If it is confirmed in STEP 7 that the maximum signal level Vh is smaller than the predetermined threshold level Va (Yes), the dynamic range of the solid-state imaging device 2 is detected as a dynamic range that is narrower than the current dynamic range. Then, it notifies the overall control unit 13 (STEP 9). Further, when it is confirmed in STEP 7 that the maximum signal level Vh is equal to or higher than the predetermined threshold level Va (No), the currently set dynamic range of the solid-state imaging device 2 is detected as the optimum dynamic range, and the overall control is performed. This is notified to the unit 13 (STEP 10).

  When a wide dynamic range is detected as the optimum dynamic range in STEP 8, the optimum dynamic range is set so that the dynamic range is widened when the number of pixels of the maximum signal level is large. Further, when a narrow dynamic range is detected as the optimum dynamic range in STEP 9, a dynamic range in which the maximum signal level is a value near the saturation level and is smaller than the saturation level is set as the optimum dynamic range.

  Therefore, for example, when the photoelectric conversion characteristic of the solid-state imaging device 2 is in a state as shown by a3 in FIG. 11, if the number of pixels of the maximum signal level that is the saturation level Vmax is large, the photoelectric conversion characteristic of the solid-state imaging device 2 is When the widest dynamic range D1 that becomes a1 in FIG. 11 is set to the optimum dynamic range and the number of pixels of the maximum signal level that becomes the saturation level is small, the photoelectric conversion characteristics of the solid-state imaging device 2 are the same as a2 in FIG. The next wide dynamic range D2 is set to the optimum dynamic range. Further, for example, when the maximum signal level is Vh1 when the photoelectric conversion characteristic of the solid-state imaging device 2 is in the state as shown in a1 of FIG. 11, the signal level with respect to the luminance value Lh1 corresponding to the maximum signal level Vh1 is saturated. A photoelectric conversion characteristic having a value closest to the level Vmax is confirmed. Therefore, the dynamic range D3 based on the photoelectric conversion characteristic a3 that becomes the value closest to the saturation level Vmax when the signal level with respect to the luminance value Lh1 is Vh2 is set to the optimum dynamic range.

<Second example of dynamic range detection operation>
A second example of the dynamic range detection operation by the AE / WB evaluation value detection circuit 7 in the imaging apparatus configured as shown in FIG. 1 will be described below with reference to the drawings. FIG. 12 is a flowchart showing the dynamic range detection operation of this example. In the flowchart of FIG. 12, the same operation steps as those in the flowchart of FIG. 8 are denoted by the same reference numerals, and detailed description thereof is omitted.

  As shown in the flowchart of FIG. 12, the AE / WB evaluation value detection circuit 7 in the present example is similar to the first example. First, in STEP 1 to STEP 4, the signal level of the RGB signal from which the FPN component has been removed by the FPN correction circuit 6 By confirming the signal levels of all the pixels of the solid-state image sensor 2, the number of pixels at each signal level is counted to form a histogram of the subject. When the histogram of the subject is confirmed in this way, the number of pixels at the saturation level Vmax is confirmed from this histogram (STEP 101).

  Then, the number of pixels Gx at the saturation level Vmax is compared with a predetermined number Gth2 (Gth2 <Gth1) (STEP 102). At this time, when the number of pixels Gx at the saturation level Vmax is smaller than the predetermined number Gth2 (No), the maximum signal level Vh is detected from the signal level at which the number of pixels is equal to or greater than the predetermined number Gth1 (No). (Step 5). The detected maximum signal level Vh is compared with a predetermined threshold level Va corresponding to the dynamic range currently set for the solid-state imaging device 2 (STEP 7).

  In STEP102, when it is confirmed that the number of pixels Gx at the saturation level Vmax is equal to or greater than the predetermined number Gth2 (Yes), the dynamic range of the solid-state imaging device 2 is detected as a dynamic range wider than the current dynamic range as the optimum dynamic range. Then, it notifies the overall control unit 13 (STEP 8). If it is confirmed in STEP 7 that the maximum signal level Vh is smaller than the predetermined threshold level Va (Yes), the dynamic range of the solid-state imaging device 2 is optimized to be narrower than the current dynamic range as in the first example. The dynamic range is detected and notified to the overall control unit 13 (STEP 9). Furthermore, when it is confirmed in STEP 7 that the maximum signal level Vh is equal to or higher than the predetermined threshold level Va (No), the dynamic range of the currently set solid-state imaging device 2 is set to the optimum dynamic range as in the first example. Is detected and notified to the overall control unit 13 (STEP 10).

  When a wide dynamic range is detected as the optimum dynamic range in STEP 8, as in the first example, the optimum dynamic range is set so that the dynamic range is widened when the number of pixels of the maximum signal level is large. Also, when a narrow dynamic range is detected as the optimum dynamic range in STEP 9, as in the first example, the dynamic range in which the maximum signal level is a value near the saturation level and smaller than the saturation level is set to the optimum dynamic range. Set as a range.

  As in this example, the number of pixels Gth2, which is a saturation level serving as a threshold for expanding the dynamic range, is smaller than the number of pixels Gth1 when detecting the maximum signal level Vh to be checked to narrow the dynamic range. By doing so, it can be confirmed with a smaller change rate when the dynamic range of the subject becomes wider than when the dynamic range of the subject becomes narrower. Therefore, the sensitivity for detecting the expansion of the dynamic range can be made higher than the sensitivity for detecting the reduction of the dynamic range, and hysteresis can be given to the change.

  Since the optimum dynamic range detected as in the first and second examples of the dynamic range detection operation described above is given to the overall control unit 13, it is set by the dynamic range control signal from the overall control unit 13. The photoelectric conversion characteristic of the solid-state imaging device 2 is a photoelectric conversion characteristic such that the luminance region for performing the photoelectric conversion with the linear conversion characteristic is as wide as possible with respect to the subject. When the dynamic range control signal is output from the overall control unit 13 to switch the photoelectric conversion characteristics of the solid-state imaging device 2, when switching so that the dynamic range is widened, the switching time is shortened and switched rapidly. When switching so that the dynamic range is narrowed, the switching time is lengthened and gradually switched.

<Black reference correction circuit>
The operation of the black reference correction circuit in the imaging apparatus configured as shown in FIG. 1 will be described in detail below with reference to the drawings. FIG. 13 is a block diagram showing the internal configuration of the black reference correction circuit.

  As shown in FIG. 13, the black reference correction circuit 5 receives a black level setting unit 51 that sets a black level based on a dynamic range control signal supplied from the overall control unit 13, and an RGB signal input from the AD conversion circuit 4. And a subtractor 52 that performs reference value correction by subtracting the black level set by the black level setting unit 51. When the black reference correction circuit 5 is configured in this way, the black level setting unit 51 corresponds to each photoelectric conversion characteristic with the same number of black levels as the photoelectric conversion characteristic of the solid-state imaging device 2 set by the dynamic range control signal. Let me store.

  This black level difference is reset when the voltage value VL of the signal φVPS during the imaging operation is changed when the photoelectric conversion characteristics are set in the solid-state imaging device 2 because the reset time of the solid-state imaging device 2 is finite. The larger the difference from the voltage value VH of the time signal φVPS, the higher the reset effect by the reset operation. This is because the difference between the voltage values VH and VL of the signal φVPS is large, so that the difference between the image signal at the time of imaging output from each pixel and the noise signal at the time of reset is large.

  Because the difference between the image signal output from each pixel at the time of imaging and the noise signal at the time of resetting becomes large, the solid state becomes more solid when the same minimum luminance is obtained and the photoelectric conversion characteristic has a wider luminance region of the linear conversion characteristic. The signal level of the image signal with respect to the minimum luminance output from the image sensor 2 becomes higher, and conversely, the image with respect to the minimum luminance output from the solid-state image sensor 2 as the logarithmic conversion characteristic has a wider luminance region. The signal level of the signal is lowered. Therefore, the black level stored in the black level setting unit 51 has a higher value for a photoelectric conversion characteristic having a wider luminance region of the linear conversion characteristic, and conversely, for a photoelectric conversion characteristic having a wider luminance region of the logarithmic conversion characteristic. Low value. In other words, when the photoelectric conversion characteristics a1 to a4 as shown in FIG. 11 are switched by the dynamic range control signal given from the overall control section 13, the black level setting section 51 in the order of the photoelectric conversion characteristics a1, a2, a3, and a4. The stored black level is a low value.

  In the black reference correction circuit 5 in which the black level corresponding to each photoelectric conversion characteristic is stored in the black level setting unit 51, when the dynamic range control signal is input from the overall control unit 13, first, the dynamic range control is performed. The same photoelectric conversion characteristic as that determined by the signal is confirmed. At this time, the photoelectric conversion characteristics determined by the dynamic range control signal and the photoelectric conversion characteristics stored corresponding to each black reference level may be compared in order to confirm the same photoelectric conversion characteristics. Absent.

  When the photoelectric conversion characteristic identical to the photoelectric conversion characteristic determined by the dynamic range control signal is confirmed, the black level corresponding to the confirmed photoelectric conversion characteristic is read and provided to the subtractor 52. That is, when the photoelectric conversion characteristics a1 to a4 as shown in FIG. 11 are switched and the black levels Vb1 to Vb4 are stored for the photoelectric conversion characteristics a1 to a4, for example, the photoelectric conversion characteristics are determined by a dynamic range control signal. When set to be a2, the black level Vb2 is read and applied to the subtractor 52. In this way, when the black level set by the black level setting unit 51 is given to the subtractor 52, the subtracter 52 sets the black level set by the black level setting unit 51 from the RGB signal output from the AD conversion circuit 4. The black level is subtracted and output to the FPN correction circuit 6. Therefore, the RGB signal subjected to the reference value correction can be output to the FPN correction circuit 6.

<Color correction circuit>
The operation of the color correction circuit in the imaging apparatus configured as shown in FIG. 1 will be described in detail below with reference to the drawings. FIG. 14 is a block diagram showing the internal configuration of the color correction circuit.

  As shown in FIG. 14, the color correction circuit 10 is set by coefficient setting units 102r, 102g, and 102b that set color correction coefficients for RGB signals by a dynamic range control signal from the overall control unit 13, and a coefficient setting unit 102r. Multipliers 103rr, 103rg, and 103rb to which the color correction coefficients a1, a2, and a3 are respectively provided, and multipliers 103gr, 103gg, and 103gb to which the color correction coefficients b1, b2, and b3 set by the coefficient setting unit 102g are respectively provided. A multiplier 103br, 103bg, 103bb to which the color correction coefficients c1, c2, c3 set by the coefficient setting unit 102b are respectively given, an adder 104r for adding outputs from the multipliers 103rr, 103rg, and a multiplier 103rb And an adder 1 for adding the outputs from the adder 104r 5r, an adder 104g for adding the outputs from the multipliers 103gr and 103gg, an adder 105g for adding the outputs from the multiplier 103gb and the adder 104g, and an adder for adding the outputs from the multipliers 103br and 103bg 104b and an adder 105b that adds outputs from the multiplier 103bb and the adder 104b.

  At this time, in each of the coefficient setting units 102r, 102g, and 102b, the same number of color correction coefficients a1 to a3, b1 to b3, and c1 to c3 as the photoelectric conversion characteristics of the solid-state imaging device 2 set by the dynamic range control signal are set. Stored in correspondence with the photoelectric conversion characteristics. Therefore, when a dynamic range control signal is input from the overall control unit 13 to each of the coefficient setting units 102r, 102g, and 102b, first, the same photoelectric conversion characteristic as the photoelectric conversion characteristic determined by the dynamic range control signal is confirmed. The color correction coefficients a1 to a3, b1 to b3, and c1 to c3 corresponding to the confirmed photoelectric conversion characteristics are read out. At this time, the photoelectric conversion characteristics determined by the dynamic range control signal and the photoelectric conversion characteristics stored corresponding to each color correction coefficient may be compared in order to confirm the same photoelectric conversion characteristics. .

  Further, as the dynamic range of the solid-state image pickup device 2 is expanded, the color tone by the RGB signal input from the color interpolation circuit 9 becomes lighter, so that the color correction coefficients a1 to a3, b1 that perform color enhancement as the dynamic range becomes wider. ˜b3, c1 to c3 are stored. Specifically, the color correction coefficients a1, b2, and c3 are set to values closer to 1.0 as the luminance region serving as the linear conversion characteristic in the photoelectric conversion characteristic of the solid-state image sensor 2 becomes wider. As the luminance region that becomes the logarithmic conversion characteristic in the photoelectric conversion characteristic of the element 2 becomes wider, the values of the color correction coefficients a1, b2, and c3 are set to values larger than 1.0. The values of the color correction coefficients a1 to a3, b1 to b3, and c1 to c3 are values satisfying a1 + a2 + a3 = 1, b1 + b2 + b3 = 1, and c1 + c2 + c3 = 1, respectively. For the photoelectric conversion characteristics ax, ay, and az in which the values of the color correction coefficients a1 to a3, b1 to b3, and c1 to c3 increase in the order of ax, ay, and az, for example, Set to

Photoelectric conversion characteristics ax
(A1, a2, a3) = (1.2, -0.1, -0.1)
(B1, b2, b3) = (− 0.1, 1.2, −0.1)
(C1, c2, c3) = (− 0.1, −0.1, 1.2)

Photoelectric conversion characteristics ay
(A1, a2, a3) = (1.35, -0.175, -0.175)
(B1, b2, b3) = (-0.175, 1.35, -0.175)
(C1, c2, c3) = (-0.175, -0.175, 1.35)

Photoelectric conversion characteristics az
(A1, a2, a3) = (1.5, -0.25, -0.25)
(B1, b2, b3) = (− 0.25, 1.5, −0.25)
(C1, c2, c3) = (− 0.25, −0.25, 1.5).

  Therefore, when a dynamic range control signal corresponding to the photoelectric conversion characteristic ax is given from the overall control unit 13 to the coefficient setting units 102r, 102g, and 102b, the coefficient setting unit 102r sets the color correction coefficients (a1, a2, a3) to (1.2 , −0.1, −0.1) to the multipliers 103rr, 103rg, 103rb, and the coefficient setting unit 102g sets the color correction coefficients (b1, b2, b3) to (−0.1, 1.2, −0.1) and the multipliers 103gr, 103gg, The coefficient setting unit 1102b supplies the color correction coefficients (c1, c2, c3) as (−0.1, −0.1, 1.2) to the multipliers 103br, 103bg, and 103bb.

  When a dynamic range control signal corresponding to the photoelectric conversion characteristic ay is given from the overall control unit 13 to the coefficient setting units 102r, 102g, and 102b, the coefficient setting unit 102r sets the color correction coefficients (a1, a2, a3) to (1.35). , -0.175, -0.175) to the multipliers 103rr, 103rg, 103rb, and the coefficient setting unit 102g sets the color correction coefficients (b1, b2, b3) to (-0.175, 1.35, -0.175) and the multipliers 103gr, 103gg, The coefficient setting unit 102b supplies the color correction coefficients (c1, c2, c3) as (−0.175, −0.175, 1.35) to the multipliers 103br, 103bg, and 103bb.

  When a dynamic range control signal corresponding to the photoelectric conversion characteristic az is given from the overall control unit 13 to the coefficient setting units 102r, 102g, and 102b, the coefficient setting unit 102r sets the color correction coefficients (a1, a2, a3) to (1.5). , −0.25, −0.25) to the multipliers 103rr, 103rg, 103rb, and the coefficient setting unit 102g sets the color correction coefficients (b1, b2, b3) to (−0.25, 1.5, −0.25) and the multipliers 103gr, 103gg, The coefficient setting unit 102b supplies the color correction coefficients (c1, c2, c3) as (−0.25, −0.25, 1.5) to the multipliers 103br, 103bg, and 103bb.

  As described above, when the color correction coefficients a1, a2, and a3 set by the coefficient setting unit 102r are respectively supplied to the multipliers 103rr, 103rg, and 103rb, the multiplier 103rr causes the color correction coefficient a1 to be applied to the R signal and the multiplier 103rg. The G signal is multiplied by the color correction coefficient a2, and the multiplier 103rb is multiplied by the B signal by the color correction coefficient a3. Similarly, when the color correction coefficients b1, b2, and b3 set by the coefficient setting unit 102g are respectively supplied to the multipliers 103gr, 103gg, and 103gg, the multiplier 103gr supplies the color correction coefficient b1 to the R signal and the multiplier 103gg. The G signal is multiplied by the color correction coefficient b2, and the multiplier 103gb multiplies the B signal by the color correction coefficient b3. Similarly, when the color correction coefficients c1, c2, and c3 set by the coefficient setting unit 102b are respectively supplied to the multipliers 103br, 103bg, and 103bb, the multiplier 103br outputs the color correction coefficient c1 to the R signal and the multiplier 103bg. The G signal is multiplied by the color correction coefficient c2, and the multiplier 103bb multiplies the B signal by the color correction coefficient c3.

  Then, the R signal multiplied by the color correction coefficient a1 by the multiplier 103rr is added by the adder 104r to the G signal obtained by multiplying the color correction coefficient a2 by the multiplier 103rg, and then the color correction coefficient by the multiplier 103rb. The B signal multiplied by a3 is added by the adder 105r, and the color-corrected R signal is output. Similarly, the R signal multiplied by the color correction coefficient b1 by the multiplier 103gr and the G signal multiplied by the color correction coefficient b2 by the multiplier 103gg are added by the adder 104g, and then the color correction is performed by the multiplier 103gg. The B signal multiplied by the coefficient b3 is added by the adder 105g, and the color-corrected G signal is output. Similarly, the R signal multiplied by the color correction coefficient c1 by the multiplier 103br and the G signal multiplied by the color correction coefficient c2 by the multiplier 103bg are added by the adder 104b, and then the color correction is performed by the multiplier 103bb. The B signal multiplied by the coefficient c3 is added by the adder 105b, and the color-corrected B signal is output.

  Therefore, for example, when a dynamic range control signal corresponding to the photoelectric conversion characteristic ay is given from the overall control unit 13 to the coefficient setting units 102r, 102g, and 102b, the coefficient setting unit 102r causes the color correction coefficients (a1, a2, and a3). (1.35, -0.175, -0.175) to the multipliers 103rr, 103rg, 103rb, and the coefficient setting unit 102g sets the color correction coefficients (b1, b2, b3) as (-0.175, 1.35, -0.175). The multipliers 103gr, 103gg, and 103gb are provided, and the coefficient setting unit 102b supplies the color correction coefficients (c1, c2, and c3) to the multipliers 103br, 103bg, and 103bb as (−0.175, −0.175, 1.35).

  The multiplier 103rr multiplies the R signal rkl by the color correction coefficient a1 of 1.35, the multiplier 103rg multiplies the G signal gkl by the color correction coefficient a2 of −0.175, and the multiplier 103rb multiplies the B signal bkl by −. The color correction coefficient a3 which is 0.175 is multiplied. Further, the multiplier 103gr multiplies the R signal rkl by a color correction coefficient b1 of −0.175, the multiplier 103gg multiplies the G signal gkl by a color correction coefficient b2 of 1.35, and the multiplier 103gg multiplies the B signal bkl by −. The color correction coefficient b3 which is 0.175 is multiplied. The multiplier 103br multiplies the R signal rkl by a color correction coefficient c1 of −0.175, the multiplier 103bg multiplies the G signal gkl by the color correction coefficient c2 of −0.175, and the multiplier 103bb multiplies the B signal bkl by the multiplier 103bb. A color correction coefficient c3 of 1.35 is multiplied.

  Thereafter, the adder 104r adds the G signal of −0.175 × gkl to the R signal of 1.35 × rkl, and then the B signal of −0.175 × bkl is added by the adder 105r to obtain 1.35 × rkl−0.175. A color-corrected R signal rxkl of xgkl-0.175xbkl is output. Further, after the adder 104g adds the G signal of 1.35 × gkl to the R signal of −0.175 × rkl, the adder 105g adds the B signal of −0.175 × bkl, and −0.175 × rkl + A color-corrected G signal gxkl of 1.35 × gkl−0.175 × bkl is output. The adder 104b adds the -0.175 × gkl G signal to the −0.175 × rkl R signal, and then the adder 105b adds the 1.35 × bkl B signal to −0.175 × rkl−. A color-corrected B signal bxkl of 0.175 × gkl + 1.35 × bkl is output.

<Tone conversion circuit>
The operation of the gradation conversion circuit in the imaging apparatus configured as shown in FIG. 1 will be described in detail below with reference to the drawings. FIG. 15 is a block diagram showing the internal configuration of the gradation conversion circuit. FIG. 16 is a graph showing the relationship between input / output and photoelectric conversion characteristics in the gradation conversion circuit. In the following description, in order to simplify the description, a case will be described in which the photoelectric conversion characteristic has two conversion characteristics of a linear conversion characteristic and a logarithmic conversion characteristic.

  As shown in FIG. 15, the gradation conversion circuit 11 includes comparators 111r, 111g, and 111b that check the photoelectric conversion characteristics of the RGB signals of the pixels that are input from the color correction circuit 10, and comparators 111r, 111g, and 111b. From the color correction circuit 10 according to the arithmetic expressions given from the arithmetic expression setting sections 112r, 112g, and 112b, and the arithmetic expression setting sections 112r, 112g, and 112b, respectively. Gradation conversion units 113r, 113g, and 113b that perform gradation conversion of RGB signals of the input pixels.

  In the gradation conversion circuit 11 having such a configuration, the comparators 111r, 111g, and 111b perform photoelectric conversion of each signal by comparing the signal level of the RGB signal input from the color correction circuit 10 with the signal level serving as a threshold value. Check the characteristics. The operation of the comparator 111r will be described below as a representative. When the photoelectric conversion characteristic representing the relationship between the signal level of the R signal output from the color correction circuit 10 and the luminance value of the subject is expressed as shown in FIG. 16, the threshold level V1 is set as the signal level serving as the threshold. . When the luminance value corresponding to the threshold level V1 is L1 as shown in FIG. 16, the photoelectric conversion characteristic is a linear conversion characteristic in the luminance range lower than the luminance value L1, and the luminance is higher than the luminance value L1. In the range, the photoelectric conversion characteristic is assumed to be a logarithmic conversion characteristic. In the following description, a luminance range lower than the luminance value L1 is a linear region, and a luminance range higher than the luminance value L1 is a logarithmic region. Further, each of the GB signals output from the color correction circuit 10 has already been subjected to the white balance processing by the WB control circuit 8, so that the photoelectric conversion characteristics as shown in FIG. The threshold level for identifying the logarithmic region is the signal level V1 corresponding to the luminance value L1.

  Therefore, the comparator 111r confirms, for each pixel, whether the photoelectric conversion characteristic of the R signal is in a linear region or a logarithmic region based on this threshold level V1. That is, when an R signal whose signal level is lower than the threshold level V1 is input, it is confirmed that the signal is in the linear region, and when an R signal whose signal level is equal to or higher than the threshold level V1 is input, logarithm Confirmed to be in the area. Similarly, in the comparator 111g, based on the threshold level V1, for each pixel, whether the photoelectric conversion characteristic of the G signal is in a linear region or a logarithmic region is determined in the comparator 111b to the threshold level V1. Based on this, it is confirmed for each pixel whether the photoelectric conversion characteristic of the B signal is in a linear region or a logarithmic region. In the comparators 111r, 111g, and 111b, the same number of threshold levels V1 as the photoelectric conversion characteristics of the solid-state imaging device 2 set by the dynamic range control signal are stored in correspondence with the respective photoelectric conversion characteristics.

  The results confirmed by the comparators 111r, 111g, and 111b are respectively given to the arithmetic expression setting units 112r, 112g, and 112b. Based on the results, the arithmetic expressions fr for performing gradation conversion of the respective RGB signals. , Fg, fb are set. As the arithmetic expressions fr, fg, and fb, two types of arithmetic expressions corresponding to the two areas of the linear region and the logarithmic region are respectively set by the photoelectric conversion of the solid-state imaging device 2 set by the dynamic range control signal of the overall control unit 13. Each characteristic is stored in the arithmetic expression setting units 112r, 112g, and 112b. Therefore, for example, the gradation conversion characteristics by the arithmetic expressions fr, fg, and fb for the photoelectric conversion characteristics ax, ay, and az in which the dynamic range increases in the order of ax, ay, and az are expressed as shown in FIGS. The In addition, the threshold levels V1 for the photoelectric conversion characteristics ax, ay, and az are V1x, V1y, and V1z, respectively.

  That is, when a dynamic range control signal corresponding to the photoelectric conversion characteristic ax is given from the overall control unit 13, the signal level of 0 to V1x is 0 to Vx1 (Vx1 = V1x / Ax, The calculation formula is such that Ax> 1) and the signal level of V1x to Vmax is converted to Vx1 to Vmax in the logarithmic domain. When a dynamic range control signal corresponding to the photoelectric conversion characteristic ay is given from the overall control unit 13, the signal level of 0 to V1y is 0 to Vy1 (Vy1 = V1y / Ay, The calculation formula is such that Ay> 1) is converted, and in the logarithmic domain, the calculation formula is such that the signal levels V1y to Vmax are converted to Vy1 to Vmax. When a dynamic range control signal corresponding to the photoelectric conversion characteristic az is given from the overall control unit 13, the signal level of 0 to V1z is 0 to Vz1 (Vz1 = V1z / Az, The arithmetic expression is such that Az> 1) and the signal level of V1z to Vmax is converted to Vz1 to Vmax in the logarithmic domain.

  Therefore, when the dynamic range control signal is given from the overall control unit 13 in the comparators 111r, 111g, and 111b, a value corresponding to the photoelectric conversion characteristic set by the dynamic range control signal is read as the threshold level V1 and input. Comparison operation with the signal level of each of the RGB signals to be performed is performed. In the arithmetic expression setting units 112r, 112g, and 112b, first, when a dynamic range control signal is given from the overall control unit 13, dynamic range control signals are used as arithmetic expressions fr, fg, and fb for gradation conversion. Two types of arithmetic expressions corresponding to the photoelectric conversion characteristics set by are read out. When the comparison results from the comparators 111r, 111g, and 111b are given to the arithmetic expression setting units 112r, 112g, and 112b, the arithmetic expression setting units 112r, 112g, and 112b respectively convert the read two types of arithmetic expressions into comparison results. Corresponding arithmetic expressions are selected as arithmetic expressions fr, fg, and fb and given to the gradation conversion units 113r, 113g, and 113b.

  As described above, when the arithmetic expressions fr, fg, and fb set in the arithmetic expression setting units 112r, 112g, and 112b are respectively given to the gradation converting units 113r, 113g, and 113b, the gradation converting unit 113r converts the arithmetic expressions fr into the arithmetic expression fr. The tone conversion of the corresponding R signal is performed by the tone conversion unit 113g, the tone conversion of the G signal by the formula fg, and the tone conversion unit 113b by the tone conversion of the B signal by the formula fb. Done. Therefore, the gradation conversion units 113r, 113g, and 113b perform gradation conversion as shown in FIGS. 17 to 19 to lower the gradation of the linear region and increase the gradation of the logarithmic region in each of the RGB signals. .

  Therefore, for example, when the dynamic range control signal given from the overall control unit 13 sets the photoelectric conversion characteristic ax, the comparators 111r, 111g, and 111b read V1x as the threshold level V1 and input the RGB signal Is compared with the threshold level V1x. Further, the arithmetic expression setting units 112r, 112g, and 112b read out two types of arithmetic expressions in the linear region and the logarithmic region having the relationship shown in FIG. 17 as the arithmetic expressions fr, fx, and fz, respectively. In each of the comparators 111r and 111g, it is confirmed that the signal levels of the RG signals rxkl and gxkl of the pixel Gkl are lower than the threshold level V1x, and that the photoelectric conversion characteristics are in the linear region. The signal level of the B signal bkl of the pixel Gkl is higher than the threshold level V1x, and it is confirmed that the photoelectric conversion characteristic is in the logarithmic region.

  At this time, the arithmetic expression setting units 112r and 112g respectively give the arithmetic expressions fr and fg based on the linear regions in FIG. 17 to the gradation conversion units 113r and 113g, respectively, and the arithmetic expression setting unit 112b according to the logarithmic regions in FIG. The arithmetic expression fb is given to the gradation conversion unit 113b. In the gradation converting unit 113r, the R signal rxkl having a signal level in the range of 0 to V1x is changed to the R signal rykl having a signal level in the range of 0 to Vx1, and in the gradation converting unit 113g, the signal level is set to 0 to 0. The G signal gxkl in the range of V1x is the G signal gykl of the signal level in the range of 0 to Vx1, the B signal bxkl in the range of the signal level of V1x to Vmax is the B signal bykl in the range of Vx1 to Vmax, respectively. The gradation is converted by the conversion.

  As another embodiment of the gradation changing circuit 11, instead of providing the comparators 111r, 111g, and 111b and the arithmetic expression setting units 112r, 112g, and 112b, an LUT that stores the input / output characteristics of FIGS. Lookup table), that is, a plurality of LUTs corresponding to the number of photoelectric conversion characteristics may be provided, and gradation conversion may be performed by switching the LUTs by a dynamic range control signal. By using this LUT, it is not necessary to provide the comparators 111r, 111g, 111b and the arithmetic expression setting units 112r, 112g, 112b, and the configuration of the gradation conversion circuit 11 can be further simplified. When such an LUT is used, the input (vertical axis) in FIGS. 17 to 19 corresponds to the input of the LUT, and the output (horizontal axis) in FIGS. 17 to 19 is an output signal obtained by gradation conversion by the LUT. Equivalent to. Therefore, even with the LUT in which the input / output characteristics shown in FIGS. 17 to 19 are stored, the gradation in the linear region can be lowered and the gradation in the logarithmic region can be raised.

<Coring circuit>
The operation of the coring circuit in the imaging apparatus configured as shown in FIG. 1 will be described in detail below with reference to the drawings. FIG. 20 is a block diagram showing the internal configuration of the coring circuit.

  As shown in FIG. 20, the coring circuit 12 includes low-pass filters (LPF) 121r, 121g, and 121b that remove noise components of the RGB signals subjected to gradation conversion by the gradation conversion circuit 11, and the gradation conversion circuit 11. Output from the subtractors 122r, 122g, 122b and the subtractors 122r, 122g, 122b, respectively, which extract the noise components by subtracting the RGB signals from which the noise components have been removed from the LPFs 121r, 121g, 121b from the RGB signals from The coring units 123r, 123g, and 123b that perform coring calculation processing on the noise components of the RGB signals to be processed, and the edge components of the RGB signals obtained by performing the coring calculation processing in the coring units 123r, 123g, and 123b Multipliers 124r and 124 multiplied by the enhancement amount β. Comprises a 124b, the multiplier 124r, 124g, 124b LPF121r the edge component of the RGB signals from each, 121g, adders 125r to be added to each RGB signal from which the noise components have been removed in 121b, 125 g, and 125b, the.

  In the coring circuit 12 having such a configuration, the low-pass filters 121r, 121g, and 121b perform a matrix operation of n pixels × n pixels, and the n × n matrix coefficient (LPF coefficient) is obtained from the overall control unit 13. Given. Further, in the coring portions 123r, 123g, and 123b, the input and output have a relationship as shown in FIG. That is, when the input is greater than -α and smaller than α, the output is 0. When the input is less than -α, the output is a value obtained by adding α to the input, or when the input is greater than or equal to α, the output is input. Is a value obtained by subtracting α. In the following, this value α is used as a coring coefficient, and this coring coefficient α is given from the overall control unit 13. An edge enhancement amount β multiplied by the multipliers 124r, 124g, and 124b is also given from the overall control unit 13. The LPF coefficient, the coring coefficient α, and the edge enhancement amount β are set according to the size of the optimum dynamic range detected by the AE / WB evaluation value detection circuit 7 as in the case of the dynamic range control signal.

  At this time, for example, the LPF coefficient is a 5 × 5 matrix coefficient, and the LPF coefficient, the coring coefficient α, and the edge enhancement amount β are converted into two types of photoelectric conversion characteristics aX, Set as follows for aY.

Photoelectric conversion characteristics aX
LPF coefficient: 4 0 9 0 4
0 9 20 9 0
9 20 88 20 9
0 9 20 9 0
4 0 9 0 1
Coring coefficient α: 2
Edge enhancement amount β: 2

Photoelectric conversion characteristics aY
LPF coefficient: 1 2 4 2 1
2 6 11 6 2
4 11 24 11 4
2 6 11 6 2
1 2 4 2 1
Coring coefficient α: 4
Edge enhancement amount β: 3.

  When the R signal rykl, the G signal gykl, and the B signal bykl of the pixel Gkl are input from the gradation conversion circuit 11 to the coring circuit 12 configured in this way, the solid-state imaging device 2 confirmed by the overall control unit 13. The LPF coefficient, the coring coefficient α, and the edge enhancement amount β based on the photoelectric conversion characteristics are set in the LPF 121r, 121g, 121b, the coring units 123r, 123g, 123b, and the multipliers 124r, 124g, 124b, respectively. The LPFs 121r, 121g, and 121b operate according to the set LPF coefficients, so that the R signal rzkl from which the noise component has been removed from the R signal rykl from the LPF 121r is the G signal from which the noise component has been removed from the G signal gykl from the LPF 121g. The signal gzkl is output from the LPF 121b as a B signal bzkl obtained by removing noise components from the B signal bykl.

  Thereafter, the R signal rzkl from the LPF 121r is subtracted from the R signal rykl from the gradation converting circuit 11 in the subtractor 122r, and the G signal gzkl from the LPF 121g is subtracted from the G signal gykl from the gradation converting circuit 11 in the subtractor 122g. The subtractor 122b subtracts the B signal bzkl from the LPF 121b from the B signal bykl from the gradation conversion circuit 11. Therefore, the noise component nrkl (= rykl-rzkl) of the R signal from the subtractor 122r, the noise component ngkl (= gykl-gzkl) of the G signal from the subtractor 122g, and the noise component nbkl (= bykl-bzkl) is output.

In the coring unit 123r, the input noise component ngkl is subjected to coring processing according to the input / output relationship of FIG. 3 to be converted into the edge component erkl, and in the coring unit 123g, the input noise component ngkl is input. 3 is converted into an edge component egkl by the input / output relationship of FIG. 3, and the input noise component nbkl is subjected to the coring processing by the input / output relationship of FIG. 3 in the coring unit 123b. Are converted into edge components ebkl. That is, the coring unit 123r will be described as a representative example. The noise component nrkl is converted into the edge component erkl and output as shown in the following equation.
erkl = nrkl + α (nrkl ≦ −α)
erkl = 0 (-α <nrkl <α)
erkl = nrkl−α (α ≦ nrkl).

  When the edge components erkl, egkl, and ebkl are obtained in the coring portions 123r, 123g, and 123b in this way, the edge enhancement factor β is applied to the edge component erkl by the multiplication unit 124r, and the edge enhancement is applied to the edge component egkl by the multiplication unit 124g. The amount β is multiplied by the edge enhancement amount β by the edge component ebkl by the multiplication unit 124b. In addition, the edge component β × erkl edge-enhanced is added to the noise-removed R signal rzkl in the adder 125r, and the edge component β × egkl edge-enhanced to the noise-removed G signal gzkl in the adder 125g. Are added, and the adder 125b adds the edge component β × ebkl with edge enhancement to the B signal bzkl from which noise has been removed. Therefore, RGB signals with edge enhancement are output from the adders 125r, 125g, and 125b.

  At this time, the state of the RGB signal in each part of the coring circuit 12 changes as shown in FIG. That is, when an RGB signal as shown in FIG. 21A is input, noise is removed by the LPFs 121r, 121g, and 121b, and an RGB signal as shown in FIG. 21B is obtained. Further, in each of the subtractors 122r, 122g, 122b, the RGB signal from the gradation conversion circuit 11 is subtracted by the RGB signal from the LPF 121r, 121g, 121b, so that the noise component of the RGB signal as shown in FIG. Is obtained. Then, the noise components of the RGB signals are input to the coring circuits 123r, 123g, and 123b to obtain edge components as shown in FIG. 21D, and then the edges are enhanced by the multipliers 124r, 124g, and 124b. The amount β is multiplied and converted into an edge component as shown in FIG. The edge components of the RGB signals from the multipliers 124r, 124g, and 124b are added to the RGB signals from the LPFs 121r, 121g, and 121b by the adders 125r, 125g, and 125b, so that edge enhancement is performed as shown in FIG. RGB signals are output.

  In the present embodiment, the gradation conversion circuit 11 lowers the gradation of a region where the photoelectric conversion characteristic is a linear conversion characteristic and increases the gradation of a region where the photoelectric conversion characteristic is a logarithmic conversion characteristic. As a result, the gradation of the photoelectric conversion characteristics in the entire luminance range is made uniform. The tonality may be uniform, or the gradation of the photoelectric conversion characteristics in the entire luminance range may be uniform only by increasing the gradation of the area where the photoelectric conversion characteristics are logarithmic conversion characteristics. It doesn't matter.

  In the present embodiment, the photoelectric conversion characteristic of the solid-state imaging device 2 is changed from the linear conversion characteristic to the logarithmic conversion characteristic. However, the linear conversion characteristic may be different between the high luminance side and the low luminance side. . At this time, the photoelectric conversion characteristics are switched by switching the exposure amount to each pixel of the solid-state imaging device 2. That is, the photoelectric conversion characteristics can be switched by switching the aperture ratio of the lens by the diaphragm and the exposure time.

  Further, in the present embodiment, a single-plate solid-state image pickup device in which a plurality of types of color filters are provided in one solid-state image pickup device. For example, a three-plate solid-state image pickup device including a solid-state image pickup device for each RGB color filter As described above, a solid-state imaging device having the same color filter may be provided for each type of color filter. At this time, the color interpolation circuit can be removed. Further, the solid-state imaging device may be a solid-state imaging device that outputs a monochrome image signal without a color filter.

  Further, in the first and second examples of the dynamic range detection operation described above, the luminance distribution of the subject is confirmed based on the signal level of the RGB signal from the solid-state imaging device 2, but the luminance state of the subject is determined. The luminance distribution of the subject may be confirmed based on the signal from the photometric element to be measured. Furthermore, in the above-described imaging apparatus and imaging system, each parameter used for the signal processing operation is stored according to a plurality of types of photoelectric conversion characteristics. However, a reference value for the photoelectric conversion characteristics serving as a reference is set for each parameter. In addition to storing, the value of each parameter may be calculated from the stored reference value based on the relationship with the reference photoelectric conversion characteristic.

<Imaging system>
In the imaging device described above, various signal processing including white balance processing is performed in the imaging device, but data obtained by the imaging device is given without performing these various signal processes in the imaging device. An imaging system that performs the above-described various signal processes in a computer can also be configured. Hereinafter, an imaging system including such an imaging apparatus and a computer will be described with reference to the drawings.

  As shown in FIG. 22, the imaging system receives an RGB signal output from the imaging device 200 by imaging an object and generates RGB signals, and performs various signal processing including white balance processing. And a computer 250. At this time, the imaging apparatus 200 performs black reference correction and FPN correction on the RGB signal obtained by imaging the subject, and detects an AE evaluation value for setting the exposure amount. On the other hand, the computer 250 detects the WB evaluation value and performs various signal processing after the white balance processing on the RGB signal given from the imaging device 200.

  In such an imaging system, the imaging apparatus 200 includes an optical system 1, a solid-state imaging device 2, an amplifier 3, an AD conversion circuit 4, a black reference correction circuit 5, an FPN correction circuit 6, an aperture control unit 14, and a timing generation circuit 15. An AE evaluation value detection circuit 7a for detecting an AE evaluation value, an overall control unit 13a for controlling each block, a memory 201 for temporarily storing RGB signals corrected by the FPN correction circuit 6, and a memory An input / output interface 202 that reads out the RGB signals temporarily stored in 201 and transmits them to the outside.

  The computer 250 is a PC (Personal Computer), a PDA (Personal Digital Assistant) or the like, and includes a CPU (Central Processing Unit) 251 that performs data processing operations, a memory 252 that temporarily stores data, an application, It includes a hard disk 253 for storing data, an input unit 254 such as a keyboard and a mouse for inputting data, a display 255 for displaying video, and an input / output interface 256 for exchanging data with the outside.

  As described above, when the imaging apparatus 200 and the computer 250 are respectively configured, an application for performing various signal processing after the white balance processing on the RGB signals in the computer 250 is stored in the hard disk 253. By reading this application from the hard disk 253 and storing it in the memory 252, it is possible to perform the WB evaluation value detection operation and various signal processing after the white balance processing. Also, by starting this application, a user interface (UI) for the user to specify parameters in various signal processing is displayed on the display 255.

  Hereinafter, operations of the imaging apparatus 200 and the computer 250 in the imaging system will be described with reference to the drawings.

  First, in the imaging apparatus 200, when light enters the solid-state imaging device 2 as in the imaging apparatus of FIG. 1, an RGB signal that is an analog signal is generated, amplified by the amplifier 3, and then A / D. The signal is converted into a digital signal by the conversion circuit 4. The RGB signal which is the digital signal is obtained by subtracting the FPN component stored in the FPN correction circuit 6 after the black level which is the lowest luminance value is corrected to the reference value in the black reference correction circuit 5, thereby obtaining the FPN. Ingredients are removed. Then, the RGB signal from which the FPN component has been removed is stored in the memory 201. When the RGB signal from which the FPN component is removed is given to the AE evaluation value detection circuit 7a when operating in this way, the luminance range of the subject is confirmed and the aperture of the diaphragm 1a set by the diaphragm controller 14 is determined. An AE evaluation value is detected.

In addition, the imaging apparatus 200 checks the dynamic range information indicating the photoelectric conversion characteristics set for the solid-state imaging device 2 when performing the imaging operation for each captured image, and stores it in the memory 201 together with the RGB signal of the image. Store. This dynamic range information includes at least one of the following four pieces of information (1) and (2) and at least one of (3) and (4), or (1) to Only at least one of (4) is provided.
(1) Inflection point Vth of the photoelectric conversion characteristics of the solid-state imaging device 2
(2) Mode number M for setting the photoelectric conversion characteristics of the solid-state imaging device 2
(3) Coefficients A, C, γ, and δ representing the photoelectric conversion characteristics of the solid-state imaging device 2 (this approximate expression will be described later)
(4) Imaging conditions of the imaging apparatus 200

  In these dynamic range information, the inflection point Vth of the photoelectric conversion characteristic of the solid-state image sensor 2 indicates the signal level of the RGB signal when the photoelectric conversion characteristic logarithmic conversion characteristic of the solid-state image sensor 2 is changed to the linear conversion characteristic. Further, the mode number M for setting the photoelectric conversion characteristics of the solid-state imaging device 2 is set on the computer 250 side, so that the photoelectric conversion characteristics of the solid-state imaging device 2 of the imaging apparatus 200 are set. Is. The coefficients of the approximate expression indicating the photoelectric conversion characteristics of the solid-state imaging device 2 are the coefficients A and C in the equation (a) V = A × L + C representing the linear conversion characteristic, and the equation (b) representing the logarithmic conversion characteristic V = It is constituted by coefficients γ and δ in γ × ln (L) + δ. When the imaging conditions of the imaging apparatus 200 are set, the voltage difference ΔVPS between the voltage values VH and VL of the signal φVPS, the exposure time tx to the solid-state imaging device 2, the aperture ratio rx of the diaphragm 1a, and the solid-state imaging device 2 The amplification factor ax of the internal amplifier and the amplifier 3 is dynamic range information. When the imaging conditions of the imaging apparatus 200 are dynamic range information, the relationship between the coefficient A in (A) and (exposure time × aperture ratio of the diaphragm 1a × amplification factor) is known, and (B) The voltage difference ΔVPS between the coefficient δ and the signal φVPS and the relationship between the coefficient γ and the amplification factor are known.

  When the input unit 254 is operated to read the application from the hard disk 253 and the operation designated by the application is started, the computer 250 displays a UI on the display 255 as shown in FIG. A screen is displayed. In the screen of FIG. 23, an image display area X1 for displaying an image obtained by imaging, an image processing condition setting area X2 for setting image processing conditions such as image sharpness and color shading, and the like. Color correction is performed for an image acquisition button X3 for instructing the start of capturing an image captured by the image capturing apparatus 200, a WB setting region X4 for setting a photoelectric conversion characteristic to be used as a reference when performing white balance processing. A gradation characteristic display area X5 indicating gradation characteristics after gradation conversion performed on the RGB signal signals is displayed.

  At this time, when the sharpness of the image is changed by operating the scroll bar SB1 in the image processing condition setting region X2, the edge enhancement amount β for setting the edge component in the coring circuit 12 is changed. When the color of the image is changed by operating the scroll bar SB2, the matrix coefficients a1 to a3, b1 to b3, and c1 to c3 in the color correction process are changed. At this time, the edge enhancement amount obtained based on the dynamic range information given from the imaging device 200 or a matrix coefficient in the color correction processing may be changed as a reference, or the edge enhancement amount of a predetermined value set in advance or It may be changed based on the matrix coefficient in the color correction process.

  In addition, when the three option buttons Ob1 to Ob3 are designated in the image processing condition setting area X2, the edge enhancement amount β and the matrix coefficients a1 to a3, b1 to b3, and c1 designated by the scroll bars SB1 and SB2 respectively. An image to be a target using ~ c3 is designated. That is, when the edge enhancement amount β and the matrix coefficients a1 to a3, b1 to b3, and c1 to c3 specified by the scroll bars SB1 and SB2 are used, and the option button Ob1 is specified, the obtained image is photoelectrically converted. When the option button Ob2 is designated, the image is obtained only when the photoelectric conversion characteristic is the linear conversion characteristic. Further, when the option button Ob3 is designated, it indicates that the obtained image is performed only when the photoelectric conversion characteristic is the logarithmic conversion characteristic.

  In addition, by specifying the five option buttons Wb1 to Wb5 in the WB setting area X4, WB evaluation values wr and wb for performing white balance processing are set. That is, when the option button Wb1 is designated, it is set to perform white balance processing using the WB evaluation values wr and wb that are automatically set based on the RGB signals given from the imaging device 200, and the option button Wb2 When .about.Wb5 is designated, white balance processing is set to be performed using WB evaluation values wr and wb which are preset values. When the option buttons Wb2 to WB5 are designated, the WB evaluation values wr and wb corresponding to the image under the fluorescent lamp, the WB evaluation values wr and wb corresponding to the image under the incandescent lamp, and the outdoor sunny day, respectively. The WB evaluation values wr and wb corresponding to the image at, and the WB evaluation values wr and wb corresponding to the image at the time of outdoor cloudy weather are set.

  In the gradation characteristic display area X5, the displayed gradation characteristic indicates a relationship before and after conversion when gradation conversion is performed on each of the RGB signals subjected to color correction processing. Then, by changing the position of the inflection point that switches from the linear conversion characteristic to the logarithmic conversion characteristic in the displayed gradation characteristic, in the dynamic range, the magnitude of the gradation in the linear region that is a signal by the linear conversion characteristic The gradation size in the logarithmic region that becomes a signal based on the logarithmic conversion characteristic can be set. The position of the inflection point of the gradation characteristic may be set with reference to a position obtained based on dynamic range information given from the imaging device 200, or set with reference to a predetermined position set in advance. It doesn't matter if it is done. Further, a plurality of LUTs indicating gradation characteristics may be provided, and the LUT displayed in the gradation characteristic display region X5 may be changed by switching the LUTs.

  As described above, when the application is started in the computer 250 and the UI as shown in FIG. 23 is displayed on the display 255, the computer 250 operates according to the flowchart of FIG. It is displayed on the display 255. First, in the computer 250, when the input unit 254 is operated and the image acquisition button X3 in the UI screen is pressed (STEP 1), the imaging apparatus 200 in which the input / output interface 202 is connected to the input / output interface 256 is displayed. A line with the imaging device 200 is secured (STEP 2). At this time, a line can be secured by repeating the communication request and response between the imaging apparatus 200 and the computer 250.

  Then, it is confirmed whether or not a line with the imaging apparatus 200 is secured (STEP 3), and when it is confirmed that a line with the imaging apparatus 200 is secured (Yes), the imaging apparatus 200 is imaged. The output of the received RGB signal is requested (STEP 4). At this time, when receiving an RGB signal output request from the computer 250, the imaging apparatus 200 reads the RGB signal and dynamic range information for one frame stored in the memory 201 and outputs them from the input / output interface 202. . At this time, when frame data based on RGB signals for one frame is generated and output, dynamic range information is added to the header portion of the frame data.

  When the frame data from the imaging device 200 is received by the input / output interface 256 and stored in the temporary storage area of the hard disk 253 (STEP 5), the dynamic range information is acquired from the header portion of the frame data (STEP 6). . Based on the acquired dynamic range information, parameters used for each signal processing are set, and display corresponding to the set parameters is performed on the UI image of the display 255 (STEP 7).

  That is, WB detection values wr and wb for white balance processing, matrix coefficients a1 to a3, b1 to b3 and c1 to c3 for color correction processing, tone characteristics for tone conversion processing, and edge enhancement amount β for coring processing, respectively. In addition to setting, display corresponding to each set parameter is performed in each of the image processing condition setting area X2, the WB setting area X4, and the WB setting area X4.

  Further, when each parameter is set in this way, in order to set the color correction coefficient used in the color correction processing for each RGB signal of each pixel, the inflection point Vth obtained from the dynamic range information, the mode number M, the coefficient A , C, γ, δ, or the imaging conditions of the imaging apparatus 200, the photoelectric conversion characteristics are confirmed. Then, the color correction coefficient corresponding to the confirmed photoelectric conversion characteristic is selected and read from the color correction coefficient corresponding to each photoelectric conversion characteristic stored together with the application of the hard disk 253 in advance.

  At this time, when the inflection point Vth is included in the dynamic range information, the photoelectric conversion characteristics are recognized by the inflection point Vth, and when the coefficients A, C, γ, and δ are included in the dynamic range information, ), (B) to recognize photoelectric conversion characteristics. When the mode number M is included in the dynamic range information, the inflection point Vth corresponding to the mode number M stored together with the application of the hard disk 253 is read in advance, and the photoelectric conversion characteristics are recognized by the inflection point Vth.

When the shooting conditions (voltage difference ΔVPS, exposure time tx, aperture ratio rx, amplification factor ax) are included in the dynamic range information, the coefficients A0, C0, γ0 serving as reference values, the exposure time tx0, the aperture ratio rx0, The amplification factor ax0 is stored together with the application of the hard disk 253 in advance. First, the coefficients A, C, γ, and δ representing the photoelectric conversion characteristics are set by calculating based on the following formulas (c) to (f). . Note that β0 is a value corresponding to the voltage difference ΔVPS of the signal φVPS stored together with the application of the hard disk 253 in advance. Thereafter, the photoelectric conversion characteristics are recognized from the equations (a) and (b) confirmed by the coefficients A, C, γ, and δ.
A = A0 × (tx / tx0) × (rx / rx0) × (ax / ax0) (C)
C = C0 (D)
γ = γ0 × (tx / tx0) (e)
δ = δ0 × (tx / tx0) (f)

  Further, in order to confirm the gradation characteristics used in the gradation conversion process, the inflection point Vth obtained from the dynamic range information, the mode number M, the coefficients A, C, γ, δ, or the imaging conditions of the imaging device 200 A threshold level V1 (see FIG. 6) for determining the photoelectric conversion characteristic of each RGB signal of each pixel is set. Also at this time, when the photoelectric conversion characteristics are confirmed from the dynamic range information as in the case of the color correction process described above, the inflection point Vth in the confirmed photoelectric conversion characteristics is obtained, and the obtained inflection point Vth is directly used as the threshold level. Set to V1.

  In order to confirm the LPF coefficient and the coring coefficient α and the edge enhancement amount β used in the coring process, the inflection point Vth, the mode number M, the coefficients A, C, γ, δ obtained from the dynamic range information, or From the imaging conditions of the imaging apparatus 200, the photoelectric conversion characteristics of each RGB signal of each pixel are confirmed. Also at this time, the photoelectric conversion characteristics are confirmed from the dynamic range information in the same manner as in the case of the color correction process described above. Then, the LPF coefficient, the coring coefficient α, and the edge enhancement amount β corresponding to the confirmed photoelectric conversion characteristic are stored in advance as the LPF coefficient, the coring coefficient α, and the edge enhancement corresponding to each photoelectric conversion characteristic stored together with the application of the hard disk 253 Select from the amount β and read.

  When the parameters set in this way are given to the application stored in the memory 252, the signal processing according to the set parameters is performed on the RGB signals obtained from the frame data received in STEP 5 by this application. Performed (STEP 8). That is, each signal processing is performed on the RGB signal obtained from the frame data in the order of white balance processing, color interpolation processing, color correction processing, gradation conversion processing, and coring processing. Each signal processing is the same as the signal processing in the imaging apparatus shown in FIG.

  The frame data based on the RGB signals subjected to the signal processing is stored in the hard disk 253, and an image based on the frame data is displayed in the image display area X1 of the display 255 (STEP 9). Then, by operating the input unit 254, it is confirmed whether or not the operation of the image processing condition setting area X2, the WB setting area X4, and the WB setting area X4 has been instructed to change each parameter (STEP 10). . If parameter change is instructed (Yes), each parameter is changed in accordance with the instructed change amount (STEP 11), and then the process proceeds to STEP 8.

  If it is not confirmed in STEP 3 that the line has been secured (No), a display indicating that the line is not connected is displayed on the display 255 (STEP 12), and the process proceeds to STEP 2. Further, in STEP 10, when the instruction for changing each parameter is not confirmed (No), the operation is terminated.

  As described above, according to the present system, each signal processing with a calculation load is performed on the computer 250 side, so that the configuration of the imaging apparatus 200 is simplified and the processing speed can be increased. In this system, the signal output from the imaging apparatus 200 to the computer 250 may be not only frame data based on the above-described still image but also frame data or stream data based on a moving image. At this time, when a moving image is formed by a plurality of still image frame data, dynamic range information is recorded in the header of each frame data, and the moving image is generated by stream data that performs image compression on the time axis. In this case, the dynamic range information is transmitted as information different from the stream data.

  Further, in the imaging apparatus 200, the overall control unit 13a converts the format of frame data or stream data based on RGB signals transmitted to the computer 250 into a compressed image such as JPEG, motion JPEG, or MPEG, and outputs the compressed image from the input / output interface 202. It does n’t matter what you do. Further, in the computer 250, the UI displayed on the display 255 displays a histogram of the image displayed in the image display area X1, and a display for identifying the pixel area based on the logarithmic conversion characteristic and the pixel area based on the linear conversion characteristic. You may make it perform.

  In the imaging system of FIG. 22, the imaging apparatus 200 includes the black reference correction circuit 5 and the FPN component correction circuit 6. However, the imaging apparatus 200 has the black reference correction circuit 5 and the FPN component from the configuration of FIG. The correction circuit 6 may be omitted, and the computer 250 may perform black reference correction processing and FPN correction processing. At this time, the black level for performing the black reference correction process and the FPN component for performing the FPN correction process may be transmitted together with the dynamic range information.

  Furthermore, when the black reference correction process and the FPN correction process are performed in the computer 250 as described above, the computer 250 may store the black level and the FPN component in the hard disk 253 together with the application. At this time, a plurality of black levels are also stored for the inflection point Vth of the photoelectric conversion characteristics, and the inflection points of the photoelectric conversion characteristics obtained from the prestored black level by the dynamic range information as described above. A black level corresponding to Vth is selected. Further, since the FPN component differs depending on the solid-state imaging device 2 in each imaging device 200, the FPN component is stored for each imaging device 200.

  Further, when the computer 250 performs white balance processing, the RGB signal is converted into a signal whose photoelectric conversion characteristics are all logarithmic conversion characteristics or a signal whose photoelectric conversion characteristics are all linear conversion characteristics, and then the white balance processing is performed. It doesn't matter what you do. That is, when all the photoelectric conversion characteristics are converted into a signal with logarithmic conversion characteristics, the signal value obtained by the linear conversion characteristics is converted into a signal value with logarithmic conversion characteristics, and all the photoelectric conversion characteristics are signals with the linear conversion characteristics. In the case of conversion to, the signal value obtained by the logarithmic conversion characteristic is converted to a signal value by the linear conversion characteristic. Note that the calculation for such conversion processing may be performed with reference to the LUT, or may be performed by an arithmetic expression.

  In the imaging system of FIG. 22, the input / output interface 202 of the imaging device 200 and the input / output interface 256 of the computer 250 are connected by communication, so that frame data obtained by imaging with the imaging device 200 is transferred to the computer 250. Although the imaging apparatus 200 includes a recording unit that records frame data or stream data on a recording medium such as a flexible disk, a compact flash, or a memory card, the computer 250 uses the frame data or stream of the recording medium. A reproducing unit for reading out data may be provided. In this way, frame data or stream data obtained by imaging can be transferred from the imaging device 200 to the computer 250 via the recording medium.

These are block diagrams which show the structure of the imaging device which is embodiment of this invention. These are figures which show the arrangement | sequence of the color filter provided in a solid-state image sensor. These are graphs showing level conversion characteristics for edge components in a coring circuit. FIG. 3 is a circuit block diagram for explaining the overall configuration of the solid-state imaging device. These are circuit diagrams which show one structural example of the pixel which comprises the solid-state image sensor of FIG. These are timing charts showing the operation of the pixel of FIG. These are graphs showing the relationship between the luminance of a subject and the output of a pixel. These are flowcharts which show an example of the dynamic range detection operation by the AE / WB evaluation value detection circuit. These are graphs showing the relationship between luminance values of RGB signals and signal levels. FIG. 6 is a diagram showing a histogram based on a luminance value of a subject. These are graphs showing the relationship between the luminance of a subject and the output of a pixel. These are the flowcharts which show the other example of the detection operation of the dynamic range by an AE * WB evaluation value detection circuit. FIG. 3 is a block diagram showing an internal configuration of a black reference correction circuit. FIG. 3 is a block diagram showing an internal configuration of a color correction circuit. FIG. 3 is a block diagram showing an internal configuration of a gradation conversion circuit. These are graphs showing the relationship between luminance values of RGB signals and signal levels. These are graphs showing the relationship between input / output and photoelectric conversion characteristics in the gradation conversion circuit. These are graphs showing the relationship between input / output and photoelectric conversion characteristics in the gradation conversion circuit. These are graphs showing the relationship between input / output and photoelectric conversion characteristics in the gradation conversion circuit. These are block diagrams which show the internal structure of a coring circuit. These are the figures for demonstrating the state transition of the signal in each part of a coring circuit. These are block diagrams which show the structure of the imaging system which is embodiment of this invention. FIG. 24 is a diagram showing an example of a screen displayed on the computer of the imaging system in FIG. 22. FIG. 24 is a flowchart showing signal processing operations in the computer of the imaging system of FIG. 22;

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Optical system 2 Solid-state image sensor 3 Amplifier 4 AD conversion circuit 5 Black reference correction circuit 6 FPN correction circuit 7 AE / WB evaluation value detection circuit 8 WB control circuit 9 Color interpolation circuit 10 Color correction circuit 11 Gradation conversion circuit 12 Coring Circuit 13 Overall control unit 14 Aperture control unit 15 Timing generation circuit

Claims (14)

A solid-state imaging device having a plurality of pixels, and a signal processing circuit for processing an image signal output from the solid-state imaging device, and selecting a photoelectric conversion characteristic of the solid-state imaging element from the plurality of photoelectric conversion characteristics In the imaging device to switch,
A luminance distribution of a subject is detected from an image signal output from the solid-state imaging device, and a photoelectric conversion characteristic corresponding to the detected luminance distribution of the subject is selected from the plurality of photoelectric conversion characteristics, and the solid-state imaging device While having a photoelectric conversion characteristic switching unit set as a photoelectric conversion characteristic,
The signal processing circuit removes fixed pattern noise from the signal, and a black reference correction circuit that performs reference value correction for converting a signal level corresponding to the lowest luminance value in the image signal output from the solid-state imaging device to a predetermined reference value As a parameter for processing the fixed pattern noise correction circuit and the image signal, a plurality of types of parameters corresponding to each of the plurality of photoelectric conversion characteristics,
The image signal when detecting the luminance distribution is image data after black reference correction and fixed pattern noise cancellation,
An imaging apparatus, wherein a parameter corresponding to the selected photoelectric conversion characteristic is selected from the plurality of types of parameters and set as a processing parameter in the signal processing circuit. The imaging apparatus according to claim 1, wherein the black reference correction circuit includes a plurality of types of parameters corresponding to the photoelectric conversion characteristics as parameters for performing reference value correction. The black reference correction circuit is
A parameter setting unit for setting a signal level for the minimum luminance value, which is a parameter for performing the reference value correction according to the selected photoelectric conversion characteristic;
A subtractor that performs reference value correction by subtracting the signal level for the lowest luminance value set by the parameter setting unit from the image signal output from the solid-state image sensor,
3. The imaging according to claim 2, wherein the black reference correction circuit includes a plurality of types of signal levels corresponding to the photoelectric conversion characteristics as signal levels for the lowest luminance value for performing reference value correction. 4. apparatus. The signal processing circuit includes a gradation conversion circuit for converting the gradation of the image signal;
The imaging according to any one of claims 1 to 3, wherein the gradation conversion circuit includes a plurality of types of parameters corresponding to the photoelectric conversion characteristics as parameters for performing gradation conversion. apparatus. The gradation conversion circuit is
A parameter setting unit for setting parameters for performing the gradation conversion according to the selected photoelectric conversion characteristics;
A gradation conversion unit that performs gradation conversion of the input image signal in accordance with the parameters set by the parameter setting unit;
The imaging apparatus according to claim 4, further comprising: 6. The imaging apparatus according to claim 1, wherein a signal output from the black reference correction circuit is input to the fixed pattern noise correction circuit. The signal processing circuit includes a coring circuit that removes noise components of the image signal and performs edge enhancement,
The coring circuit, as a parameter for the noise component removing image pickup apparatus according to any one of claims 1 to 6, characterized in that it comprises a plurality of types of parameters corresponding to the respective photoelectric conversion characteristics . The signal processing circuit includes a coring circuit that removes noise components of the image signal and performs edge enhancement,
The coring circuit, as a parameter for performing edge enhancement, the imaging device according to any one of claims 1 to 6, characterized in that it comprises a plurality of types of parameters corresponding to the respective photoelectric conversion characteristics. The signal processing circuit includes a coring circuit that removes noise components of the image signal and performs edge enhancement,
The coring circuit, as a parameter for the noise component removing and edge enhancement, according to any one of claims 1 to 6, characterized in that it comprises a plurality of types of parameters corresponding to the respective photoelectric conversion characteristics Imaging device. The coring circuit is
A low pass filter for removing noise components from the input image signal;
A subtractor that subtracts the image signal that has passed through the low-pass filter from the input image signal to extract the noise component;
Coring the noise component obtained by the subtractor to remove a noise component other than the edge portion;
A multiplier that multiplies the edge component obtained by the coring unit by an edge enhancement amount that is a predetermined amount;
An adder that adds the edge component multiplied by the edge enhancement amount by the multiplier to the image signal from which noise components have been removed by the low-pass filter, and outputs an edge-enhanced image signal;
With
A plurality of types of parameters set according to the photoelectric conversion characteristics remove matrix-like low-pass filter coefficients for removing noise components in the low-pass filter and noise components other than edge portions in the coring circuit. coring factor for, and, an imaging device according to any one of claims 7 to claim 9, characterized in that either of the edge enhancement amount of said multiplier. The image signal output from the solid-state imaging device is a plurality of types of color signals, and the plurality of types of color signals are generated for one pixel and supplied to the signal processing circuit.
The signal processing circuit includes a color correction circuit that performs color correction by giving a correlation with another color signal for each type of the color signal of each pixel,
The color correction circuit, as a parameter for performing color correction, image pickup apparatus according to any one of claims 1 to 10, characterized in that it comprises a plurality of types of parameters corresponding to the respective photoelectric conversion characteristics. The color correction circuit is
For each type of color signal,
A parameter setting unit for setting a parameter for performing the color correction according to the selected photoelectric conversion characteristic;
A multiplier that multiplies each of the color signals of the same pixel as the input color signal by the parameter set by the parameter setting unit;
An adder that adds all the color signals of the same pixel multiplied by the parameter by the multiplier and corrects the color of the input color signal;
The imaging apparatus according to claim 11 , further comprising: In the photoelectric conversion characteristic switching unit, when it is confirmed that an image signal having a saturation level that maximizes the signal level is output from the solid-state imaging device by a number equal to or more than a first predetermined number of pixels, the dynamic range of the solid-state imaging device is expanded. imaging device according to any one of claims 1 to 12, characterized by selecting the photoelectric conversion characteristics. In the photoelectric conversion characteristic switching unit, when the maximum signal level of the image signal output from the solid-state image sensor is a value lower by a predetermined signal level than the saturation level at which the signal level is maximum, the solid-state image sensor imaging device according to any one of claims 1 to 13, characterized by selecting the photoelectric conversion characteristics to narrow the dynamic range of the.

Images