JP4479429B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus having a solid-state imaging device capable of switching a plurality of photoelectric conversion characteristics.

  Conventionally, in a solid-state imaging device that performs a linear conversion operation that linearly converts the amount of incident light, its dynamic range is as narrow as two orders of magnitude, so when imaging a subject that constitutes a luminance distribution in a wide luminance range, other than the dynamic range The luminance information in the range is not output. Further, as a conventional solid-state imaging device, there is one that performs a logarithmic conversion operation for logarithmically converting the amount of incident light (see Patent Document 1). In this solid-state imaging device, the dynamic range is as wide as 5 to 6 digits. Therefore, even if an image of a subject constituting a luminance distribution in a slightly wide luminance range is captured, all luminance information in the luminance distribution is converted into an electrical signal. Can be output. However, since the imageable area becomes wider with respect to the luminance distribution of the subject, an area without luminance data is formed in the low luminance area or the high luminance area in the imageable area. On the other hand, the present applicant has proposed what can switch between the above-described linear conversion operation and logarithmic conversion operation (see Patent Document 2).

In addition, when the linear conversion operation and the logarithmic conversion operation of the solid-state imaging device can be switched, the applicant of the present invention performs signal processing on the image signal obtained by the linear conversion operation and the image signal obtained by the logarithmic conversion operation. In addition, there is proposed an imaging apparatus that can process an image signal having one photoelectric conversion characteristic to the other photoelectric conversion characteristic so that it can be processed by a common circuit (see Patent Document 3). In such an imaging apparatus, for example, even when a solid-state imaging device that switches to an optimum photoelectric conversion characteristic for each frame is provided, all output image signals can be converted into one of the photoelectric conversion characteristics. Therefore, it is not necessary to provide a signal processing circuit corresponding to each photoelectric conversion characteristic as a subsequent signal processing circuit.
JP-A-11-313257 JP 2002-77733 A JP 2002-33961 A

  However, in the imaging device in Patent Document 3, the photoelectric conversion characteristics are not confirmed for each image signal of each pixel, and therefore, in the same frame as the image signal output from the solid-state imaging device as in Patent Document 2. It is impossible to deal with signals having different photoelectric conversion characteristics. In addition, image signals with different photoelectric conversion characteristics are always converted to image signals with the same photoelectric conversion characteristics, and then various signal processing such as edge enhancement and gamma correction are performed by a common circuit. Therefore, when various signal processing is performed with the same parameters, an image with poor contrast and color balance may be obtained.

  In view of such problems, the present invention provides an imaging apparatus that performs signal processing using parameters according to the photoelectric conversion characteristics of image signals having different photoelectric conversion characteristics obtained by imaging in the same frame. With the goal.

In order to achieve the above object, an imaging apparatus of the present invention performs processing of a solid-state imaging device having a plurality of pixels whose photoelectric conversion characteristics are switched according to a plurality of luminance ranges, and an image signal output from the solid-state imaging device. the example Bei and a signal processing circuit, wherein the signal processing circuitry, as a parameter for processing the image signal, which includes a plurality of types of parameters corresponding to each luminance range, is processed by the signal processing circuit for performing In the imaging apparatus for confirming the luminance range of the image signal and selecting a parameter corresponding to the photoelectric conversion characteristic of the confirmed luminance range from the plurality of types of parameters and setting the parameter as a processing parameter in the signal processing circuit , the solid state The image signal output from the image sensor is a plurality of types of color signals, and the plurality of types of color signals are generated for one pixel to generate the signal. A gradation conversion circuit provided with a first parameter for converting the gradation of the image signal and performing gradation conversion, and removing a noise component of the image signal. A color is obtained by correlating a noise removal circuit having a second parameter for performing edge enhancement and noise component removal and / or edge enhancement, and another color signal for each type of the color signal of each pixel. And a color correction circuit including a third parameter for performing color correction, and at least one of the first parameter, the second parameter, and the third parameter is included in each luminance range. It is characterized by a plurality of types of parameters set accordingly .

In such an imaging apparatus, when the luminance range is confirmed from the signal level of the input image signal, the photoelectric conversion characteristics in the luminance range are recognized. And the parameter according to the recognized said photoelectric conversion characteristic is set with respect to the said signal processing circuit. Therefore, the signal processing circuit can optimally perform signal processing on the input image signal by performing signal processing using a parameter corresponding to the photoelectric conversion characteristic. The image signal output from the solid-state imaging device is a plurality of types of color signals, and the plurality of types of color signals are generated for one pixel and applied to the signal processing circuit. A gradation conversion circuit having a first parameter for converting the gradation of the image signal and performing gradation conversion, a noise component of the image signal, edge enhancement and noise component removal, and / or Alternatively, a noise removal circuit having a second parameter for performing edge enhancement and a color correction by giving a correlation with another color signal for each type of the color signal of each pixel and color correction A color correction circuit having a third parameter, wherein at least one of the first parameter, the second parameter, and the third parameter is It has a plurality of types of parameters to be set according to the range. Therefore, since optimum signal processing can be performed for each photoelectric conversion characteristic, it is possible to obtain a highly accurate image with good contrast and high SN ratio in the entire luminance range.

Here, by setting the first parameter for the gradation characteristics in accordance with the photoelectric conversion characteristics of each luminance range, the image signal can have uniform gradation characteristics in the entire luminance range. At this time, a parameter for increasing the gradation is set for a photoelectric conversion characteristic with low gradation, and a parameter for decreasing the gradation is set for a photoelectric conversion characteristic with high gradation. In this way, uniform gradation may be provided in the entire luminance range. Further, when a plurality of kinds of color signals are generated for the same pixel as the image signal, the luminance range may be confirmed for each color signal.

Also, by setting the second parameter for noise component removal or edge enhancement according to the photoelectric conversion characteristics of each luminance range, the image signal can be converted into a high-accuracy image signal with good contrast in the entire luminance range.

Also, by setting the third parameter for color correction according to the photoelectric conversion characteristics of each luminance range, the image signal can be converted into an image signal that has been subjected to equivalent color enhancement in the entire luminance range. This color correction is a process for enhancing a certain hue. The solid-state imaging device may be a single-plate solid-state imaging device, and may include a color interpolation circuit that generates the plurality of types of color signals for each pixel. The solid-state image sensor may be a multi-plate solid-state image sensor.

In addition, the gradation conversion circuit includes a luminance range detection unit that confirms a luminance range of the input image signal, and the gradation conversion unit that performs gradation conversion according to the luminance range detected by the luminance range detection unit. A parameter setting unit that sets a first parameter; and a tone conversion unit that performs tone conversion of the input image signal in accordance with the first parameter set by the parameter setting unit.

The noise removal circuit also includes a low-pass filter that removes a noise component from the input image signal, and a subtraction that subtracts the image signal that has passed through the low-pass filter from the input image signal to extract the noise component. A coring unit that removes noise components other than the edge portion by coring the noise component obtained by the subtractor, and an edge that is a predetermined amount in the edge component obtained by the coring portion An edge-enhanced image signal obtained by adding the edge component multiplied by the edge enhancement amount by the multiplier to the image signal from which the noise component has been removed by the multiplier and a multiplier for multiplying the enhancement amount An adder that outputs a matrix, and the second parameter is a matrix-like low pass for removing noise components in the low pass filter Filter coefficients, and, coring coefficient for eliminating the noise components other than the edge portion by the coring portion, and is one of the edge enhancement amount of said multiplier.

At this time, the low-pass filter coefficient is an n × n matrix coefficient, for example, a 3 × 3 or 5 × 5 matrix coefficient is used. Further, the luminance range of the image signal may be confirmed for each pixel outside the noise removal circuit. When a plurality of types of color signals are generated for the same pixel as the image signal, the luminance range is confirmed for each color signal, and the same parameters are used for all types of color signals of the same pixel. It does n’t matter.

Further, the color correction circuit, for each type of the color signal, a luminance-range detector to check the brightness range of the inputted color signals in response to the luminance range detected in the luminance-range detector color A parameter setting unit for setting the third parameter for correction, and multiplication for multiplying the third parameter set by the parameter setting unit for each of all types of color signals of the same pixel as the input color signal And an adder that adds the color signals of all the same pixels multiplied by the third parameter by the multiplier and corrects the color of the input color signal.

  Further, in each of the above-described solid-state imaging devices, a logarithmic conversion characteristic in which an electrical signal logarithmically converted with respect to an incident light amount is used as the image signal, and a linear signal in which the electric signal linearly converted with respect to the incident light amount is used as the image signal. The photoelectric conversion characteristics may be switched according to the luminance between the conversion characteristics. Further, it may be provided with a different linear conversion characteristic for each luminance range.

  Further, the photoelectric conversion characteristics in the entire luminance range of the solid-state imaging device can be selected and switched, and a plurality of types of parameters corresponding to the respective luminance ranges are selected based on the selected photoelectric conversion characteristics in the entire luminance range. May be set. That is, the luminance range is divided into m ranges, and when there are n photoelectric conversion characteristics in the entire luminance range that can be selected, n groups of m parameters are provided for each photoelectric conversion characteristic. Therefore, n × m types of parameters are provided.

According to the present invention, since the signal processing parameters corresponding to the photoelectric conversion characteristics of the image signal are set for each pixel, signal processing that is optimal for the photoelectric conversion characteristics can be performed on the image signal. . Therefore, it is possible to obtain a highly accurate image with good contrast and high SN ratio in the entire luminance range. Here, since the first parameter for the gradation characteristic can be set according to the photoelectric conversion characteristic of each luminance range, the image signal can have uniform gradation in the entire luminance range. Further, since the second parameter for noise component removal or edge enhancement can be set according to the photoelectric conversion characteristics of each luminance range, the image signal can be converted into a high-accuracy image signal with good contrast in the entire luminance range. it can. Further, since the third parameter for color correction can be set according to the photoelectric conversion characteristics of each luminance range, the image signal can be converted into an image signal that has been subjected to equivalent color enhancement in the entire luminance range.

<Configuration of imaging device>
A configuration of an imaging apparatus according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing the internal configuration of the imaging apparatus of the present invention.

  1 is output from an optical system 1 composed of a plurality of lenses, a solid-state image sensor 2 that converts an incident light amount of light incident through the optical system 1 into an electrical signal, and the solid-state image sensor 2. An amplifier 3 for amplifying the electric signal, an AD conversion circuit 4 for converting the electric signal amplified by the amplifier 3 into a digital signal, a black reference correction circuit 5 for setting a minimum level of the digital signal from the AD conversion circuit 4, An FPN correction circuit 6 that removes FPN (Fixed Pattern Noise) due to the sensitivity of each pixel of the solid-state imaging device 2 from the digital signal corrected by the black reference value in the black reference correction circuit 5, and the FPN correction circuit 6 An AE / WB evaluation value detection circuit 7 for detecting respective evaluation values for performing automatic exposure control (AE) and white balance (WB) from the digital signal from which FPN has been removed at Based on the WB control circuit 8 that performs correction for each color signal so that the color balance of the digital signal from which FPN is removed by the FPN correction circuit 6 is obtained, and the color signals of a plurality of adjacent pixels output from the WB control circuit 8. The color interpolation circuit 9 that interpolates each color signal, the color correction circuit 10 that corrects the hue of each color signal output from the color interpolation circuit 9 for each pixel by another color signal, and the color correction circuit 10 output the color signal. A gradation conversion circuit 11 that performs gradation conversion of a digital signal, a coring circuit 12 that performs processing such as edge enhancement on the digital signal output from the gradation conversion circuit 11, and an overall control unit 13 that controls each block An aperture control unit 14 for controlling the exposure amount by the aperture 1 a provided in the optical system 1, and a clock for supplying a clock for operation timing to the solid-state imaging device 2 and the AD conversion circuit 4. It includes a timing generating circuit 15, a.

  In the imaging apparatus configured as described above, when light is incident on the solid-state imaging device 2 having a different color filter for each pixel via the optical system 1, a photoelectric conversion operation is performed in each pixel, and each pixel An analog signal that is a different color signal is output every time. That is, as shown in FIG. 2, when a color filter having a Bayer array with RGB is provided in the solid-state imaging device 2, an R signal representing red is a color that becomes G from a pixel provided with an R color filter. A G signal representing green is outputted from the pixel provided with the filter, and a B signal representing blue is outputted from the pixel provided with the color filter B. As will be described later, in the solid-state imaging device 2, the luminance position at which the linear conversion operation and the logarithmic conversion operation are switched is changed by changing the driving condition by the overall control unit 13. As a result, the dynamic range of the solid-state imaging device 2 changes. The dynamic range becomes wider as the switching point becomes the higher luminance side.

  The R signal, the G signal, and the B signal that are serially output from the solid-state imaging device 2 are amplified by the amplifier 3 and then converted into a digital signal by the AD conversion circuit 4. When the R signal, the G signal, and the B signal thus converted into digital signals are supplied to the black reference correction circuit 5, the black level that becomes the minimum luminance value based on the dynamic range control signal supplied from the overall control unit 13 is obtained. Is corrected to the reference value (0). That is, since the black level varies depending on the dynamic range of the solid-state imaging device 2, the signal level that becomes the black level is subtracted from the signal levels of the R signal, the G signal, and the B signal output from the AD conversion circuit 4. Thus, the reference value correction is performed.

  The FPN component is removed from the R signal, the G signal, and the B signal subjected to the black reference correction by subtracting the FPN component stored in the FPN correction circuit 6. This FPN component is an offset variation caused by a threshold variation of MOS transistors constituting each pixel in the solid-state imaging device 2. When extracting the FPN component, for each of the RGB signals, an offset based on the difference in transmittance of the color filter is subtracted from the image signal for each pixel output from the solid-state imaging device 2 during uniform light irradiation. At this time, the offset based on the transmittance of each color filter is determined by the average value of each RGB signal at the time of uniform light irradiation, and each average value is subtracted from the RGB signal at the time of uniform light irradiation. The FPN component of the pixel may be extracted. The R signal, G signal, and B signal from which the FPN component has been removed in this way are supplied to the AE / WB evaluation value detection circuit 7 and the WB control circuit 8.

  The AE / WB evaluation value detection circuit 7 calculates the average value distribution range of the luminance representing the luminance range of the subject by confirming the luminance value of the image signal composed of the given R signal, G signal, and B signal, It is sent to the overall control unit 13 as an AE evaluation value for setting the exposure amount. Based on this AE evaluation value, the overall control unit 13 controls the aperture of the diaphragm 1a, whereby the exposure amount is controlled. The AE / WB evaluation value detection circuit 7 confirms the respective luminance ratios and luminance differences from the given R, G, and B signals, and calculates a WB evaluation value that is a reference value for white balance. And sent to the overall control unit 13. The WB control circuit 8 performs white balance processing based on the WB evaluation value and the dynamic range control signal given from the overall control unit 13 so that the R signal, the G signal, and the B signal have the same photoelectric conversion characteristics. Applied.

  The R, G, and B signals that have been subjected to white balance processing by the WB control circuit 8 are subjected to color interpolation processing by the color interpolation circuit 9. When a color filter having an RGB Bayer array as shown in FIG. 2 is provided in the solid-state imaging device 2, the color signal output from each pixel is only the color signal from the color filter provided in the pixel. Therefore, the color interpolation circuit 9 performs color interpolation processing by generating other color signals from the color signals of adjacent pixels.

  When the RGB color filters are arranged in the pixels G11 to G44 as shown in FIG. 2, the R signals r11, r31, r13, and r33 are converted from the pixels G11, G31, G13, and G33 to the pixels G21, G41, and G33, respectively. G12, G32, G23, G43, G14, G34 to G signals g21, g41, g12, g32, g23, g43, g14, g34, and pixels G22, G42, G24, G44 to B signals b22, b42, b24, b44 Is output. At this time, RGB signals of the pixels G22, G23, G32, and G33 are expressed by the following equations.

R signal r22, G signal g22, B signal b22 of pixel G22
r22 = (r11 + r31 + r13 + r33) / 4
g22 = (g21 + g12 + g32 + g23) / 4
b22 = b22
R signal r32, G signal g32, B signal b32 of the pixel G32
r32 = (r31 + r33) / 2
g32 = g32
b32 = (b22 + b42) / 2
R signal r23, G signal g23, B signal b23 of pixel G23
r23 = (r13 + r33) / 2
g23 = g23
b23 = (b22 + b24) / 2
R signal r33, G signal g33, B signal b33 of pixel G33
r33 = r33
g33 = (g32 + g23 + g43 + g34) / 4
b33 = (b22 + b42 + b24 + b44) / 4.

When the RGB signal is obtained for each pixel by performing the color interpolation processing in this way, the RGB signal of each pixel is given to the color correction circuit 10 and the color correction processing for emphasizing the hue of each pixel is performed. Is done. At this time, each of the RGB signals is subjected to color correction by the values of other color signals. That is, the RGB signals rxkl, gxkl, bxkl of the pixel Gkl subjected to the hue correction are generated by substituting the RGB signals rkl, gkl, bkl of the pixel Gkl into the following expression. At this time, the color correction coefficients based on a1 to a3, b1 to b3, and c1 to c3 are switched based on the photoelectric conversion characteristics of the input RGB signal, and the color tone of the RGB signal of each pixel is emphasized. Details of the switching operation of the color correction coefficients a1 to a3, b1 to b3, and c1 to c3 in the color correction circuit 10 will be described later.
rxkl = a1 * rkl + a2 * gkl + a3 * bkl
gxkl = b1 * rkl + b2 * gkl + b3 * bkl
bxkl = c1 * rkl + c2 * gkl + c3 * bkl.

When the color correction process is not performed in the color correction circuit 10, the following color correction coefficients a1 to a3, b1 to b3, and c1 to c3 are given from the overall control unit 13.
(A1, a2, a3) = (1.0, 0.0, 0.0)
(B1, b2, b3) = (0.0, 1.0, 0.0)
(C1, c2, c3) = (0.0, 0.0, 1.0).

  The RGB signal subjected to the color correction by the color correction circuit 10 is given to the gradation conversion circuit 11 so that the dynamic range control signal and the AE evaluation value are input from the overall control unit 13 so as to obtain an appropriate output level. Based on the above, the gradation characteristics are changed by a change based on the γ curve or a change in digital gain. Then, in the coring circuit 12 having the level conversion characteristics of the relationship shown in FIG. 3 with respect to the edge component, the noise component superimposed on each of the RGB signals is removed, and the edge component is extracted and subjected to edge enhancement processing. Is done. The operations of the gradation conversion circuit 11 and the coring circuit 12 will be described later.

<Configuration example of solid-state imaging device>
The configuration of the solid-state imaging device 2 in the imaging apparatus configured as shown in FIG. 1 will be described with reference to the drawings. FIG. 4 is a block diagram schematically showing a part of the configuration of the solid-state imaging device of this example, and FIG. 5 is a circuit diagram showing the configuration of each pixel.

  As shown in FIG. 4, in the solid-state imaging device 2, G11 to Gmn represent pixels arranged in a matrix (matrix arrangement). Reference numeral 21 denotes a vertical scanning circuit, which sequentially scans rows (lines) 23-1, 23-2,..., 23-n that apply a signal φV to each pixel and lines 24-1 and 24-2. ,..., 24-n, and a signal φVD is applied to each pixel via lines 25-1, 25-2,. A horizontal scanning circuit 22 sequentially reads out photoelectric conversion signals derived from the pixels to the output signal lines 26-1, 26-2, ..., 26-m in the horizontal direction for each pixel. Reference numeral 20 denotes a power supply line. For each pixel, not only the lines 23-1 to 23-n, 24-1 to 24-n, 25-1 to 25-n, the output signal lines 26-1 to 26-m, and the power supply line 20, but also These lines (for example, a clock line and a bias supply line) are also connected, but these are omitted in FIG.

  Also, constant current sources 27-1 to 27-m are connected to the output signal lines 26-1 to 26-m, and pixels are provided via the signal lines 26-1 to 26-m, respectively. Selection circuits 28-1 to 28-m for sample-holding image signals and noise signals supplied from G11 to Gmn are provided. Then, when the image signal and the noise signal are sequentially transmitted from the selection circuits 28-1 to 28-m to the correction circuit 29, the correction circuit 29 performs correction processing and outputs the image signal from which noise has been removed to the outside. Is done. The DC voltage VPS is applied to one end of the constant current sources 27-1 to 27-m.

  In such a solid-state imaging device, an image signal and a noise signal output from the pixel Gab (a natural number of a: 1 ≦ a ≦ m, b: 1 ≦ b ≦ n) are respectively output signal lines 26- The signal is output via a and amplified by a constant current source 27-a connected to the output signal line 26-a. The image signal and noise signal output from the pixel Gab are sequentially sent to the selection circuit 28-a, and the sent image signal and noise signal are sampled and held in the selection circuit 28-a. Thereafter, after the sampled and held image signal is sent from the selection circuit 28-a to the correction circuit 29, the similarly sampled and held noise signal is sent to the correction circuit 29. The correction circuit 29 corrects the image signal given from the selection circuit 28-a based on the noise signal similarly given from the selection circuit 28-a, and outputs the noise-removed image signal to the amplifier 3.

  In the solid-state imaging device 2 having such a configuration, as shown in FIG. 5, the pixels G11 to Gmn have the drain of the MOS transistor T1 connected to the anode of the photodiode PD to which the DC voltage VPD is applied to the cathode. The gate and drain of the MOS transistor T2 and the gate of the MOS transistor T3 are connected to the source of T1. The gate of the MOS transistor T4 and the drain of the MOS transistor T5 are connected to the source of the MOS transistor T3, and the drain of the MOS transistor T6 is connected to the source of the MOS transistor T4. The drain of the MOS transistor T6 is connected to the output signal line 26 (corresponding to the output signal lines 26-1 to 26-m in FIG. 4). The MOS transistors T1 to T6 are P-channel MOS transistors.

  The signal φVPS is input to the source of the MOS transistor T2 via the line 25 (corresponding to the lines 25-1 to 25-n in FIG. 4), and the DC voltage VPD is applied to the drains of the MOS transistors T3 and T4. Further, the other end of the capacitor C to which the signal φVD is applied via the line 24 (corresponding to the lines 24-1 to 24-n in FIG. 4) is connected to the source of the MOS transistor T3. The DC voltage VRG is input to the source of the MOS transistor T5, and the signal φRS is input to the gate thereof. Further, signals φS and φV are input to the gates of the MOS transistors T1 and T6, respectively.

  The signal φVPS is a binary voltage signal. When the amount of incident light exceeds a predetermined value, the voltage for operating the MOS transistor T2 in the subthreshold region is VL, and the voltage higher than this voltage is used for the MOS transistor T2. The voltage for making the conductive state is VH. The signal φVD is a ternary voltage signal, the voltage value when integrating the capacitor C is set to the highest Vh, the voltage value when reading the image signal is set to Vm lower than Vh, and the noise signal is read The voltage value is set to Vl lower than Vm.

  The operation of the pixels G11 to Gmn in the solid-state imaging device 2 configured as described above will be described with reference to the time chart of FIG. First, when a pulse signal φVD having a voltage value Vm and a pulse signal φV are supplied and an image signal is output, the signal φVD is set to Vh, then the signal φS is set high to turn off the MOS transistor T1, and the reset operation is performed. Begins. Next, the signal φVPS applied to the source of the MOS transistor T2 is set to VH, and the source voltage of the MOS transistor T2 is increased, so that the negative charge accumulated in the gate and drain of the MOS transistor T2 and the gate of the MOS transistor T3. Are quickly recombined. At this time, the signal φRS is set to low, the MOS transistor T5 is turned on, and the voltage at the connection node between the capacitor C and the gate of the MOS transistor T4 is initialized.

  Then, the signal φVPS applied to the source of the MOS transistor T2 is set to VL to return the potential state of the MOS transistor T2 to the original state, and then the signal φRS is set to high to turn off the MOS transistor T5. Thereafter, the capacitor C performs an integration operation, and the voltage at the connection node between the capacitor C and the gate of the MOS transistor T4 becomes in accordance with the reset gate voltage of the MOS transistor T2. Then, the pulse signal φV is applied to the gate of the MOS transistor T6 to turn on the MOS transistor T6 and set the voltage value of the signal φVD to Vl. At this time, since the MOS transistor T4 operates as a source follower type MOS transistor, a noise signal appears on the output signal line 26 as a voltage signal. Thereafter, the pulse signal φRS is again applied to the MOS transistor T5 to reset the voltage at the connection node between the capacitor C and the gate of the MOS transistor T4, and then the signal φS is set low to turn on the MOS transistor T1 to perform the imaging operation. Make it ready.

  After the noise signal is output in this way, the imaging operation is started when the MOS transistor T1 is turned on. At this time, the signal φRS is set high and the MOS transistor T5 is turned off. Further, the signal φVPS applied to the source of the MOS transistor T2 is set to VL, and the voltage value of the signal φVD applied to the capacitor C is set to Vh to perform the integration operation. When photocharge corresponding to the amount of incident light flows from the photodiode PD into the MOS transistor T2, the MOS transistor T2 is in a cut-off state, so that the photocharge is accumulated at the gate of the MOS transistor T2.

  Therefore, when the brightness of the subject to be imaged is low and the amount of incident light entering the photodiode PD is small, a voltage corresponding to the amount of photocharge accumulated at the gate of the MOS transistor T2 appears at the gate of the MOS transistor T2, A voltage linearly proportional to the integrated value of the light quantity appears at the gate of the MOS transistor T3. Also, when the luminance of the subject to be imaged is high and the amount of incident light entering the photodiode PD is large, and the voltage according to the amount of photocharge accumulated at the gate of the MOS transistor T2 increases, the MOS transistor T2 operates in the subthreshold region. Therefore, a voltage that changes logarithmically with respect to the amount of incident light appears at the gate of the MOS transistor T3.

  Since a drain current that is linearly or naturally logarithmically changed with respect to the incident light amount flows from the capacitor C, the gate voltage of the MOS transistor T4 is set to the integrated value of the incident light amount. Thus, the voltage changes linearly or in a natural logarithm. Then, by setting the voltage value of the signal φVD to Vm and applying the pulse signal φV to the MOS transistor T6, a source current corresponding to the gate voltage of the MOS transistor T4 flows to the output signal line 6 via the MOS transistor T6. . At this time, since the MOS transistor T4 operates as a source follower type MOS transistor, an image signal appears as a voltage signal on the output signal line 6. Thereafter, the signal φV is set high to turn off the MOS transistor T6, and the voltage value of the signal φVD is set to Vh.

  When operating in this way, the voltage value VL of the signal φVPS at the time of imaging decreases, and the potential difference between the gate and the source of the MOS transistor T2 increases as the difference from the voltage value VH of the signal φVPS at the time of reset increases. The ratio of subject luminance at which the MOS transistor T2 operates in the cutoff state increases. Therefore, as shown in FIG. 7, the lower the voltage value VL, the larger the ratio of subject luminance to be linearly converted. Therefore, for example, the luminance range of the subject is detected, and if the luminance range of the subject is narrow, the voltage value VL is lowered to widen the luminance range for linear conversion, and if the luminance range of the subject is wide, the voltage value VL is increased. By widening the luminance range for logarithmic conversion, photoelectric conversion characteristics that match the characteristics of the subject can be obtained. It should be noted that when the voltage value VL is minimized, it is possible to always perform linear conversion. When the voltage value VL is maximized, it is possible to always perform logarithmic conversion.

  The solid-state image sensor that can switch the dynamic range according to the luminance range of the subject, etc., by the overall controller 13 switching the voltage value VL of the signal φVPS given to the pixels G11 to Gmn of the solid-state image sensor 2 operating in this way. 2 can be used. That is, the inflection point (luminance value) for switching from the linear conversion operation to the logarithmic conversion operation in the pixels G11 to Gmn of the solid-state imaging device 2 can be set by switching the voltage value VL of the signal φVPS. it can. Note that the amount of photoelectric charge flowing into the MOS transistor T2 until reaching the gate voltage of the MOS transistor T2 when changing to the logarithmic conversion operation at the time of imaging is the same in all the pixels.

  In this configuration example, a solid-state imaging device including pixels configured as shown in FIG. 5 is used. However, the present invention is not limited to this configuration, and linear conversion operation and logarithmic conversion operation are automatically performed in each pixel. As long as it can be switched to the above, it may be composed of a pixel having another configuration such as a pixel having a configuration as shown in Patent Document 2. In addition, the inflection point between the linear conversion operation and the logarithmic conversion operation is changed by changing the voltage value VL of the signal φVPS at the time of imaging. However, the linear value can be changed by changing the voltage value VH of the signal φVPS at the time of resetting. The inflection point between the conversion operation and the logarithmic conversion operation may be changed. The inflection point may be changed by changing the reset time. The inflection point may be changed by changing the reset time. Further, although each pixel is provided with an RGB filter, other color filters such as cyan, magenta, and yellow may be provided.

<Color correction circuit>
The operation of the color correction circuit in the imaging apparatus configured as shown in FIG. 1 will be described in detail below with reference to the drawings. FIG. 8 is a block diagram showing the internal configuration of the color correction circuit.

  As shown in FIG. 8, the color correction circuit 10 includes comparators 101r, 101g, and 101b that check the photoelectric conversion characteristics of the RGB signals of the pixels input from the color interpolation circuit 9, and comparators 101r, 101g, and 101b. Coefficient setting units 102r, 102g, and 102b for setting color correction coefficients for RGB signals by output; multipliers 103rr, 103rg, and 103rb to which color correction coefficients a1, a2, and a3 set by the coefficient setting unit 102r are respectively provided; Multipliers 103gr, 103gg, 103gb to which the color correction coefficients b1, b2, b3 set by the coefficient setting unit 102g are respectively provided, and color correction coefficients c1, c2, c3 set by the coefficient setting unit 102b are respectively provided. Multipliers 103br, 103bg, 103bb, An adder 104r for adding outputs from the calculators 103rr and 103rg; an adder 105r for adding outputs from the multiplier 103rb and the adder 104r; an adder 104g for adding outputs from the multipliers 103gr and 103gg; An adder 105g for adding the outputs from the multiplier 103gb and the adder 104g, an adder 104b for adding the outputs from the multipliers 103br and 103bg, and an adder 105b for adding the outputs from the multiplier 103bb and the adder 104b And comprising.

  In the color correction circuit 10 having such a configuration, the comparators 101r, 101g, and 101b compare the signal level of the RGB signal input from the color interpolation circuit 9 with the signal level serving as a threshold value, thereby comparing the photoelectric conversion characteristics of each signal. Confirm. The operation of the comparator 101r will be described below as a representative. When photoelectric conversion characteristics representing the relationship between the signal level of the R signal output from the color interpolation circuit 9 and the luminance value of the subject are expressed as shown in FIG. 9, threshold levels V1 and V2 are set as threshold signal levels. Is done. When the luminance values corresponding to the threshold levels V1 and V2 are L1 and L2 as shown in FIG. 9, in the luminance range lower than the luminance value L1, the photoelectric conversion characteristics are linear conversion characteristics and higher than the luminance value L2. In the luminance range, the photoelectric conversion characteristic is a logarithmic conversion characteristic. Further, in the luminance range between the luminance values L1 and L2, both characteristics in which the photoelectric conversion characteristic is switched from the linear conversion characteristic to the logarithmic conversion characteristic are mixed. Hereinafter, a luminance range lower than the luminance value L1 is a linear region, a luminance range between the luminance values L1 and L2 is a linear / logarithmic region, and a luminance range higher than the luminance value L2 is a logarithmic region.

  Based on the threshold levels V1 and V2, the comparator 101r checks for each pixel whether the photoelectric conversion characteristic of the R signal is in a linear region, a linear / logarithmic region, or a logarithmic region. That is, when an R signal whose signal level is lower than the threshold level V1 is input, it is confirmed that the signal is in the linear region, and an R signal whose signal level is equal to or higher than the threshold level V1 and lower than the threshold level V2 is input. When it is determined that the signal is in the linear / logarithmic region, it is confirmed that the signal is in the logarithmic region when an R signal having a signal level equal to or higher than the threshold level V2 is input. Note that since the white balance processing is performed in the WB control circuit 8 so that the photoelectric conversion characteristics of the RGB signals match, the photoelectric conversion characteristics of the GB signals are also expressed as shown in FIG. Therefore, similarly to the comparator 101r, the comparator 101g has a G signal photoelectric conversion characteristic in any of the linear region, linear / logarithmic region, and logarithmic region for each pixel based on the threshold levels V1 and V2. In the comparator 101b, on the basis of the threshold levels V1 and V2, the photoelectric conversion characteristic of the B signal is in each of the linear region, linear / logarithmic region, and logarithmic region for each pixel. Check if it is a thing.

  The results confirmed by the comparators 101r, 101g, and 101b are respectively given to the coefficient setting units 102r, 102g, and 102b, and based on the results, the color correction coefficients a1 to a3, b1 to b3, and c1 to c3 are obtained. Is set. As the color correction coefficients a1 to a3, b1 to b3, and c1 to c3, three types of values corresponding to the three regions of the linear region, the linear / logarithmic region, and the logarithmic region are obtained from the overall control unit 13 as the coefficient setting unit 102r, 102g and 102b are given and stored. Further, the values of the color correction coefficients a1 to a3, b1 to b3, and c1 to c3 are values satisfying a1 + a2 + a3 = 1, b1 + b2 + b3 = 1, and c1 + c2 + c3 = 1, respectively. For example, the values of the color correction coefficients a1 to a3, b1 to b3, and c1 to c3 are set as follows for the three regions of the linear region, the linear / logarithmic region, and the logarithmic region.

Linear region (a1, a2, a3) = (1.2, -0.1, -0.1)
(B1, b2, b3) = (− 0.1, 1.2, −0.1)
(C1, c2, c3) = (− 0.1, −0.1, 1.2)

Linear / logarithmic region (a1, a2, a3) = (1.35, -0.175, -0.175)
(B1, b2, b3) = (-0.175, 1.35, -0.175)
(C1, c2, c3) = (-0.175, -0.175, 1.35)

Logarithmic region (a1, a2, a3) = (1.5, -0.25, -0.25)
(B1, b2, b3) = (− 0.25, 1.5, −0.25)
(C1, c2, c3) = (− 0.25, −0.25, 1.5).

  Therefore, when the comparator 101r confirms that the R signal is a signal in the linear region, the coefficient setting unit 102r sets the color correction coefficients (a1, a2, a3) as (1.2, −0.1, −0.1) and a multiplier. 103rr, 103rg, 103rb, and when the comparator 101r confirms that the R signal is a linear / logarithmic domain signal, the color correction coefficients (a1, a2, a3) are (1.35, -0.175, -0.175). ) To the multipliers 103rr, 103rg, 103rb, and if the comparator 101r confirms that the R signal is a logarithmic domain signal, the color correction coefficients (a1, a2, a3) are set to (1.5, -0.25, -0.25) to the multipliers 103rr, 103rg, and 103rb.

  Similarly, when the comparator 101g confirms that the G signal is a signal in the linear region, the coefficient setting unit 102g multiplies the color correction coefficients (b1, b2, b3) as (−0.1, 1.2, −0.1). When the comparator 101g confirms that the G signal is a linear / logarithmic domain signal, the color correction coefficients (b1, b2, b3) are set to (−0.175, 1.35, −3). 0.175) to the multipliers 103gr, 103gg, 103gb, and when the comparator 101g confirms that the G signal is a logarithmic domain signal, the color correction coefficients (b1, b2, b3) are set to (-0.25, 1.5). , −0.25) to the multipliers 103gr, 103gg, and 103gb.

  Similarly, when the comparator 101b confirms that the B signal is a linear region signal, the coefficient setting unit 102b multiplies the color correction coefficients (c1, c2, c3) as (−0.1, −0.1, 1.2). When the comparator 103b confirms that the B signal is a linear / logarithmic domain signal, the color correction coefficients (c1, c2, c3) are set to (−0.175, −0.175, 1.35) is supplied to the multipliers 103br, 103bg, 103bb, and when the comparator 101b confirms that the B signal is a logarithmic domain signal, the color correction coefficients (c1, c2, c3) are set to (−0.25, − 0.25, 1.5) to the multipliers 103br, 103bg, 103bb.

  In this way, the RGB signals in the same pixel are confirmed for their respective photoelectric conversion characteristics by the comparators 101r, 101g, and 101b, and the color correction coefficients for the respective signals are set to the coefficient setting units 102r, 102g, and 102b. Set by. Accordingly, when the photoelectric conversion characteristics of the RGB signals rkl, gkl, bkl in the same pixel Gkl are confirmed by the comparators 101r, 101g, 101b, the photoelectric conversion characteristics of the different regions are provided, and the coefficient setting units 102r, 102g are provided. , 102b may set color correction coefficients for different regions identified for each of the RGB signals.

  As described above, when the color correction coefficients a1, a2, and a3 set by the coefficient setting unit 102r are respectively supplied to the multipliers 103rr, 103rg, and 103rb, the multiplier 103rr causes the color correction coefficient a1 to be applied to the R signal and the multiplier 103rg. The G signal is multiplied by the color correction coefficient a2, and the multiplier 103rb is multiplied by the B signal by the color correction coefficient a3. Similarly, when the color correction coefficients b1, b2, and b3 set by the coefficient setting unit 102g are respectively supplied to the multipliers 103gr, 103gg, and 103gb, the multiplier 103gr supplies the color correction coefficient b1 to the R signal and the multiplier 103gg. The G signal is multiplied by the color correction coefficient b2, and the multiplier 103gb multiplies the B signal by the color correction coefficient b3. Similarly, when the color correction coefficients c1, c2, and c3 set by the coefficient setting unit 102b are respectively supplied to the multipliers 103br, 103bg, and 103bb, the multiplier 103br supplies the color correction coefficient c1 to the R signal and the multiplier 103bg. The G signal is multiplied by the color correction coefficient c2, and the multiplier 103bb multiplies the B signal by the color correction coefficient c3.

  Then, the R signal multiplied by the color correction coefficient a1 by the multiplier 103rr is added by the adder 104r to the G signal obtained by multiplying the color correction coefficient a2 by the multiplier 103rg, and then the color correction coefficient by the multiplier 103rb. The B signal multiplied by a3 is added by the adder 105r, and the color-corrected R signal is output. Similarly, the R signal multiplied by the color correction coefficient b1 by the multiplier 103gr and the G signal multiplied by the color correction coefficient b2 by the multiplier 103gg are added by the adder 104g, and then the color correction is performed by the multiplier 103gg. The B signal multiplied by the coefficient b3 is added by the adder 105g, and the color-corrected G signal is output. Similarly, the R signal multiplied by the color correction coefficient c1 by the multiplier 103br and the G signal multiplied by the color correction coefficient c2 by the multiplier 103bg are added by the adder 104b, and then the color correction is performed by the multiplier 103bb. The B signal multiplied by the coefficient c3 is added by the adder 105b, and the color-corrected B signal is output.

  Therefore, for example, in the comparator 101r, the signal level of the R signal rkl of the pixel Gkl becomes a value between the threshold levels V1 and V2, and it is confirmed that the photoelectric conversion characteristic is in the linear / logarithmic region. At 101g, the signal level of the G signal gkl of the pixel Gkl becomes lower than the threshold level V1, and it is confirmed that the photoelectric conversion characteristic is in the linear region. In the comparator 101b, the B signal bkl of the pixel Gkl It is assumed that the signal level is higher than the threshold level V2, and that the photoelectric conversion characteristic is in the logarithmic region. At this time, the coefficient setting unit 102r gives the color correction coefficients (a1, a2, a3) as (1.35, -0.175, -0.175) to the multipliers 103rr, 103rg, 103rb, and the coefficient setting unit 102g performs color correction. The coefficients (b1, b2, b3) are given as (-0.1, 1.2, -0.1) to the multipliers 103gr, 103gg, 103gg, and the coefficient setting unit 102b gives the color correction coefficients (c1, c2, c3) to (- 0.25, -0.25, 1.5) to the multipliers 103br, 103bg, 103bb.

  The multiplier 103rr multiplies the R signal rkl by the color correction coefficient a1 of 1.35, the multiplier 103rg multiplies the G signal gkl by the color correction coefficient a2 of -0.175, and the multiplier 103rb multiplies the B signal bkl by-. The color correction coefficient a3 which is 0.175 is multiplied. The multiplier 103gr multiplies the R signal rkl by a color correction coefficient b1 of −0.1, the multiplier 103gg multiplies the G signal gkl by the color correction coefficient b2 of 1.2, and the multiplier 103gg multiplies the B signal bkl by −. The color correction coefficient b3 which becomes 0.1 is multiplied. Further, the multiplier 103br multiplies the R signal rkl by a color correction coefficient c1 of −0.25, the multiplier 103bg multiplies the G signal gkl by a color correction coefficient c2 of −0.25, and the multiplier 103bb multiplies the B signal bkl by the color correction coefficient c2. A color correction coefficient c3 of 1.5 is multiplied.

  Thereafter, the adder 104r adds the G signal of −0.175 × gkl to the R signal of 1.35 × rkl, and then the B signal of −0.175 × bkl is added by the adder 105r to obtain 1.35 × rkl−0.175. A color-corrected R signal rxkl of xgkl-0.175xbkl is output. The adder 104g adds the G signal of 1.2 × gkl to the R signal of −0.1 × rkl, and then the B signal of −0.1 × bkl is added by the adder 105g to obtain −0.1 × rkl +. A color-corrected G signal gxkl of 1.2 × gkl−0.1 × bkl is output. The adder 104b adds the G signal of −0.25 × gkl to the R signal of −0.25 × rkl, and then the B signal of 1.5 × bkl is added by the adder 105b to obtain −0.25 × rkl−. A color-corrected B signal bxkl of 0.25 × gkl + 1.5 × bkl is output.

<Tone conversion circuit>
The operation of the gradation conversion circuit in the imaging apparatus configured as shown in FIG. 1 will be described in detail below with reference to the drawings. FIG. 10 is a block diagram showing the internal configuration of the gradation conversion circuit. FIG. 12 is a graph showing the relationship between input / output and photoelectric conversion characteristics in the gradation conversion circuit. In the following description, in order to simplify the description, a case will be described in which the photoelectric conversion characteristic has two conversion characteristics of a linear conversion characteristic and a logarithmic conversion characteristic.

  As shown in FIG. 10, the gradation conversion circuit 11 includes comparators 111r, 111g, and 111b that check the photoelectric conversion characteristics of the RGB signals of the pixels that are input from the color correction circuit 10, and comparators 111r, 111g, and 111b. From the color correction circuit 10 according to the arithmetic expressions given from the arithmetic expression setting sections 112r, 112g, 112b and the arithmetic expression setting sections 112r, 112g, 112b, respectively. Gradation conversion units 113r, 113g, and 113b that perform gradation conversion of the RGB signals of the input pixels.

  In the gradation conversion circuit 11 having such a configuration, the comparators 111r, 111g, and 111b are similar to the comparators 101r, 101g, and 101b of the color correction circuit 10 and the signal levels and threshold values of the RGB signals input from the color correction circuit 10 are as follows. The photoelectric conversion characteristics of each signal are confirmed by comparing the signal levels. Hereinafter, the operation of the comparator 111r will be described as a representative. When the photoelectric conversion characteristic representing the relationship between the signal level of the R signal output from the color correction circuit 10 and the luminance value of the subject is expressed as shown in FIG. 11, the threshold level V3 is set as the signal level serving as the threshold. . When the luminance value corresponding to the threshold level V3 is L3 as shown in FIG. 11, the photoelectric conversion characteristic is a linear conversion characteristic in the luminance range lower than the luminance value L3, and the luminance is higher than the luminance value L3. In the range, the photoelectric conversion characteristic is a logarithmic conversion characteristic.

  Hereinafter, a luminance range lower than the luminance value L3 is defined as a linear region, and a luminance range higher than the luminance value L3 is defined as a logarithmic region. Further, each of the GB signals output from the color correction circuit 10 has already been subjected to the white balance processing by the WB control circuit 8, so that the photoelectric conversion characteristics as shown in FIG. The threshold level for identifying the logarithmic region is a signal level V3 corresponding to the luminance value L3.

  Therefore, the comparator 111r confirms, for each pixel, whether the photoelectric conversion characteristic of the R signal is in a linear region or a logarithmic region based on this threshold level V3. That is, when an R signal whose signal level is lower than the threshold level V3 is input, it is confirmed that the signal is in the linear region, and when an R signal whose signal level is equal to or higher than the threshold level V3 is input, logarithm Confirmed to be in the area. Similarly, in the comparator 111g, based on the threshold level V3, for each pixel, whether the photoelectric conversion characteristic of the G signal is in a linear region or a logarithmic region is determined in the comparator 111b to the threshold level V3. Based on this, it is confirmed for each pixel whether the photoelectric conversion characteristic of the B signal is in a linear region or a logarithmic region.

  The results confirmed by the comparators 111r, 111g, and 111b are respectively given to the arithmetic expression setting units 112r, 112g, and 112b. Based on the results, the arithmetic expressions fr for performing gradation conversion of the respective RGB signals. , Fg, fb are set. As the arithmetic expressions fr, fg, and fb, two types of arithmetic expressions corresponding to the two areas of the linear area and the logarithmic area are given to the arithmetic expression setting sections 112r, 112g, and 112b from the overall control section 13 and stored therein. At this time, the arithmetic expressions fr, fg, and fb are calculated such that the signal level of 0 to V3 is converted to 0 to Vx3 (Vx3 = V3 / A, A> 1) in the linear region as shown in FIG. Further, in the logarithmic domain, the arithmetic expression is such that the signal level of V3 to Vmax is converted to Vx3 to Vmax. That is, as the arithmetic expressions fr, fg, and fb, the arithmetic expressions that are multiplied by a coefficient smaller than 1 in the linear domain and converted to signals having a signal level in the range of 0 to Vx3 are partially inverse functions in the logarithmic domain. Arithmetic expressions for conversion to signals having signal levels in the range of Vx3 to Vmax are stored.

  As described above, when the arithmetic expressions fr, fg, and fb set by the arithmetic expression setting units 112r, 112g, and 112b are respectively provided to the gradation converting units 113r, 113g, and 113b, the gradation converting unit 113r converts the arithmetic expressions fr into the arithmetic expression fr. The gradation conversion of the R signal corresponding to the gradation conversion of the G signal according to the arithmetic expression fg in the gradation conversion section 113g, and the gradation conversion of the B signal according to the arithmetic expression fb in the gradation conversion section 113b, respectively. Done. Therefore, the gradation conversion units 113r, 113g, and 113b perform gradation conversion as shown in FIG. 12, thereby lowering the gradation of the linear region and increasing the gradation of the logarithmic region in each of the RGB signals.

  Therefore, for example, in each of the comparators 111r and 111g, it is confirmed that the signal levels of the RG signals rxkl and gxkl of the pixel Gkl are lower than the threshold level V3, and the photoelectric conversion characteristics thereof are in the linear region. In 111b, it is assumed that the signal level of the B signal bkl of the pixel Gkl is higher than the threshold level V3, and that the photoelectric conversion characteristics are in the logarithmic region. At this time, the arithmetic expression setting units 112r and 112g respectively give the arithmetic expressions fr and fg based on the linear region to the gradation conversion parts 113r and 113g, respectively, and the arithmetic expression setting unit 112b determines the arithmetic expression fb based on the logarithmic region and the gradation This is given to the conversion unit 113b. In the gradation converter 113r, the R signal rxkl having a signal level in the range of 0 to V3 is changed to the R signal rykl in the range of 0 to Vx3, and in the gradation converter 113g, the signal level is set to 0 to 0. The G signal gxkl in the range of V3 is a G signal gykl with a signal level in the range of 0 to Vx3, and the B signal bxkl in the range of V3 to Vmax is a B signal bykl in the range of Vx3 to Vmax, respectively. The gradation is converted by the conversion.

  As another embodiment of the gradation changing circuit 11, instead of providing the comparators 111r, 111g, 111b and the arithmetic expression setting units 112r, 112g, 112b, an LUT (lookup table) in which the input / output characteristics of FIG. Thus, gradation conversion may be performed. By using this LUT, it is not necessary to provide the comparators 111r, 111g, 111b and the arithmetic expression setting units 112r, 112g, 112b, and the configuration of the gradation conversion circuit 11 can be further simplified. When such an LUT is used, the input (vertical axis) in FIG. 12 corresponds to the input of the LUT, and the output (horizontal axis) in FIG. 12 corresponds to an output signal subjected to gradation conversion by the LUT. Therefore, even with the LUT in which the input / output characteristics of FIG. 12 are stored, the gradation in the linear region can be lowered and the gradation in the logarithmic region can be raised.

<Coring circuit>
The operation of the coring circuit in the imaging apparatus configured as shown in FIG. 1 will be described in detail below with reference to the drawings. FIG. 13 is a block diagram showing an internal configuration of the coring circuit.

  As shown in FIG. 13, the coring circuit 12 includes low-pass filters (LPF) 121 r, 121 g, and 121 b that remove noise components of the RGB signals subjected to gradation conversion by the gradation conversion circuit 11, and the gradation conversion circuit 11. Output from the subtracters 122r, 122g, 122b and the subtractors 122r, 122g, 122b that extract the noise components by subtracting the RGB signals from which the noise components have been removed by the LPFs 121r, 121g, 121b from the RGB signals from The coring units 123r, 123g, and 123b that perform coring calculation processing on noise components of RGB signals to be processed, and the edge components of the RGB signals obtained by performing the coring calculation processing in the coring units 123r, 123g, and 123b are edge-enhanced Multipliers 124r, 124g to which the quantity β is multiplied; Comprising a 24b, a multiplier 124r, 124g, 124b LPF121r the edge component of the RGB signals from each, 121g, adders 125r to be added to each RGB signal from which the noise components have been removed in 121b, 125 g, and 125b, the.

  In the coring circuit 12 having such a configuration, the low-pass filters 121r, 121g, and 121b perform a matrix operation of n pixels × n pixels, and the n × n matrix coefficient (LPF coefficient) is obtained from the overall control unit 13. Given. Further, in the coring portions 123r, 123g, and 123b, the input and output have a relationship as shown in FIG. That is, when the input is greater than -α and smaller than α, the output is 0. When the input is less than -α, the output is a value obtained by adding α to the input, or when the input is greater than or equal to α, the output is input. Is a value obtained by subtracting α. In the following, this value α is used as a coring coefficient, and this coring coefficient α is given from the overall control unit 13. An edge enhancement amount β multiplied by the multipliers 124r, 124g, and 124b is also given from the overall control unit 13.

  The LPF coefficient, the coring coefficient α, and the edge enhancement amount β are set by the photoelectric conversion characteristics in the pixel corresponding to the RGB signal input from the gradation conversion circuit 11. At this time, the overall control unit 13 confirms the photoelectric conversion characteristics of each pixel based on the image signal after the color interpolation process is performed in the color interpolation circuit 9. The LPF coefficient, coring coefficient α, and edge enhancement amount β are set for each pixel, and the same LPF coefficient, coring coefficient α, and edge enhancement amount β are set for the RGB signal of the same pixel. It doesn't matter. Thus, when the same LPF coefficient, coring coefficient α, and edge enhancement amount β are set for the RGB signal of the same pixel, the photoelectric conversion characteristics of each pixel are further confirmed from the RGB signal before color interpolation processing. Alternatively, the photoelectric conversion characteristics of each pixel may be confirmed from the G signal after the color interpolation process. In any case, when the RGB signal has a photoelectric conversion characteristic as shown in FIG. 11, the linear region and the threshold level V3 are compared by comparing the signal level of the RGB signal with the threshold level V3 as in the above-described gradation conversion circuit 11. It is confirmed which region of the logarithmic region has the photoelectric conversion characteristic.

  At this time, for example, the LPF coefficient is a 5 × 5 matrix coefficient, and the LPF coefficient, the coring coefficient α, and the edge enhancement amount β are set as follows for the linear region and the logarithmic region.

Linear region LPF coefficient: 4 0 9 0 4
0 9 20 9 0
9 20 88 20 9
0 9 20 9 0
4 0 9 0 4
Coring coefficient α: 2
Edge enhancement amount β: 2

Logarithmic domain LPF coefficient: 1 2 4 2 1
2 6 11 6 2
4 11 24 11 4
2 6 11 6 2
1 2 4 2 1
Coring coefficient α: 4
Edge enhancement amount β: 3.

  When the R signal rykl, the G signal gykl, and the B signal bykl of the pixel Gkl are input from the gradation conversion circuit 11 to the coring circuit 12 configured in this way, the RGB in the pixel Gkl confirmed by the overall control unit 13 is detected. LPF coefficients and coring coefficients α and edge enhancement amounts β based on photoelectric conversion characteristics of the signals are set in the LPFs 121r, 121g, and 121b, the coring units 123r, 123g, and 123b, and the multipliers 124r, 124g, and 124b, respectively. . Then, the LPFs 121r, 121g, and 121b operate according to the set LPF coefficients, so that the R signal rzkl from which the noise component is removed from the R signal rykl from the LPF 121r, and the G signal from which the noise component is removed from the G signal gykl from the LPF 121g. The signal gzkl is output from the LPF 121b as a B signal bzkl obtained by removing noise components from the B signal bykl.

  Thereafter, the R signal rzkl from the LPF 121r is subtracted from the R signal rykl from the gradation conversion circuit 11 in the subtractor 122r, and the G signal gzkl from the LPF 121g is subtracted from the G signal gykl from the gradation conversion circuit 11 in the subtractor 122g. In the subtractor 122b, the B signal bzkl from the LPF 121b is subtracted from the B signal bykl from the gradation conversion circuit 11. Therefore, the noise component nrkl (= rykl-rzkl) of the R signal from the subtractor 122r, the noise component ngkl (= gykl-gzkl) of the G signal from the subtractor 122g, and the noise component nbkl (= bykl-bzkl) is output.

Then, in the coring unit 123r, the input noise component ngkl is subjected to coring processing according to the input / output relationship of FIG. 3 and converted into the edge component erkl, and in the coring unit 123g, the input noise component ngkl. 3 is converted into an edge component egkl by the input / output relationship of FIG. 3, and the input noise component nbkl is subjected to the coring processing by the input / output relationship of FIG. 3 in the coring unit 123b. Are converted into edge components ebkl. That is, the coring unit 123r will be described as a representative example. The noise component nrkl is converted into the edge component erkl and output as shown in the following equation.
erkl = nrkl + α (nrkl ≦ −α)
erkl = 0 (-α <nrkl <α)
erkl = nrkl-α (α ≦ nrkl)

  When the edge components erkl, egkl, and ebkl are obtained in the coring portions 123r, 123g, and 123b in this way, the edge enhancement factor β is applied to the edge component erkl in the multiplication unit 124r, and the edge enhancement is applied to the edge component egkl in the multiplication unit 124g. The amount β is multiplied by the edge enhancement amount β by the edge component ebkl by the multiplication unit 124b. In addition, the edge component β × erkl edge-enhanced is added to the noise-removed R signal rzkl in the adder 125r, and the edge component β × egkl edge-enhanced to the noise-removed G signal gzkl in the adder 125g. Are added, and the adder 125b adds the edge component β × ebkl with edge enhancement to the B signal bzkl from which noise has been removed. Therefore, RGB signals with edge enhancement are output from the adders 125r, 125g, and 125b.

  At this time, the state of the RGB signal in each part of the coring circuit 12 changes as shown in FIG. That is, when an RGB signal as shown in FIG. 14A is input, noise is removed by the LPFs 121r, 121g, and 121b, and an RGB signal as shown in FIG. 14B is obtained. Further, in each of the subtractors 122r, 122g, 122b, the RGB signal from the gradation conversion circuit 11 is subtracted by the RGB signal from the LPF 121r, 121g, 121b, so that the noise component of the RGB signal as shown in FIG. Is obtained. Then, the noise components of the RGB signals are input to the coring circuits 123r, 123g, and 123b to obtain edge components as shown in FIG. 14D, and then edge enhancement is performed by the multipliers 124r, 124g, and 124b. The amount β is multiplied and converted into an edge component as shown in FIG. The edge components of the RGB signals from the multipliers 124r, 124g, and 124b are added to the RGB signals from the LPFs 121r, 121g, and 121b by the adders 125r, 125g, and 125b, so that edge enhancement is performed as shown in FIG. RGB signals are output.

  In the present embodiment, the gradation conversion circuit 11 lowers the gradation of a region where the photoelectric conversion characteristic is a linear conversion characteristic and increases the gradation of a region where the photoelectric conversion characteristic is a logarithmic conversion characteristic. As a result, the gradation of the photoelectric conversion characteristics in the entire luminance range is made uniform, but the gradation of the photoelectric conversion characteristics in the entire luminance range is reduced only by reducing the gradation of the area in which the photoelectric conversion characteristics become the linear conversion characteristics. The tonality may be uniform, or the gradation of the photoelectric conversion characteristics in the entire luminance range may be uniform only by increasing the gradation of the area where the photoelectric conversion characteristics are logarithmic conversion characteristics. It doesn't matter.

  In the present embodiment, the color correction coefficient in the color correction circuit 10 is set for three regions, a linear region, a linear / logarithmic region, and a logarithmic region, and the gradation conversion arithmetic expression in the gradation converting circuit 11 is linear. The LPF coefficient, the coring coefficient α, and the edge enhancement amount β in the coring circuit 12 are set for the two areas of the linear area and the logarithmic area. It is not limited to such area division. Therefore, the color correction coefficient in the color correction circuit 10 may be set for two regions of a linear region and a logarithmic region, and an arithmetic expression for gradation conversion in the gradation conversion circuit 11 is a linear region and a logarithmic region. The LPF coefficient, the coring coefficient α, and the edge enhancement amount β in the coring circuit 12 may be set for the three areas of the linear / logarithmic area and the logarithmic area. It may be set for three logarithmic areas.

  When the LPF coefficient, the coring coefficient α, and the edge enhancement amount β in the coring circuit 12 are set for the three regions of the linear region, the linear / logarithmic region, and the logarithmic region, the LPF coefficient and the core in the linear / logarithmic region are set. The ring coefficient α and the edge enhancement amount β may be average values of the LPF coefficient, the coring coefficient α, and the edge enhancement amount β in the linear region and the logarithmic region. Further, the LPF coefficient may be a 3 × 3 matrix coefficient.

  Further, when the photoelectric conversion characteristics of the solid-state imaging device 2 are switched by the dynamic range control signal from the overall control unit 13, based on the dynamic range control signal, the color correction coefficient in the color correction circuit 10 and the gradation conversion circuit 11 An arithmetic expression for gradation conversion, an LPF coefficient in the coring circuit 12, a coring coefficient α, and an edge enhancement amount β may be set. Therefore, when the photoelectric conversion characteristics of the solid-state imaging device 2 are selected and set from the m photoelectric conversion characteristics by the dynamic range control signal from the overall control unit 13, each of the m photoelectric conversion characteristics to be selected is set. In addition, the color correction coefficient, the gradation conversion arithmetic expression, the LPF coefficient, the coring coefficient α, and the edge enhancement amount β are set for each region.

  That is, for example, as described above, color correction coefficients are set for three regions of a linear region, a linear / logarithmic region, and a logarithmic region, and an arithmetic expression for gradation conversion, an LPF coefficient, a coring coefficient α, and edge enhancement are used. When the amount β is set for two regions, a linear region and a logarithmic region, the overall control unit 13 includes m groups of color correction coefficients corresponding to the three regions, and an operation for gradation conversion corresponding to the two regions. Formula, LPF coefficient, coring coefficient α, and edge enhancement amount β are provided for m groups. Then, the color correction coefficient corresponding to the three areas according to the photoelectric conversion characteristics of the solid-state imaging device 2 selected based on the dynamic range control signal, the arithmetic expression for gradation conversion corresponding to the two areas, the LPF coefficient, and the coring coefficient α and edge enhancement amount β are extracted and provided to the color correction circuit 10, the gradation conversion circuit 11, and the coring circuit 12. Furthermore, at this time, the overall control unit 13 sets threshold levels V1 to V3 for identifying regions of photoelectric conversion characteristics according to the photoelectric conversion characteristics of the solid-state imaging device 2 selected based on the dynamic range control signal. The set threshold levels V1 to V3 are also given to the color correction circuit 10 and the gradation conversion circuit 11.

  In the present embodiment, the photoelectric conversion characteristic of the solid-state imaging device 2 is changed from the linear conversion characteristic to the logarithmic conversion characteristic. However, the linear conversion characteristic may be different between the high luminance side and the low luminance side. . At this time, the photoelectric conversion characteristics are switched by switching the exposure amount to each pixel of the solid-state imaging device 2. That is, the photoelectric conversion characteristics can be switched by switching the aperture ratio of the lens by the diaphragm and the exposure time.

  Further, in the present embodiment, a single-plate solid-state image pickup device in which a plurality of types of color filters are provided in one solid-state image pickup device. For example, a three-plate solid-state image pickup device including a solid-state image pickup device for each RGB color filter As described above, a solid-state imaging device having the same color filter may be provided for each type of color filter. At this time, the color interpolation circuit can be removed. Further, the solid-state imaging device may be a solid-state imaging device that outputs a monochrome image signal without a color filter.

  Furthermore, in the above-described imaging apparatus and imaging system, each parameter used for the signal processing operation is stored in accordance with a plurality of types of photoelectric conversion characteristics. However, a reference value for a photoelectric conversion characteristic serving as a reference is set for each parameter. In addition to storing the values, the values of the parameters may be calculated from the stored reference values based on the relationship with the photoelectric conversion characteristics serving as the reference.

<Imaging system>
In the imaging device described above, various signal processing including white balance processing is performed in the imaging device, but data obtained by the imaging device is given without performing these various signal processes in the imaging device. An imaging system that performs the above-described various signal processes in a computer can also be configured. Hereinafter, an imaging system including such an imaging apparatus and a computer will be described with reference to the drawings.

  As shown in FIG. 15, this imaging system receives an RGB signal output from the imaging device 200 by imaging an object and generates RGB signals, and performs various signal processing including white balance processing. And a computer 250. At this time, the imaging apparatus 200 performs black reference correction and FPN correction on the RGB signal obtained by imaging the subject, and detects an AE evaluation value for setting the exposure amount. On the other hand, the computer 250 detects the WB evaluation value and performs various signal processing after the white balance processing on the RGB signal given from the imaging device 200.

  In such an imaging system, the imaging apparatus 200 includes an optical system 1, a solid-state imaging device 2, an amplifier 3, an AD conversion circuit 4, a black reference correction circuit 5, an FPN correction circuit 6, an aperture control unit 14, and a timing generation circuit 15. An AE evaluation value detection circuit 7a for detecting an AE evaluation value, an overall control unit 13a for controlling each block, a memory 201 for temporarily storing RGB signals corrected by the FPN correction circuit 6, and a memory An input / output interface 202 that reads out the RGB signals temporarily stored in 201 and transmits them to the outside.

  The computer 250 is a PC (Personal Computer), PDA (Personal Digital Assistant), or the like, and includes a CPU (Central Processing Unit) 251 that performs data processing operations, a memory 252 that temporarily stores data, an application, It includes a hard disk 253 for storing data, an input unit 254 such as a keyboard and mouse for inputting data, a display 255 for displaying video, and an input / output interface 256 for exchanging data with the outside.

  As described above, when the imaging apparatus 200 and the computer 250 are respectively configured, an application for performing various signal processing after the white balance processing on the RGB signals in the computer 250 is stored in the hard disk 253. By reading this application from the hard disk 253 and storing it in the memory 252, it is possible to perform the WB evaluation value detection operation and various signal processing after the white balance processing. Also, by starting this application, a user interface (UI) for the user to specify parameters in various signal processing is displayed on the display 255.

  Hereinafter, operations of the imaging apparatus 200 and the computer 250 in the imaging system will be described with reference to the drawings.

  First, in the imaging apparatus 200, when light enters the solid-state imaging device 2 as in the imaging apparatus of FIG. 1, an RGB signal that is an analog signal is generated, amplified by the amplifier 3, and then A / D. The signal is converted into a digital signal by the conversion circuit 4. The RGB signal which is the digital signal is obtained by subtracting the FPN component stored in the FPN correction circuit 6 after the black level which is the lowest luminance value is corrected to the reference value in the black reference correction circuit 5, thereby obtaining the FPN. Ingredients are removed. Then, the RGB signal from which the FPN component is removed is stored in the memory 201. When the RGB signal from which the FPN component is removed is given to the AE evaluation value detection circuit 7a when operating in this way, the luminance range of the subject is confirmed and the aperture of the diaphragm 1a set by the diaphragm controller 14 is determined. An AE evaluation value is detected.

In addition, the imaging apparatus 200 checks the dynamic range information indicating the photoelectric conversion characteristics set for the solid-state imaging device 2 when performing the imaging operation for each captured image, and stores it in the memory 201 together with the RGB signal of the image. Store. This dynamic range information includes at least one of the following four pieces of information (1) and (2) and at least one of (3) and (4), or (1) to Only at least one of (4) is provided.
(1) Inflection point Vth of the photoelectric conversion characteristics of the solid-state imaging device 2
(2) Mode number M for setting the photoelectric conversion characteristics of the solid-state imaging device 2
(3) Coefficients A, C, γ, and δ representing the photoelectric conversion characteristics of the solid-state imaging device 2 (this approximate expression will be described later)
(4) Imaging conditions of the imaging apparatus 200

  In these dynamic range information, the inflection point Vth of the photoelectric conversion characteristic of the solid-state image sensor 2 indicates the signal level of the RGB signal when the photoelectric conversion characteristic logarithmic conversion characteristic of the solid-state image sensor 2 is changed to the linear conversion characteristic. Further, the mode number M for setting the photoelectric conversion characteristics of the solid-state imaging device 2 is set on the computer 250 side, so that the photoelectric conversion characteristics of the solid-state imaging device 2 of the imaging apparatus 200 are set. Is. The coefficients of the approximate expression indicating the photoelectric conversion characteristics of the solid-state imaging device 2 are the coefficients A and C in the equation (a) V = A × L + C representing the linear conversion characteristic, and the equation (b) representing the logarithmic conversion characteristic V = It is constituted by coefficients γ and δ in γ × ln (L) + δ. Further, when the imaging condition of the imaging apparatus 200 is set, the voltage difference ΔVPS between the voltage values VH and VL of the signal φVPS, the exposure time tx to the solid-state imaging device 2, the aperture ratio rx of the diaphragm 1a, and the solid-state imaging device 2 The amplification factor ax of the internal amplifier and the amplifier 3 is dynamic range information. When the imaging conditions of the imaging apparatus 200 are dynamic range information, the relationship between the coefficient A in (A) and (exposure time × aperture ratio of the diaphragm 1a × amplification factor) is known, and (B) The voltage difference ΔVPS between the coefficient δ and the signal φVPS and the relationship between the coefficient γ and the amplification factor are known.

  In addition, when the application 250 is read from the hard disk 253 and the operation designated by the application is started by operating the input unit 254, the computer 250 displays the UI on the display 255 as shown in FIG. A screen is displayed. The screen of FIG. 16 includes an image display area X1 for displaying an image obtained by imaging, an image processing condition setting area X2 for setting image processing conditions such as image sharpness and color shading, and the like. Color correction is performed for an image acquisition button X3 for instructing the start of capturing an image captured by the imaging apparatus 200, a WB setting region X4 for setting a photoelectric conversion characteristic that is used as a reference when performing white balance processing. A gradation characteristic display region X5 indicating gradation characteristics after gradation conversion performed on the RGB signal signals is displayed.

  At this time, when the sharpness of the image is changed by operating the scroll bar SB1 in the image processing condition setting region X2, the edge enhancement amount β for setting the edge component in the coring circuit 12 is changed, and When the color of the image is changed by operating the scroll bar SB2, the matrix coefficients a1 to a3, b1 to b3, and c1 to c3 in the color correction process are changed. At this time, the edge enhancement amount obtained based on the dynamic range information given from the imaging device 200 or a matrix coefficient in the color correction processing may be changed as a reference, or the edge enhancement amount of a predetermined value set in advance or It may be changed based on the matrix coefficient in the color correction process.

  In addition, when the three option buttons Ob1 to Ob3 are designated in the image processing condition setting area X2, the edge enhancement amount β and the matrix coefficients a1 to a3, b1 to b3, and c1 designated by the scroll bars SB1 and SB2 respectively. An image to be a target using ~ c3 is designated. That is, when the edge enhancement amount β and the matrix coefficients a1 to a3, b1 to b3, and c1 to c3 specified by the scroll bars SB1 and SB2 are used, and the option button Ob1 is specified, the obtained image is photoelectrically converted. When the option button Ob2 is designated, the image is obtained only when the photoelectric conversion characteristic is the linear conversion characteristic. Further, when the option button Ob3 is designated, it indicates that the obtained image is performed only when the photoelectric conversion characteristic is the logarithmic conversion characteristic.

  In addition, by specifying the five option buttons Wb1 to Wb5 in the WB setting area X4, WB evaluation values wr and wb for performing white balance processing are set. That is, when the option button Wb1 is designated, it is set to perform white balance processing using the WB evaluation values wr and wb that are automatically set based on the RGB signals given from the imaging device 200, and the option button Wb2 When .about.Wb5 is designated, white balance processing is set to be performed using WB evaluation values wr and wb which are preset values. When option buttons Wb2 to WB5 are designated, WB evaluation values wr and wb corresponding to the image under the fluorescent lamp, WB evaluation values wr and wb corresponding to the image under the incandescent lamp, and when the weather is sunny outside, respectively. The WB evaluation values wr and wb corresponding to the image at, and the WB evaluation values wr and wb corresponding to the image at the time of outdoor cloudy weather are set.

  In the gradation characteristic display area X5, the displayed gradation characteristic indicates a relationship before and after conversion when gradation conversion is performed on each of the RGB signals subjected to color correction processing. Then, by changing the position of the inflection point that switches from the linear conversion characteristic to the logarithmic conversion characteristic in the displayed gradation characteristic, in the dynamic range, the gradation size in the linear region that is a signal by the linear conversion characteristic The gradation size in the logarithmic region that becomes a signal based on the logarithmic conversion characteristic can be set. The position of the inflection point of the gradation characteristic may be set with reference to a position obtained based on dynamic range information given from the imaging device 200, or set with reference to a predetermined position set in advance. It doesn't matter if it is done. Also, a plurality of LUTs indicating gradation characteristics may be provided, and the LUT displayed in the gradation characteristic display area X5 may be changed by switching the LUTs.

  As described above, when the application is started in the computer 250 and the UI as shown in FIG. 16 is displayed on the display 255, the computer 250 operates according to the flowchart of FIG. It is displayed on the display 255. First, in the computer 250, when the input unit 254 is operated and the image acquisition button X3 in the UI screen is pressed (STEP 1), the imaging apparatus 200 in which the input / output interface 202 is connected to the input / output interface 256 is displayed. A line with the imaging device 200 is secured (STEP 2). At this time, a line can be secured by repeating the communication request and response between the imaging apparatus 200 and the computer 250.

  Then, it is confirmed whether or not a line with the imaging apparatus 200 is secured (STEP 3). When it is confirmed that a line with the imaging apparatus 200 is secured (Yes), the imaging apparatus 200 is imaged. The output of the received RGB signal is requested (STEP 4). At this time, when receiving an RGB signal output request from the computer 250, the imaging apparatus 200 reads the RGB signal and dynamic range information for one frame stored in the memory 201 and outputs them from the input / output interface 202. . At this time, when frame data based on RGB signals for one frame is generated and output, dynamic range information is added to the header portion of the frame data.

  When the frame data from the imaging device 200 is received by the input / output interface 256 and stored in the temporary storage area of the hard disk 253 (STEP 5), the dynamic range information is acquired from the header portion of this frame data (STEP 6). . Based on the acquired dynamic range information, parameters used for each signal processing are set, and display corresponding to the set parameters is performed on the UI image of the display 255 (STEP 7).

  That is, WB detection values wr and wb for white balance processing, matrix coefficients a1 to a3, b1 to b3 and c1 to c3 for color correction processing, tone characteristics for tone conversion processing, and edge enhancement amount β for coring processing, respectively. In addition to the setting, display according to each set parameter is performed in each of the image processing condition setting area X2, the WB setting area X4, and the WB setting area X4.

  Further, when each parameter is set in this way, in order to set the color correction coefficient used in the color correction processing for each RGB signal of each pixel, the inflection point Vth obtained from the dynamic range information, the mode number M, the coefficient A , C, γ, δ, or threshold values V1 and V2 (see FIG. 9) used to divide into three regions, a linear region, a linear / logarithmic region, and a logarithmic region, from the imaging conditions of the imaging apparatus 200.

  At this time, when the inflection point Vth is included in the dynamic range information, a threshold value V1 is set by subtracting a predetermined offset amount from the inflection point Vth, and a predetermined offset amount is added to the inflection point Vth. Level V2 is set. When the mode number M is included in the dynamic range information, the inflection point Vth corresponding to the mode number M stored in advance with the application of the hard disk 253 is read, and a predetermined offset amount is added to or subtracted from the inflection point Vth. Threshold levels V1 and V2 are set. When the coefficients A, C, γ, and δ are included in the dynamic range information, the inflection point Vth is first set based on the photoelectric conversion characteristics confirmed from the coefficients A, C, γ, and δ, and then the inflection point. Threshold levels V1 and V2 are set by adding and subtracting a predetermined offset amount to Vth.

Further, when the shooting conditions (voltage difference ΔVPS, exposure time tx, aperture ratio rx, amplification factor ax) are included in the dynamic range information, coefficients A0, C0, γ0 serving as reference values, exposure time tx0, aperture ratio rx0, The amplification factor ax0 is stored together with the application of the hard disk 253 in advance. First, the coefficients A, C, γ, and δ representing the photoelectric conversion characteristics are set by calculating based on the following formulas (c) to (f). . Note that δ0 is a value corresponding to the voltage difference ΔVPS of the signal φVPS stored together with the application of the hard disk 253 in advance. Thereafter, after setting the inflection point Vth from the photoelectric conversion characteristics confirmed by the coefficients A, C, γ, and δ, threshold values V1 and V2 are set by adding and subtracting a predetermined offset amount to the inflection point Vth.
A = A0 × (tx / tx0) × (rx / rx0) × (ax / ax0) (C)
C = C0 (D)
γ = γ0 × (tx / tx0) (e)
δ = δ0 × (tx / tx0) (f)

  Further, in order to confirm the gradation characteristics used in the gradation conversion process, the inflection point Vth obtained from the dynamic range information, the mode number M, the coefficients A, C, γ, δ, or the imaging conditions of the imaging apparatus 200 A threshold level V3 (see FIG. 11) for determining the photoelectric conversion characteristic of each RGB signal of each pixel is set. Also at this time, when the inflection point Vth is obtained in the same manner as the above-described threshold levels V1 and V2, the obtained inflection point Vth is set as the threshold level V3 as it is.

  When the parameters set in this way are given to the application stored in the memory 252, the signal processing according to the set parameters is performed on the RGB signals obtained from the frame data received in STEP 5 by this application. Performed (STEP 8). In other words, each signal processing is performed in the order of white balance processing, color interpolation processing, color correction processing, gradation conversion processing, and coring processing on the RGB signal obtained from the frame data. Each signal processing is the same as the signal processing in the imaging apparatus shown in FIG.

  The frame data based on the RGB signals subjected to the signal processing is stored in the hard disk 253, and an image based on the frame data is displayed in the image display area X1 of the display 255 (STEP 9). Then, by operating the input unit 254, it is confirmed whether or not the operation of the image processing condition setting area X2, the WB setting area X4, and the WB setting area X4 has been instructed to change each parameter (STEP 10). . If parameter change is instructed (Yes), each parameter is changed in accordance with the instructed change amount (STEP 11), and then the process proceeds to STEP 8.

  If it is not confirmed in STEP 3 that the line has been secured (No), a display indicating that the line is not connected is displayed on the display 255 (STEP 12), and the process proceeds to STEP 2. Further, in STEP 10, when the instruction for changing each parameter is not confirmed (No), the operation is terminated.

  As described above, according to the present system, each signal processing with a calculation load is performed on the computer 250 side, so that the configuration of the imaging apparatus 200 is simplified and the processing speed can be increased. In this system, the signal output from the imaging apparatus 200 to the computer 250 may be not only frame data based on the above-described still image but also frame data or stream data based on a moving image. At this time, when a moving image is formed by a plurality of still image frame data, dynamic range information is recorded in the header of each frame data, and the moving image is generated by stream data that also performs image compression on the time axis. In this case, the dynamic range information is transmitted as information different from the stream data.

  Further, in the imaging apparatus 200, the overall control unit 13 a converts the format of frame data or stream data based on RGB signals transmitted to the computer 250 into a compressed image such as JPEG, motion JPEG, or MPEG, and outputs it from the input / output interface 202. It does n’t matter what you do. Further, in the computer 250, the UI displayed on the display 255 displays a histogram of the image displayed in the image display area X1, and a display for identifying the pixel area based on the logarithmic conversion characteristic and the pixel area based on the linear conversion characteristic. You may make it perform.

  In the imaging system of FIG. 15, the imaging apparatus 200 includes the black reference correction circuit 5 and the FPN component correction circuit 6. However, the imaging apparatus 200 has the black reference correction circuit 5 and the FPN component from the configuration of FIG. The configuration may be such that the correction circuit 6 is omitted, and the computer 250 performs black reference correction processing and FPN correction processing. At this time, the black level for performing the black reference correction process and the FPN component for performing the FPN correction process may be transmitted together with the dynamic range information.

  Furthermore, when the black reference correction process and the FPN correction process are performed in the computer 250 as described above, the computer 250 may store the black level and the FPN component in the hard disk 253 together with the application. At this time, a plurality of black levels are also stored for the inflection point Vth of the photoelectric conversion characteristic, and the inflection point of the photoelectric conversion characteristic obtained from the previously stored black level by the dynamic range information as described above. A black level corresponding to Vth is selected. Further, since the FPN component differs depending on the solid-state imaging device 2 in each imaging device 200, it is stored for each imaging device 200.

  Further, when the computer 250 performs white balance processing, the RGB signal is converted into a signal whose photoelectric conversion characteristics are all logarithmic conversion characteristics or a signal whose photoelectric conversion characteristics are all linear conversion characteristics, and then the white balance processing is performed. It doesn't matter what you do. That is, when all the photoelectric conversion characteristics are converted into a signal with logarithmic conversion characteristics, the signal value obtained by the linear conversion characteristics is converted into a signal value with logarithmic conversion characteristics, and all the photoelectric conversion characteristics are signals with the linear conversion characteristics. In the case of conversion to, the signal value obtained by the logarithmic conversion characteristic is converted to a signal value by the linear conversion characteristic. Note that the calculation for such conversion processing may be performed with reference to the LUT, or may be performed by an arithmetic expression.

  In addition, in the imaging system of FIG. 15, frame data obtained by imaging with the imaging device 200 is transferred to the computer 250 by the communication connection between the input / output interface 202 of the imaging device 200 and the input / output interface 256 of the computer 250. Although the imaging apparatus 200 includes a recording unit that records frame data or stream data on a recording medium such as a flexible disk, a compact flash, or a memory card, the computer 250 uses the frame data or stream of the recording medium. A reproducing unit that reads data may be provided. In this way, frame data or stream data obtained by imaging can be transferred from the imaging device 200 to the computer 250 via the recording medium.

These are block diagrams which show the structure of the imaging device which is embodiment of this invention. These are figures which show the arrangement | sequence of the color filter provided in a solid-state image sensor. These are graphs showing level conversion characteristics for edge components in a coring circuit. FIG. 3 is a circuit block diagram for explaining the overall configuration of the solid-state imaging device. These are circuit diagrams which show one structural example of the pixel which comprises the solid-state image sensor of FIG. These are timing charts showing the operation of the pixel of FIG. These are graphs showing the relationship between the luminance of a subject and the output of a pixel. FIG. 3 is a block diagram showing an internal configuration of a color correction circuit. These are graphs showing the relationship between luminance values of RGB signals and signal levels. FIG. 3 is a block diagram showing an internal configuration of a gradation conversion circuit. These are graphs showing the relationship between luminance values of RGB signals and signal levels. These are graphs showing the relationship between input / output and photoelectric conversion characteristics in the gradation conversion circuit. These are block diagrams which show the internal structure of a coring circuit. These are the figures for demonstrating the state transition of the signal in each part of a coring circuit. These are block diagrams which show the structure of the imaging system which is embodiment of this invention. These are figures which show an example of the screen displayed on the computer of the imaging system of FIG. FIG. 16 is a flowchart showing a signal processing operation in a computer of the imaging system of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Optical system 2 Solid-state image sensor 3 Amplifier 4 AD conversion circuit 5 Black reference correction circuit 6 FPN correction circuit 7 AE / WB evaluation value detection circuit 8 WB control circuit 9 Color interpolation circuit 10 Color correction circuit 11 Gradation conversion circuit 12 Coring Circuit 13 Overall control unit 14 Aperture control unit 15 Timing generation circuit

Claims (6)

E Bei a solid-state imaging device having a plurality of pixels photoelectric conversion characteristic is switched in accordance with a plurality of luminance ranges, and a signal processing circuit for processing the image signal output from the solid-state imaging device, a
The signal processing circuit includes a plurality of types of parameters corresponding to the respective luminance ranges as parameters for processing the image signal,
Confirming the luminance range of the image signal processed by the signal processing circuit, and selecting a parameter corresponding to the photoelectric conversion characteristic of the confirmed luminance range from the plurality of types of parameters, and processing parameters in the signal processing circuit In the imaging device set to
The image signal output from the solid-state imaging device is a plurality of types of color signals, and the plurality of types of color signals are generated for one pixel and supplied to the signal processing circuit.
The signal processing circuit converts a gradation of the image signal and a gradation conversion circuit having a first parameter for performing gradation conversion, and removes a noise component of the image signal and performs edge enhancement and noise. Color correction is performed by giving a correlation between a noise removal circuit having a second parameter for performing component removal and / or edge enhancement, and another color signal for each type of the color signal of each pixel. A color correction circuit having a third parameter for performing
An imaging apparatus , wherein at least one of the first parameter, the second parameter, and the third parameter is a plurality of types of parameters set in accordance with each of the luminance ranges . The gradation conversion circuit is
A luminance range detector for confirming the luminance range of the input image signal;
A parameter setting unit for setting the first parameter for performing gradation conversion according to the luminance range detected by the luminance range detection unit;
A gradation conversion unit that performs gradation conversion of the input image signal in accordance with the first parameter set by the parameter setting unit;
The imaging apparatus according to claim 1, further comprising: The noise removing circuit is
A low pass filter for removing noise components from the input image signal;
A subtractor that subtracts the image signal that has passed through the low-pass filter from the input image signal to extract the noise component;
Coring the noise component obtained by the subtractor to remove a noise component other than the edge portion;
A multiplier that multiplies the edge component obtained by the coring unit by an edge enhancement amount that is a predetermined amount;
An adder that adds the edge component multiplied by the edge enhancement amount by the multiplier to the image signal from which noise components have been removed by the low-pass filter, and outputs an edge-enhanced image signal;
With
The second parameter is a matrix-like low-pass filter coefficient for removing noise components in the low-pass filter, a coring coefficient for removing noise components other than edge portions in the coring unit, and the multiplication The image pickup apparatus according to claim 1, wherein the image pickup device is one of edge enhancement amounts in a container. The color correction circuit is
For each type of color signal,
A luminance range detector for confirming the luminance range of the input color signal;
A parameter setting unit that sets the third parameter for performing color correction according to the luminance range detected by the luminance range detection unit;
A multiplier that multiplies each of all types of color signals of the same pixel as the input color signal by the third parameter set by the parameter setting unit;
An adder that adds all color signals of the same pixel multiplied by the third parameter by the multiplier and corrects the color of the input color signal;
The imaging apparatus according to claim 1, further comprising: The photoelectric conversion between a logarithmic conversion characteristic in which an electric signal logarithmically converted with respect to an incident light quantity is used as the image signal and a linear conversion characteristic in which an electric signal linearly converted with respect to the incident light quantity is used as the image signal. The imaging device according to claim 1, wherein the characteristics are switched according to luminance. While can select and switch the photoelectric conversion characteristics in the entire luminance range of the solid-state imaging device,
6. The imaging apparatus according to claim 1, wherein a plurality of types of parameters corresponding to each luminance range are set based on the photoelectric conversion characteristics in the selected entire luminance range. .

Images