JP2007189405A - Imaging apparatus and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus and an imaging method for reducing a conversion error to a minimum at the time of unification of an output signal state of an imaging element having two or more kinds of photoelectric conversion characteristics. <P>SOLUTION: The imaging apparatus 1 includes an imaging element 5 having two or more pixels for converting incident light to an electric signal under two or more kinds of conversion characteristics; a point-of-inflection control unit 9 for setting two or more points of inflection; a linear log conversion unit for converting an output signal of the imaging element 5 to a uniformly converted state under one kind of conversion characteristics; and a linearization signal correction unit 11 for correcting the conversion error in a region near to the switching point of the output signal of the pixel to be interpolated, by using a region corresponding to the output signal of nearby pixels having different switching points. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an imaging method, and more particularly, to an imaging apparatus and an imaging method using imaging elements having a plurality of types of photoelectric conversion characteristics.

  Conventionally, an imaging device such as a digital camera has been provided with an imaging element having a plurality of pixels that convert incident light into an electrical signal. In recent years, image sensors having a plurality of types of photoelectric conversion characteristics have been proposed.

  As an image sensor having a plurality of types of photoelectric conversion characteristics, for example, there is a linear log conversion type sensor that switches between a linear conversion operation for linearly converting incident light into an electrical signal and a logarithmic conversion operation for logarithmic conversion based on the amount of incident light ( Patent Document 1). According to this, since the dynamic range of the electric signal is wide compared to an image sensor that performs only a linear conversion operation, it is possible to express all luminance information as an electric signal even when shooting a subject with a wide luminance range. Become.

  However, when an image pickup device having a plurality of types of photoelectric conversion characteristics is used, an output signal that varies depending on the plurality of characteristics can be obtained.

  Therefore, for example, in the linear log conversion type sensor, the output signal in the logarithmic region out of the output signal including the linear region and the logarithmic region is linearized by calculation or conversion using LUT, and the entire output signal is converted into a single linear conversion. Processing to make a state is performed.

In such a linearization process, a graph obtained by formulating output signals in the linear domain and logarithmic domain is used as an output signal model.
JP 2005-110234 A

  However, the output signal of the actual image sensor changes smoothly from the linear region to the logarithmic region, and is logarithmically transformed with the linearly transformed output signal in the region near the inflection point that is the boundary between the linear region and the logarithmic region. There are areas where output signals are mixed. Therefore, when linearization processing is performed according to the above output signal model, there is a problem that a deviation from the output signal model occurs in the region near the inflection point, resulting in a conversion error.

  An object of the present invention is to minimize a conversion error when converting an output signal of an image sensor into a single linear signal in an image pickup apparatus and an image pickup method using an image sensor having a plurality of types of photoelectric conversion characteristics. It is an object of the present invention to provide an imaging apparatus and an imaging method that enable this.

  In order to solve the above-mentioned problem, an invention according to claim 1 is an imaging device, wherein the imaging device has a plurality of pixels for converting incident light into an electric signal by a plurality of types of conversion characteristics, and switching between the plurality of conversion characteristics. A switching point control unit for setting a point, a characteristic converting unit for converting the output signal of the image sensor into a state converted uniformly by a single conversion characteristic, and conversion of the output signal of the pixel of interest in the region near the switching point And a signal correction unit that corrects an error using a corresponding region of an output signal of a neighboring pixel having a different switching point.

  According to the first aspect of the present invention, by setting a plurality of switching points of conversion characteristics, it is possible to obtain output signals from a plurality of pixels that are correlated with each other and that do not overlap in the vicinity of the switching point. As a result, it is possible to correct the conversion error that occurs in the area near the switching point due to the characteristic conversion by using an area other than the area near the switching point in the output signal from other neighboring pixels corresponding to the area near the switching point. It becomes.

  That is, when characteristic conversion is performed using an output signal model obtained by formulating the output signal of each characteristic, the actual output signal changes gently, so that a deviation from the output signal model occurs in the region near the switching point. Therefore, the conversion error can be corrected by obtaining an output signal when there is no conversion error by using a region other than the switching point vicinity region in the output signals from other neighboring pixels.

  The invention according to claim 2 is the imaging apparatus according to claim 1, wherein the switching point control unit divides the plurality of pixels into regular groups, and a switching point vicinity region of the conversion characteristic is determined for each group. The switching point is set so as not to overlap.

  According to the invention of claim 2, by dividing a plurality of pixels into a regular group and setting a switching point for each group, a plurality of switching points of conversion characteristics are suppressed while suppressing an influence on a photographed image. Can be set.

  A third aspect of the present invention is the imaging apparatus according to the first aspect, wherein the switching point control unit sets the switching point so that regions near the switching point of the conversion characteristic do not overlap each other for a plurality of consecutive frames. It is characterized by doing.

  According to the third aspect of the present invention, when a moving image is shot, by setting a switching point for each of a plurality of frames, setting a switching point for a plurality of conversion characteristics while suppressing an influence on a captured image. Is possible.

  The invention according to claim 4 is the imaging apparatus according to claim 3, wherein the switching point control unit divides the plurality of pixels into regular groups, and a switching point vicinity region of the conversion characteristic is determined for each group. The switching point is set so as not to overlap, and the switching point is set by inverting the switching point for each frame.

  According to the fourth aspect of the present invention, when shooting a moving image, it is possible to easily set different switching points for a plurality of frames by inverting the switching points set for each group of pixels for each frame. Become.

  The invention according to claim 5 is the imaging apparatus according to claim 2 or 4, wherein the switching point control unit sets two types of the switching points for each vertical pixel column of the imaging device. Features.

  According to the fifth aspect of the present invention, it is possible to correct the conversion error by using the output signals from other neighboring pixels by setting two types of switching points for each vertical pixel column. . Thus, for example, when each pixel includes an RGB filter, correction can be performed between the same colors or between different colors.

  The invention according to claim 6 is the imaging apparatus according to claim 2 or 4, wherein the switching point control unit sets four types of switching points for each vertical pixel column of the imaging element. Features.

  According to the sixth aspect of the invention, by setting four types of switching points for each vertical pixel column, it becomes possible to correct conversion errors using output signals from other neighboring pixels. . Thus, for example, when each pixel is provided with an RGB filter, it is possible to perform correction between the same colors for each of RGB.

  The invention according to claim 7 is the imaging apparatus according to claim 2 or claim 4, wherein the switching point control unit sets two types of the switching points for each horizontal pixel row of the imaging element. Features.

  According to the seventh aspect of the present invention, by setting two kinds of switching points for each horizontal pixel row, it becomes possible to correct the conversion error using the output signals from other neighboring pixels. . Thus, for example, when each pixel includes an RGB filter, correction can be performed between the same colors or between different colors. In addition, correction can be performed without sacrificing the vertical resolution.

  The invention according to claim 8 is the imaging apparatus according to claim 2 or 4, wherein the switching point control unit sets four types of switching points for each pixel row next to the imaging element. Features.

  According to the invention described in claim 8, by setting four kinds of switching points for each horizontal pixel row, it becomes possible to correct the conversion error using the output signals from other neighboring pixels. . Thus, for example, when each pixel is provided with an RGB filter, it is possible to perform correction between the same colors for each of RGB. In addition, correction can be performed without sacrificing the vertical resolution.

  The invention according to claim 9 is the imaging apparatus according to any one of claims 1 to 8, wherein the signal correction unit outputs an output signal value of a pixel of interest in which the conversion error has occurred to the other neighborhood. The correction is performed so that the value corresponds to the interpolation value obtained from the output signal of the pixel.

  According to the ninth aspect of the invention, since a plurality of output signals from neighboring pixels having different switching points are correlated with each other, the output of the target pixel when there is no conversion error from the output signals from other neighboring pixels Find the interpolation value corresponding to the signal value and replace the output signal value of the pixel of interest with the interpolation value, or correct the conversion error by multiplying the coefficient according to the ratio between the output signal value of the pixel of interest and the interpolation value Is possible.

  According to a tenth aspect of the present invention, in the imaging device according to the ninth aspect, the signal correction unit performs correction when a difference between an output signal value of the target pixel and the interpolation value is outside an allowable error range. It is characterized by that.

  According to the tenth aspect of the present invention, the conversion is performed only when the difference between the output signal value of the target pixel and the interpolation value is outside the allowable error range, that is, when the conversion error is large enough to affect the captured image. Since the error is corrected, unnecessary processing can be omitted and signal processing can be made more efficient.

  The invention according to claim 11 is the imaging apparatus according to any one of claims 1 to 10, wherein at least one of the plurality of types of conversion characteristics is a logarithmic conversion characteristic. .

  According to the eleventh aspect of the present invention, in the image pickup apparatus including the image pickup element having the logarithmic conversion characteristic, the same effects as in the first to tenth aspects can be obtained.

  The invention according to claim 12 is the imaging apparatus according to any one of claims 1 to 10, wherein at least one of the plurality of types of conversion characteristics is a linear conversion characteristic. .

  According to the twelfth aspect of the present invention, in the image pickup apparatus including the image pickup element having the linear conversion characteristic, the same effects as those of the first to tenth aspects can be obtained.

  A thirteenth aspect of the present invention is the imaging apparatus according to any one of the first to tenth aspects, wherein the plurality of types of conversion characteristics are logarithmic conversion characteristics and linear conversion characteristics.

  According to the thirteenth aspect of the present invention, in the image pickup apparatus including the linear log conversion type sensor, the same effects as those of the first to tenth aspects can be obtained.

  The invention according to claim 14 is the imaging apparatus according to any one of claims 1 to 10, wherein the plurality of types of conversion characteristics are a plurality of linear conversion characteristics having different conversion coefficients. To do.

  According to the fourteenth aspect of the present invention, in the image pickup apparatus including the plurality of image pickup elements having the linear conversion characteristics having different conversion coefficients, the same effects as in the first to tenth aspects can be obtained.

  The invention according to claim 15 is an imaging method using an imaging device having a plurality of pixels for converting incident light into an electric signal by a plurality of types of conversion characteristics, and a switching point for setting a switching point for the plurality of conversion characteristics. A control step, a characteristic conversion step of converting the output signal of the image sensor into a state uniformly converted by a single conversion characteristic, and a conversion error in the vicinity of the switching point of the output signal of the target pixel. And a signal correction step of correcting using corresponding regions of output signals of different neighboring pixels.

  According to the fifteenth aspect of the present invention, by setting a plurality of conversion characteristic switching points, it is possible to obtain a plurality of output signals that are correlated with each other and that do not overlap in the vicinity of the switching points. As a result, it is possible to correct the conversion error that occurs in the area near the switching point due to the characteristic conversion by using an area other than the area near the switching point in the output signal from other neighboring pixels corresponding to the area near the switching point. It becomes.

  That is, when characteristic conversion is performed using an output signal model obtained by formulating the output signal of each characteristic, the actual output signal changes gently, so that a deviation from the output signal model occurs in the region near the switching point. Therefore, the conversion error can be corrected by obtaining an output signal when there is no conversion error by using a region other than the switching point vicinity region in the output signals from other neighboring pixels.

  A sixteenth aspect of the invention is the imaging method according to the fifteenth aspect of the invention, wherein the plurality of pixels are divided into regular groups in the switching point control step, and a switching point vicinity region of the conversion characteristic is determined for each group. The switching point is set so as not to overlap.

  According to the sixteenth aspect of the present invention, by dividing a plurality of pixels into a regular group and setting a switching point for each group, a plurality of switching points of conversion characteristics are suppressed while suppressing an influence on a photographed image. Can be set.

  According to a seventeenth aspect of the present invention, in the imaging method according to the fifteenth aspect, in the switching point control step, the switching points are set so that regions near the switching points of the conversion characteristics do not overlap each other for a plurality of consecutive frames. It is characterized by doing.

  According to the seventeenth aspect of the present invention, in the case of shooting a moving image, by setting a switching point for each of a plurality of frames, setting a switching point of a plurality of conversion characteristics while suppressing an influence on a captured image. Is possible.

  The invention according to claim 18 is the imaging method according to claim 17, wherein in the switching point control step, the plurality of pixels are divided into regular groups, and a switching point vicinity region of the conversion characteristic is determined for each group. The switching point is set so as not to overlap, and the switching point is set by inverting the switching point for each frame.

  According to the invention described in claim 18, when shooting a moving image, it is possible to easily set different switching points for a plurality of frames by inverting the switching points set for each group of pixels for each frame. Become.

  The invention according to claim 19 is the imaging method according to claim 16 or claim 18, wherein in the switching point control step, two types of the switching points are set for each vertical pixel column of the imaging element. Features.

  According to the nineteenth aspect of the present invention, by setting two types of switching points, it is possible to correct conversion errors using output signals from other neighboring pixels. Thus, for example, when each pixel includes an RGB filter, correction can be performed between the same colors or between different colors.

  The invention according to claim 20 is the imaging method according to claim 16 or claim 18, wherein in the switching point control step, four types of the switching points are set for each vertical pixel column of the imaging element. Features.

  According to the twentieth aspect of the present invention, by setting four types of switching points, it is possible to correct conversion errors using output signals from other neighboring pixels. Thus, for example, when each pixel is provided with an RGB filter, it is possible to perform correction between the same colors for each of RGB.

  The invention according to claim 21 is the imaging method according to claim 16 or claim 18, wherein in the switching point control step, two types of the switching points are set for each horizontal pixel row of the imaging element. Features.

  According to the twenty-first aspect of the present invention, by setting two types of switching points, it is possible to correct conversion errors using output signals from other neighboring pixels. Thus, for example, when each pixel includes an RGB filter, correction can be performed between the same colors or between different colors. In addition, correction can be performed without sacrificing the vertical resolution.

  According to a twenty-second aspect of the present invention, in the imaging method according to the sixteenth or eighteenth aspect, in the switching point control step, four types of the switching points are set for each horizontal pixel row of the imaging element. Features.

  According to the twenty-second aspect, by setting four types of switching points, it is possible to correct conversion errors using output signals from other neighboring pixels. Thus, for example, when each pixel is provided with an RGB filter, it is possible to perform correction between the same colors for each of RGB. In addition, correction can be performed without sacrificing the vertical resolution.

  The invention according to claim 23 is the imaging method according to any one of claims 15 to 22, wherein in the signal correction step, the output signal value of the target pixel in which the conversion error has occurred is set to the other neighborhood. The correction is performed so that the value corresponds to the interpolation value obtained from the output signal of the pixel.

  According to the twenty-third aspect of the present invention, since the plurality of output signals having different switching points are correlated with each other, this corresponds to the output signal value of the target pixel when there is no conversion error from the output signals from other neighboring pixels. Conversion error can be corrected by obtaining the interpolation value to be replaced and replacing the output signal value of the target pixel with the interpolation value or by multiplying the coefficient according to the ratio between the output signal value of the target pixel and the interpolation value. .

  According to a twenty-fourth aspect of the present invention, in the imaging method according to the twenty-third aspect, in the signal correction step, correction is performed when a difference between the output signal value of the target pixel and the interpolation value is outside an allowable error range. It is characterized by that.

  According to the twenty-fourth aspect of the present invention, the conversion is performed only when the difference between the output signal value of the target pixel and the interpolation value is outside the allowable error range, that is, when the conversion error is large enough to affect the captured image. Since the error is corrected, unnecessary processing can be omitted and signal processing can be made more efficient.

  The invention according to claim 25 is the imaging method according to any one of claims 15 to 24, wherein at least one of the plurality of types of conversion characteristics is a logarithmic conversion characteristic. .

  According to the twenty-fifth aspect of the present invention, in the image pickup apparatus including the image pickup element having the logarithmic conversion characteristic, the same effects as in the fifteenth to twenty-fourth aspects can be obtained.

  The invention according to claim 26 is the imaging method according to any one of claims 15 to 24, wherein at least one of the plurality of types of conversion characteristics is a linear conversion characteristic. .

  According to the twenty-sixth aspect of the present invention, in the image pickup apparatus including the image pickup element having linear conversion characteristics, the same effects as in the fifteenth to twenty-fourth aspects can be obtained.

  A twenty-seventh aspect of the present invention is the imaging method according to any one of the fifteenth to twenty-fourth aspects, wherein the plurality of types of conversion characteristics are logarithmic conversion characteristics and linear conversion characteristics.

  According to the twenty-seventh aspect of the present invention, in the imaging apparatus including the linear log conversion type sensor, the same effects as those of the fifteenth to twenty-fourth aspects can be obtained.

  The invention according to claim 28 is the imaging method according to any one of claims 15 to 24, wherein the plurality of types of conversion characteristics are a plurality of linear conversion characteristics having different conversion coefficients. To do.

  According to the twenty-eighth aspect of the present invention, in the image pickup apparatus including the plurality of image pickup elements having linear conversion characteristics having different conversion coefficients, the same effects as in the fifteenth to twenty-fourth aspects can be obtained.

  According to the first or fifteenth aspect of the present invention, correction is performed using a region other than the region near the switching point in the output signal from another neighboring pixel having a different switching point to minimize the conversion error in the region near the switching point. It can be suppressed to the limit.

  According to the second or sixteenth aspect of the present invention, it is possible to set a plurality of conversion characteristic switching points while suppressing the influence on the photographed image.

  According to the invention described in claim 3 or claim 17, it is possible to set a plurality of conversion characteristic switching points while suppressing the influence on the photographed image.

  According to the invention of claim 4 or claim 18, it is possible to easily set different switching points for a plurality of frames.

  According to the fifth or nineteenth aspect of the present invention, it is possible to correct the conversion error using output signals from other neighboring pixels having different switching points.

  According to the invention of claim 6 or claim 20, it is possible to correct the conversion error between the same colors by using the output signals from other neighboring pixels having different switching points.

  According to the seventh or twenty-first aspect of the present invention, it is possible to correct the conversion error using output signals from other neighboring pixels having different switching points. Further, the correction can be performed without sacrificing the vertical resolution.

  According to the invention described in claim 8 or claim 22, it is possible to correct the conversion error between the same colors using output signals from other neighboring pixels having different switching points. Further, the correction can be performed without sacrificing the vertical resolution.

  According to the ninth or twenty-third aspect of the present invention, it is possible to obtain an interpolation value using the correlation between a plurality of output signals having different switching points, and to correct the conversion error.

  According to the tenth or twenty-fourth aspect of the present invention, signal processing can be made efficient by omitting unnecessary processing.

  According to the eleventh or twenty-fifth aspect of the present invention, an effect similar to the above can be obtained in an imaging device including an imaging device having logarithmic conversion characteristics.

  According to the twelfth or twenty-sixth aspect of the present invention, the same effect as described above can be obtained in an imaging device including an imaging device having linear conversion characteristics.

  According to the thirteenth or twenty-seventh aspect of the present invention, the same effect as described above can be obtained in the imaging apparatus including the linear log conversion type sensor.

  According to the fourteenth or twenty-eighth aspect of the present invention, an effect similar to the above can be obtained in an imaging apparatus including an imaging device having a plurality of linear conversion characteristics having different conversion coefficients.

  Embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows a functional configuration of the imaging apparatus 1 according to the present embodiment.

  As shown in FIG. 1, the imaging apparatus 1 includes a system control unit 2. The system control unit 2 includes a CPU (Central Processing Unit), a RAM (Random Access Memory) composed of a rewritable semiconductor element, and a ROM (Read Only Memory) composed of a nonvolatile semiconductor memory.

  Further, each component of the imaging apparatus 1 is connected to the system control unit 2, and the system control unit 2 develops a processing program recorded in the ROM on the RAM and executes the processing program by the CPU. These components are driven and controlled.

As shown in FIG. 1, the system control unit 2 includes a lens unit 3, an aperture control unit 4, an image sensor 5, an amplifier 6, an AD converter (ADC) 7, a timing generation unit 8, an inflection point control unit 9, black A reference correction unit 10, a linearization signal correction unit 11, an AE / AWB evaluation value detection unit 12, and an image processing unit 13 are connected.

  The lens unit 3 includes a plurality of lenses that form a subject light image on the imaging surface of the image sensor 5 and a diaphragm unit that adjusts the amount of light collected by the lenses.

  The diaphragm control unit 4 drives and controls the diaphragm unit that adjusts the amount of light collected by the lens in the lens unit 3. That is, based on the control value input from the system control unit 2, the aperture control unit 4 closes the aperture unit after a predetermined exposure time has elapsed since the aperture unit was opened just before the imaging operation of the image sensor 5 was started, In addition, the incident light quantity is controlled by blocking the incident light to the image sensor 5 during non-imaging.

  The imaging element 5 captures incident light transmitted through the lens unit 3 by photoelectric conversion into an electric signal.

  Here, the imaging device of the present invention is an imaging device having a plurality of pixels that convert incident light into an electric signal by a plurality of types of conversion characteristics, and an output signal of the plurality of conversion characteristics continuously changes via a switching point. It is supposed to be.

  The imaging device 5 of the present embodiment is configured as a linear log conversion type sensor that switches between a linear conversion operation for linearly converting incident light into an electrical signal and a logarithmic conversion operation for logarithmic conversion based on the amount of incident light. In other words, the output signal of the image sensor 5 continuously changes from the linear region to the log region via the inflection point. Here, the “inflection point” means a boundary between the linear conversion operation and the logarithmic conversion operation, and is a term that is a subordinate concept of the “switching point” of the output signal of the image sensor having a plurality of types of conversion characteristics.

  Below, the structural example of the image pick-up element 5 of this embodiment is shown.

As shown in FIG. 2, the image sensor 5 of the present embodiment has a plurality of pixels G _{11 to} G _mn (where n and m are integers of 1 or more) arranged in a matrix (matrix arrangement). Each pixel G ₁₁ ~G _mn is for outputting an electric signal incident light by photoelectric conversion.

Also, a Bayer array color (RGB) filter is arranged on the lens unit 3 side of the pixels G _{11 to} G _mn . In this embodiment, the arrangement of the color filters is a Bayer array, but the arrangement of the color filters is not limited to this.

  In addition, the image pickup device 5 of the present embodiment is provided with an RGB filter in each pixel, but may be provided with other color filters such as cyan, magenta, and yellow.

As shown in FIG. 2, each of the pixels G _{11 to} G _mn includes a photodiode D that photoelectrically converts light incident from the lens unit 3 into electric charge, and a capacitor C that accumulates electric charge read from the photodiode D. , Transistors Q ₁ and Q ₂ for controlling the reset operation of the photodiode D and the capacitor C and reading of the charge, the transistor Q ₃ for converting the charge read from the capacitor C into a voltage, and the pixels G _{11 to} G _mn a transistor Q ₄ for controlling the reading of the voltage signal from, respectively.

As shown in FIG. 2, the transistor _{Q 1} is together with the signal input line _L R1 _{~L Rn} are connected, the power supply voltage _{V DD} is connected. Further, the transistor _{Q 2} is the signal input line _L T1 _{~L Tn} is connected, together with the power supply voltage _{V DD} to the transistor _{Q 3} is connected, the transistor _{Q 4} is connected. Further, the transistor _{Q 4} are together with the signal reading line _L SY1 _{~L SYn} are connected, the transistor _{Q 5} is connected. Additionally, the signal reading line _L Sx1 _{~L Sxn} is connected to the transistor _{Q 5.} The transistors Q _{1 to} Q ₅ are P-channel MOS transistors.

The signal input lines L _{R1 to} L _Rn and the signal input lines L _{T1 to} L _Tn correspond to the transistors Q ₁ and Q _{2 included in the} pixels G _{11 to} G _mn , the transfer signal φ _TRF and the reset signal φ _RST (see FIG. 3). Are entered respectively. A timing generator 8 is connected to the signal input lines L _{R1 to} L _Rn and the signal input lines L _{T1 to} L _Tn , and the transfer signal φ _TRF and the reset signal φ _{RST are supplied} to the transistors Q ₁ and Q ₂ at a predetermined timing. Is input to control the reset operation and the start and end of the exposure operation of each of the pixels G _{11 to} G _mn .

The signal readout lines L _{SY1 to} L _SYn are adapted to input a signal φ _SY (see FIG. 3) to the transistors Q ₄ provided in the pixels G _{11 to} G _mn . It is connected a vertical shift register 19, switching the signal reading line _L SY1 _{~L SYn} eligible for inputting a signal based on a signal from the timing generation unit 8 in the Y direction to the signal reading line _L SY1 _{~L SYn} Thus, a signal can be sequentially read from each of the pixels G _{m1 to} G _mn arranged in the Y direction.

Signal reading line _L Sx1 _{~L Sxn} is adapted to input a signal to the transistor _{Q 5} phi _Sx (see Figure 3). Horizontal shift register 20 is connected, by switching the signal reading line _L Sx1 _{~L Sxn} eligible for inputting a signal based on a signal from the timing generation unit 8 in the X direction in the signal reading line _L Sx1 _{~L Sxn} As a result, signals can be sequentially read from the pixels G _{1n to} G _mn arranged in the X direction.

As described above, the electric signals generated by the pixel G _mn at an arbitrary position among the plurality of pixels G _{11 to} G _mn arranged in a matrix by the input of the signals φ _Sx and φ _SY by the vertical shift register 19 and the horizontal shift register 20. Signals can be sequentially derived, and a signal change between the electrical signal at the output end of the image sensor 5 derived from each of the pixels G _{11 to} G _mn and the ground is output to the amplifier 6. Yes.

By adopting such a circuit configuration, each of the pixels G _{11 to} G _mn performs an operation as shown in FIG.

For convenience of description in FIG. 3, the four pixels shown in FIG. 2 are designated as pixels G ₁₁ , G ₁₂ , G ₂₁ , and G ₂₂ , and the timing generator 8 uses the signal input line L _{R1 as} a transistor of the pixels G ₁₂ and G ₂₂ . Q ₁ to enter the reset signal phi _RST2 to the transistor to _{Q 1} signal input line _{L R2} than the pixel G _11, G ₂₁ inputs the reset signal phi _RST1, also pixel G ₁₂ from the signal input line _{L T1,} G ₂₂ shall be of the transistor _{Q 2} inputs a transfer signal phi _TRF2 transistor _{Q 2} of the signal input line _{L T2} from pixel G _11, G ₂₁ inputs the transfer signal phi _TRF1. The signal from the signal reading line _{L SY2} transistor _{Q 4} pixels G _11, G ₂₁ together with the vertical shift register 19 to input a signal phi _SY1 transistor _{Q 4} of the signal pixels G ₁₂ from the read line _{L SY1,} G ₂₂ phi shall enter _SY2, pixels G ₁₁ from the signal reading line _{L Sx2} inputs the signal phi _Sx1 to the transistor _{Q 5} of the horizontal shift register 20 is connected from the signal reading line _{L Sx1} to the pixel G _21, G _22, G It shall enter signal phi _Sx2 the transistor Q ₅ which is connected to _12.

At the start of shooting, as shown in FIG. 3, the transfer signal phi _TRF1, phi _TRF2 ON the transistor _{Q 2} to which each pixel is provided with a H, the transistor Q included in each pixel reset signal phi _RST1, the phi _RST2 as H _{When 1} is turned on, the voltages of the photodiode D and the capacitor C are reset to the same voltage as the power supply voltage V _DD . Then, a transfer signal phi _TRF1 M, a transfer signal phi _TRF2 with the transistor _{Q 2} is turned OFF as L, a reset signal phi _RST1, when the OFF transistor _{Q 1} and phi _RST2 as L, so that the reset operation is completed It has become.

The timing generator 8, a transfer signal phi _TRF1 to be input to each pixel, and transfers the charges so that the transistor _{Q 2} and ON the phi _TRF2 as H, the transfer signal phi _TRF1, transistors _{Q 2} and phi _TRF2 as L Is set to OFF to terminate the transfer of charges.

Further, when the charge transfer is completed, the signal phi _SY1 in a state that was a transistor _{Q 4} pixels G _12, G ₂₂ is turned ON as H, the signal phi _Sx1 pixel G ₂₁ as H, it is connected to the G ₂₂ transistors Q By turning ON ₅ , the voltage signal converted from the electric charge by the transistor Q ₃ is read from the pixel G ₂₂ . Further, in a state where the signal phi _SY1 transistor _{Q 4} pixels G _12, G ₂₂ is turned ON as H, by the transistors _{Q 5} connected to the pixel G _11, G ₁₂ the signal phi _Sx2 as H in ON , signals are read from the pixels G _12. Further, in a state where the signal phi _SY2 the transistor _{Q 4} of the pixel G _11, G ₂₁ is turned ON as H, by the transistors _{Q 5} connected to the pixel G _21, G ₂₂ the signal phi _Sx1 as H in ON , signals are read from the pixels G _21. Further, in a state where the signal phi _SY2 the transistor _{Q 4} of the pixel G _11, G ₂₁ is turned ON as H, by the transistors _{Q 5} connected to the pixel G _11, G ₁₂ the signal phi _Sx2 as H in ON , signals are read out from the pixel G _11. In this manner, signals are sequentially read from the pixels G ₁₁ , G ₁₂ , G ₂₁ , and G ₂₂ .

Here, in each of the pixels G _{11 to} G _mn of this embodiment, a capacitor (not shown) is provided in parallel with the photodiode D, and charges generated according to the amount of incident light are accumulated in the capacitor (not shown) in the linear conversion operation. When the capacitor (not shown) is saturated, the linear conversion operation is switched to the logarithmic conversion operation.

  As a result, in the linear conversion operation, a signal corresponding to the integral value of the incident light amount is output, and in the logarithmic conversion operation, a signal corresponding to the incident light amount at the time of reading the signal is output. As described above, the output signal of the image sensor 5 has a linear region and a logarithmic region that continuously change in accordance with the amount of incident light.

Further, as shown in FIG. 3, the transfer signal phi _TRF1 to be input to the transistor _{Q 2,} phi _TRF2 has a voltage signal, respectively binary. More specifically, the transfer signal phi _TRF1 is a voltage value VM for operating the transistor Q ₂ when the amount of incident light exceeds a predetermined threshold value in the sub-threshold region, and the voltage value VH to the transistor Q ₂ to a conductive state The two values are taken. The transfer signal phi _TRF2 is a voltage value VL for operating the transistor Q ₂ in the sub-threshold region when the amount of incident light exceeds a predetermined threshold value, the two of the voltage value VH to the transistor Q ₂ to a conductive state It is supposed to take a value.

The transfer signal phi _TRF1, Lowering the voltage at the time of off of phi _TRF2, the potential difference between the gate and source of the transistor _{Q 2} to which included in each pixel _G m1 ~G _mn increases, the operating transistor _{Q 2} is a cut-off state The ratio of subject brightness to be increased.

Therefore, when the luminance range of the incident light amount is narrow, the voltage value at the time of off is lowered to widen the luminance range for linear conversion operation, and when the luminance range of the incident light amount is wide, the voltage value at the off time is increased. It is possible to widen the luminance range in which the logarithmic conversion operation is performed so as to obtain photoelectric conversion characteristics that match the characteristics of the subject. Further, the linear conversion operation can always be performed with the voltage value minimized, or the logarithmic conversion operation can always be performed with the voltage value maximized. Thereby, it is possible to arbitrarily set an inflection point which is a boundary between the linear conversion operation and the logarithmic conversion operation by controlling the voltage values when the transfer signals _φTRF1 and _φTRF2 are turned off.

In this embodiment, the inflection point setting can be changed for each horizontal pixel row by setting the voltage when the transfer signal _φTRF1 is off to the M level and the voltage when _φTRF2 is off to the L level. It has become.

  As described above, the imaging device 5 of the present embodiment has a configuration in which the setting of the inflection point can be changed for each horizontal pixel row, but the setting of the inflection point is changed for each vertical pixel column. It can also be set as the structure which can be.

For example, as shown in FIG. 5, inflection points are set for each vertical pixel column by connecting the signal input lines L _{TA1 to} L _TAn and the signal input lines L _{TB1 to} L _TBn for each vertical pixel column. Can be changed.

In this case, as shown in FIG. 6, the off-time voltage may be set to M level in the signal input lines L _{TA1 to} L _TAn and the off-time voltage may be set to L level in the signal input lines L _{TB1 to} L _TBn .

  In addition, although the imaging device 5 of the present embodiment is configured as a linear log conversion type sensor, the imaging device of the present invention is an imaging device in which at least one of a plurality of types of conversion characteristics is a logarithmic conversion characteristic or a linear conversion characteristic. As shown by the solid line in the graph of FIG. 7, it is possible to employ a configuration including an image sensor having a plurality of (three types in FIG. 7) linear conversion characteristics having different conversion coefficients.

  Returning to FIG. 1, an amplifier 6 and an AD converter (ADC) 7 are connected to the image sensor 5 in this order, and a timing generation unit 8 is connected to the image sensor 5 and the AD converter 7. An inflection point control unit 9 is connected to the timing generation unit 8, and the inflection point control unit 9 is connected to the image sensor 5.

  Further, a black reference correction unit 10 and a linearization signal correction unit 11 are connected to the AD converter 7 in this order. The linearization signal correction unit 11 includes an AE / AWB evaluation value detection unit 12 and an image processing unit 13, respectively. It is connected.

  Of these, the amplifier 6 amplifies the electrical signal output from the image sensor 5 to a predetermined specified level to compensate for a lack of level in the captured image.

  The AD converter 7 converts the electrical signal amplified by the amplifier 6 from an analog signal to a digital signal.

In addition, the timing generation unit 8 generates a predetermined timing pulse and outputs it to the image sensor 5 to control the photographing operation of the image sensor 5 (charge accumulation based on exposure, reading of accumulated charge, etc.). Yes. That is, based on the imaging control signal from the system controller ₂ , the transfer signal φ is transmitted from the signal input lines L _{R1 to} L _Rn and the signal input lines L _{T1 to} L _Tn to the transistors Q ₁ and Q ₂ of the pixels G _{11 to} G _{mn. TRF} and reset signal φ _RST are input, and a horizontal synchronizing signal, a vertical synchronizing signal, a horizontal scanning circuit driving signal, a vertical scanning circuit driving signal, and the like are input to the vertical shift register 19 and the horizontal shift register 20.

  The timing generation unit 8 also generates an A / D conversion clock used in the A / D converter 7.

The inflection point control section 9 functions as a switching point control section for setting a plurality of conversion characteristic switching points. The imaging apparatus 1 according to the present embodiment sets a plurality of inflection points, and converts a conversion error in the inflection point vicinity region of the output signal of the target pixel to other neighboring pixels corresponding to the inflection point vicinity region of the output signal. Therefore, the pixels G _{11 to} G _mn of the image sensor 5 are divided into regular groups, and the inflection points are set so that the inflection point neighboring areas do not overlap each group. Is set.

  As the “plural groups”, for example, a predetermined pixel row or a predetermined pixel column may be considered. That is, two or four types of inflection points can be set for each vertical pixel column, or two or four types of inflection points can be set for each horizontal pixel row.

  In addition, when continuously capturing images, the inflection points are set so that the inflection point neighboring areas do not overlap each other for a plurality of consecutive frames. In other words, when image data is output continuously like a moving image and a subject that does not move very fast is taken, the inflection point is different so that it differs between the two frames before and after or two or more frames. By setting, correction is performed.

In this case, the inflection points between the pixels need not be set differently, and the pixel configuration can be simplified. In addition, when continuously capturing images, the plurality of pixels G _{11 to} G _mn are divided into regular groups, and inflection points are set so that the inflection point neighboring areas do not overlap for each group, It is also possible to set the inflection point by inverting the inflection point for each frame.

  In the imaging device 5 of the present embodiment, the arrangement of the color filters is a Bayer array as described above, and it is preferable to set at least two different inflection points for each RGB. Further, it is preferable to set at least two different inflection points with respect to G and to set different inflection points between RBs.

  For example, when the image sensor 5 having the configuration shown in FIG. 2 is used, the inflection point can be changed for each horizontal pixel row. In this case, an inflection point can be set without sacrificing the vertical resolution.

  For example, as shown in FIG. 8, it is possible to set the GB line as an inflection point a and the RG line as an inflection point b. As a result, the inflection point is set to 2 for G and 2 is set between colors for RB, but is not set to 2 for the same color. As a result, G can be corrected between the same colors, but RB is corrected from other colors (such as G).

  Further, as shown in FIG. 9, the inflection point a and the inflection point b can be set for every two pixel rows. As a result, two inflection points are set for each of RGB. Thereby, it can correct | amend between the same colors about each of RGB.

  In addition, when the image sensor 5 having the configuration shown in FIG. 5 is used, the inflection point can be changed for each vertical pixel column.

  For example, as shown in FIG. 10, it is possible to set an inflection point a for the GR row and an inflection point b for the BG row. As a result, the inflection point is set to 2 for G and 2 for RB, but not 2 for the same color. As a result, G can be corrected between the same colors, but RB is corrected from other colors (such as G).

  In addition, as shown in FIG. 11, an inflection point a and an inflection point b can be set for every two pixel columns. As a result, two inflection points are set for each of RGB. Thereby, it can correct | amend between the same colors about each of RGB.

  Furthermore, as shown in FIG. 12, inflection points a to d can be set for each pixel column. As a result, G is set to 4 and RB is set to 2. Thereby, it can correct | amend between the same colors about each of RGB.

  8 to 12 show preferable examples as the inflection point setting patterns, and other setting patterns can be used. For example, it is possible to set 2 inflection points only for G, 2 settings for G, 2 settings vertically or horizontally for R, and 2 settings horizontally or vertically for B. . It is also possible to invert the value of the inflection point a and the value of the inflection point b for each frame.

  Returning to FIG. 1, the black reference correction unit 10 corrects the black level that is the lowest luminance value to the reference value. That is, since the black level of the image sensor 5 fluctuates, the black reference correction is performed by subtracting the difference between the signal level of each RGB signal output from the AD converter 7 and the reference signal level that becomes the black level. It is like that.

  Further, the linearization signal correction unit 11 serves as a signal correction unit that corrects a conversion error in an inflection point vicinity region of an output signal of a pixel of interest using a corresponding region of an output signal of a neighboring pixel having a different switching point. It fulfills its function. That is, the linearization signal correction unit 11 performs correction so that the output signal value of the target pixel in which the conversion error has occurred is a value corresponding to the interpolated value obtained from the output signals of neighboring pixels having different switching points. Yes.

  Here, “the value corresponding to the interpolation value” refers to not only replacing the output signal value of the target pixel with the interpolation value but also depending on the magnitude of the difference between the output signal value of the target pixel and the interpolation value. It means that the output signal value of the pixel is multiplied by a coefficient.

  As shown in FIG. 13, the linearization signal correction unit 11 includes a linear log conversion unit 21, an interpolation value calculation unit 22, a comparison unit 23, and a switching unit 24, and performs logarithmic conversion of output signals from pixels. The converted part is linearized and the conversion error is corrected.

  The linear log converter 21 functions as a characteristic converter that converts the output signal of the image sensor 5 into a state that is uniformly converted by one conversion characteristic. Logarithmic conversion of the output signal of the image sensor 5 is performed. It is designed to linearize the part. For example, FIG. 14 is an example of linearizing the output signal of the image sensor 5 in which two types of inflection points are set.

  The interpolation value calculation unit 22 calculates an interpolation value for correcting a conversion error in the linear log conversion unit 21.

  Here, the “conversion error” is a linearization process using an output signal model obtained by formulating output signals in the linear region and logarithmic region as shown by dotted lines in FIG. 4, FIG. 7 or FIG. This is the conversion error that occurs in some cases.

  That is, the actual output signal changes smoothly from the linear region to the logarithmic region as shown by the solid line in FIG. Therefore, when linearization is performed using the signal model indicated by the dotted line in FIG. 4, FIG. 7, or FIG. 14, a deviation from the output signal model occurs in the vicinity of the inflection point, resulting in a conversion error.

  Therefore, the conversion error in the region near the inflection point is corrected using the output signals of the peripheral pixels. That is, if at least two different inflection points are set for each RGB as described above, the inflection point is different between the pixel of interest and the surrounding pixels, so the output signal of the pixel of interest is the inflection point neighborhood region. Sometimes, the output signals of the surrounding pixels are in the complete linear region or the complete logarithmic region. Therefore, the conversion error in the vicinity of the inflection point is corrected by using the output signal of the peripheral pixels having no conversion error.

  For example, when two types of inflection points are set, as shown in FIG. 15, the conversion error in the region near the inflection point of the output signal (A) uses the complete logarithmic region of the output signal (B). It can be corrected. Further, the conversion error in the region near the inflection point of the output signal (B) can be corrected using the complete linear region of the output signal (A).

  As shown in FIG. 12, when four types of inflection points are set, as shown in FIG. 16, the conversion error in the region near the inflection point of the output signal (C) is the output signal (E) or the output. The complete logarithm region of the signal (F) is used, and the conversion error in the region near the inflection point of the output signal (D) uses the complete logarithm region of the output signal (F) and is near the inflection point of the output signal (E). The conversion error in the region uses the complete logarithmic region of the output signal (C), and the conversion error in the region near the inflection point of the output signal (F) uses the complete logarithmic region of the output signal (C) or the output signal (D). Can be corrected.

  Specifically, based on the output signals of surrounding pixels, an interpolation value corresponding to the output signal value when there is no conversion error in the target pixel is obtained, and the difference between the actual output signal value of the target pixel and the interpolation value is an error tolerance. If the value is greater than or equal to the value, the conversion error is corrected by replacing the output signal value of the target pixel with the interpolation value.

For example, FIG. 17 is a conceptual diagram when correction is performed between the same colors. As shown in FIG. 17, since there is a correlation between the output signal of the target pixel and the output signal of the peripheral pixel, an interpolation value can be obtained based on the output signal of the peripheral pixel. Correction can be performed by replacing the value with an interpolation value.

  FIG. 18 is a conceptual diagram when correction is performed from another color. In this case as well, correction can be performed by obtaining an interpolation value based on the output signal of the peripheral pixel and replacing it with the output signal value of the target pixel.

  Next, the content of “interpolation value” will be described.

  The “interpolation value” calculated by the interpolation value calculation unit 22 is a value obtained by multiplying the “average value” of the output signals of a plurality of pixels located around the target pixel by “predetermined value”.

19, the vertical pixel column different inflection point for each line and setting, if the pixel of interest and the target pixel G _alpha, an average value G _Avarufa near pixel _G Sα1 ~G Sα4. In the case where the target pixel and the target pixel G _beta, the average value _{G Avbeta} near pixel _{G Sβ1} ~G _Sβ4.

  The “predetermined value” multiplied by the “average value” is a value corresponding to the level difference between the output signal of the target pixel and the output signals of the peripheral pixels. This “predetermined value” can be obtained by taking the difference between the average value of the output signals of the target pixel and the average value of the output signals of the peripheral pixels.

  Next, the formula for calculating the “interpolation value” will be specifically described with reference to an example in which two types of inflection points (i) and (ii) are set as shown in FIG.

In the example of FIG. 20, when correcting G2 _{2m, 2n} (when G2 is in the inflection point vicinity region), the following equation (1) is obtained. However, in the following formulas (1) to (8), “α” means a coefficient corresponding to “predetermined value”, and “Sg”, “Sb”, “Sg1”, and “Sg3” mean error tolerance values, respectively. To do.

In the example of FIG. 20, when G1 _2m-1 and _2n-1 are corrected (when G1 is in the vicinity of the inflection point), the following equation (2) is obtained.

Next, the case where B2 _{2m-1 and 2n} are corrected will be described.

  First, when the G1 pixel is not in the inflection point vicinity region, the following equation (3) is obtained.

  When the G2 pixel is not in the inflection point vicinity region, the following equation (4) is obtained.

Next, a case where R1 _{2m, 2n-1} is corrected will be described.

  First, when the G2 pixel is not in the vicinity of the inflection point, the following equation (5) is obtained.

  When the G1 pixel is not in the inflection point vicinity region, the following equation (6) is obtained.

  Further, the formula for calculating the “interpolation value” will be specifically described with reference to an example in which four types of inflection points (i) to (iv) are set as shown in FIG.

In the example of FIG. 21, when G1 _{2m-1, 4n-3} is corrected, the following equation (7) is obtained.

In the example of FIG. 21, when G3 _{2m−1,4n−1} is corrected, the following equation (8) is obtained.

  In the example of FIG. 21, G2 and G4, B2 and B4, and R1 and R3 can also be corrected by the same calculation formula as the above formula (7) or (8).

  The comparison unit 23 compares the level difference between the target pixel and the interpolation value, and outputs the comparison result to the switching unit 24.

  Based on the comparison result in the comparison unit 23, the switching unit 24 outputs the output signal of the linear log conversion unit 21 as it is when the level difference between the target pixel and the interpolation value is equal to or less than the error allowable value. When the level difference is larger than the allowable error value, the interpolation value calculated by the interpolation value calculation unit 22 is output. That is, the linearization signal correction unit 11 performs correction only when the difference between the output signal value of the target pixel and the interpolation value is outside the allowable error range.

  Returning to FIG. 1, the AE / AWB evaluation value detection unit 12 performs respective evaluations for performing automatic exposure control (AE) and automatic white balance (AWB) from the electric signal corrected by the black reference in the black reference correction unit 10. The value is to be detected.

  The image processing unit 13 includes an AWB control unit 14, a color interpolation unit 15, a color correction unit 16, a gradation conversion unit 17, and a color space conversion unit 18, and these components are connected in this order. Yes.

  Among these, the AWB control unit 14 calculates the correction coefficient from the output signal of the linearization signal correction unit 11, thereby calculating the level ratio (R / G, B / G) of each color component of R, G, B of the captured image. It is adjusted so that the white color is displayed correctly.

  In addition, the color interpolation unit 15 can obtain R, G, and B color component values for each pixel when the signals obtained in the pixels of the image sensor 5 are only one or two of the primary colors. In addition, color interpolation processing for interpolating the missing color components for each pixel is performed.

  The color correcting unit 16 corrects the color component value for each pixel of the image data input from the color interpolating unit 15 to generate an image in which the hue of each pixel is emphasized.

  Further, the gradation conversion unit 17 sets the response characteristics of the gradation of the image to the imaging element in order to realize the ideal gradation reproduction characteristic with gamma being 1 from the input of the image to the final output in order to faithfully reproduce the image. A gamma correction process for correcting the curve to an optimum curve corresponding to the gamma value of 5 is performed.

  Further, the color space conversion unit 18 converts the color space from RGB to YUV. YUV is a color space management method that expresses colors with two chromaticities: luminance (Y) signal, blue color difference (U, Cb), and red color difference (V, Cr), and converts the color space to YUV. This facilitates data compression of the color difference signal.

  Next, the imaging method of the present invention using the imaging device 1 of the present embodiment will be described.

  First, the inflection point control unit 9 divides the pixels of the image sensor 5 into a plurality of groups, and sets different inflection points for each group so that the inflection point neighboring areas of the output signals of each group do not overlap. To do.

  In the imaging device 5 of the present embodiment, since the arrangement of the color filters is a Bayer array as described above, it is preferable to set at least two different inflection points for each RGB.

  For example, when the imaging device 5 having the configuration shown in FIG. 2 is used, the inflection point can be changed for each horizontal pixel row as shown in FIG. 8 or FIG. In this case, an inflection point can be set without sacrificing the vertical resolution. In addition, when the imaging device 5 having the configuration shown in FIG. 5 is used, the inflection point can be changed for each vertical pixel row as shown in FIG. 10 or FIG.

Next, when an electrical signal is output from the pixels G _{11 to} G _{mn of} the image sensor 5 to the signal processing unit 6, the amplifier 6 amplifies the electrical signal output from the image sensor 5 to a predetermined specified level, and AD The converter 7 converts the amplified electrical signal into a digital signal.

  Subsequently, the black reference correction unit 10 corrects the black level that is the lowest luminance value to the reference value and outputs it to the linearization signal correction unit 11.

  Next, the linearization signal correction unit 11 corrects the conversion error in the inflection point vicinity region of the output signal of the target pixel using the corresponding region of the output signal of the vicinity pixel having a different switching point.

  That is, first, the linear log conversion unit 21 of the linearization signal correction unit 11 linearizes the logarithmically converted portion of the electrical signal output from the image sensor 5. For example, FIG. 14 is an example of linearizing the output signal of the image sensor 5 in which two types of inflection points are set.

  Subsequently, the interpolation value calculation unit 22 calculates an interpolation value for correcting the conversion error in the linear log conversion unit 21.

  For example, when two types of inflection points are set, as shown in FIG. 15, the conversion error in the region near the inflection point of the output signal (A) uses the complete logarithmic region of the output signal (B). to correct. Further, the conversion error in the region near the inflection point of the output signal (B) is corrected using the complete linear region of the output signal (A).

  As shown in FIG. 12, when four types of inflection points are set, as shown in FIG. 16, the conversion error in the region near the inflection point of the output signal (C) is the output signal (E) or the output. The complete logarithm region of the signal (F) is used, and the conversion error in the region near the inflection point of the output signal (D) uses the complete logarithm region of the output signal (F) and is near the inflection point of the output signal (E). The conversion error in the region uses the complete logarithmic region of the output signal (C), and the conversion error in the region near the inflection point of the output signal (F) uses the complete logarithmic region of the output signal (C) or the output signal (D). To correct.

  Specifically, an interpolation value corresponding to the output signal value when there is no conversion error in the target pixel is obtained based on the output signals of the surrounding pixels.

  That is, the interpolation value calculation unit 22 calculates the “interpolation value” by multiplying the “predetermined value” by the “average value” of the output signals of a plurality of pixels located around the target pixel.

For example, in FIG. 19, the vertical pixel column different inflection point for each line and setting, if the pixel of interest and the target pixel G _alpha, an average value G _Avarufa near pixel _G Sα1 ~G Sα4. In the case where the target pixel and the target pixel G _beta, the average value _{G Avbeta} near pixel _{G Sβ1} ~G _Sβ4.

  The “predetermined value” multiplied by the “average value” is a value corresponding to the level difference between the output signal of the target pixel and the output signals of the peripheral pixels. This “predetermined value” can be obtained by taking the difference between the average value of the output signals of the target pixel and the average value of the output signals of the peripheral pixels.

That is, when the specifically described method for calculating the "interpolated value", if you set the two inflection points (i) and (ii) as shown in FIG. 20, G2 _2m, correction of _2n is the formula ( 1), the correction of G1 _2m-1,2n-1 is performed when the correction of B2 _2m-1,2n is performed according to the above equation (2), and when the G1 pixel is not in the vicinity of the inflection point, According to equation (3), if the G2 pixel is not in the vicinity of the inflection point in the same case, correction can be performed by equation (4). In the example of FIG. 20, when R1 _{2m, 2n-1} is corrected and the G2 pixel is not in the inflection point vicinity region, the G1 pixel is in the inflection point vicinity region according to the above equation (5). If not, it can be corrected by the above equation (6).

Further, when four types of inflection points (i) to (iv) are set as shown in FIG. 21, G1 _{2m-1, 4n-3} is corrected by G3 _{2m-1, 4n} according to the above equation (7). The correction of ₋₁ can be performed by the above equation (8). Furthermore, in the example of FIG. 21, correction of G2 and G4, B2 and B4, and R1 and R3 can also be performed by the same calculation formula as the above formula (7) or (8).

  Next, when the interpolation value calculation unit 22 calculates the interpolation value, the comparison unit 23 compares the level difference between the target pixel and the interpolation value, and outputs the comparison result to the switching unit 24.

  Next, based on the comparison result in the comparison unit 23, the switching unit 24 outputs the output signal of the linear log conversion unit 21 as it is when the level difference between the target pixel and the interpolated value is equal to or less than the error allowable value. When the level difference from the interpolation value is larger than the allowable error value, the interpolation value calculated by the interpolation value calculation unit 22 is output.

  Next, the AE / AWB evaluation value detection unit 12 detects an evaluation value necessary for AE (automatic exposure) from the output signal of the linearization signal correction unit 11 and sends it to the system control unit 2.

  When the AWB control unit 14 of the image processing unit 13 calculates the correction coefficient from the output signal of the linearization signal correction unit 11 and performs the AWB process, the color interpolation unit 15 performs the color interpolation process, and the color correction unit 16 Perform color correction processing. Subsequently, the gradation conversion unit 17 performs gamma correction processing, and the color space conversion unit 18 converts the color space from RGB to YUV.

  As described above, according to the present embodiment, by setting a plurality of inflection points, it is possible to obtain a plurality of output signals that are correlated with each other and that do not overlap in the vicinity of the switching point. As a result, the conversion error generated in the inflection point vicinity region by the linear log conversion is corrected by using a region other than the inflection point vicinity region in the output signal from other neighboring pixels corresponding to the inflection point vicinity region. It becomes possible.

  In other words, when characteristic conversion is performed using an output signal model obtained by formulating the output signal of each characteristic, the actual output signal changes gently, and thus there is a discrepancy with the output signal model in the region near the inflection point. . Therefore, the conversion error can be corrected by obtaining an output signal when there is no conversion error by using a region other than the inflection point vicinity region in the output signal from other neighboring pixels.

Also, by dividing the plurality of pixels G _{11 to} G _mn into regular groups and setting inflection points for each group, the inflection points of the linear log sensor are set while suppressing the influence on the photographed image. It becomes possible to do.

  In addition, when shooting a moving image, by setting an inflection point for each of a plurality of frames, it is possible to set an inflection point of a plurality of conversion characteristics while suppressing an influence on a captured image.

In addition, when shooting a moving image, it is possible to easily set different inflection points for a plurality of frames by inverting the inflection points set for each group of pixels G _{11 to} G _mn for each frame. .

  Also, by setting two types of inflection points for each vertical pixel row, it is possible to correct conversion errors using output signals from other neighboring pixels. Thus, for example, when each pixel includes an RGB filter, correction can be performed between the same colors or between different colors.

  Also, by setting four types of inflection points for each vertical pixel column, it is possible to correct conversion errors using output signals from other neighboring pixels. Thus, for example, when each pixel is provided with an RGB filter, it is possible to perform correction between the same colors for each of RGB.

  Also, by setting two types of inflection points for each horizontal pixel row, it is possible to correct conversion errors using output signals from other neighboring pixels. Thus, for example, when each pixel includes an RGB filter, correction can be performed between the same colors or between different colors. In addition, correction can be performed without sacrificing the vertical resolution.

  In addition, by setting four types of inflection points for each horizontal pixel row, it is possible to correct conversion errors using output signals from other neighboring pixels. Thus, for example, when each pixel is provided with an RGB filter, it is possible to perform correction between the same colors for each of RGB. In addition, correction can be performed without sacrificing the vertical resolution.

  Since a plurality of output signals having different inflection points are correlated with each other, an interpolation value corresponding to the output signal value of the target pixel when there is no conversion error is obtained from the output signals from other neighboring pixels. The conversion error can be corrected by replacing the output signal value of the pixel with an interpolation value, or by applying a coefficient according to the ratio between the output signal value of the target pixel and the interpolation value.

  In addition, since the conversion error is corrected only when the difference between the output signal value of the target pixel and the interpolation value is outside the allowable error range, that is, when the conversion error is large enough to affect the captured image, It is possible to improve the efficiency of signal processing by omitting unnecessary processing.

  The same applies to not only the linear log conversion type sensor but also an imaging device having a logarithmic conversion characteristic, an imaging element having a linear conversion characteristic, or an imaging element having a plurality of linear conversion characteristics having different conversion coefficients. Can be obtained.

  As described above, according to the imaging apparatus and imaging method of the present invention, in the imaging apparatus and imaging method using an imaging element having a plurality of types of photoelectric conversion characteristics, the state of the output signal of the imaging element is unified. Conversion errors can be minimized.

It is a block diagram which shows the functional structure of the imaging device which concerns on this embodiment. It is a circuit diagram showing an example of composition of an image sensor concerning this embodiment. It is a time chart which shows operation of an image sensor concerning this embodiment. It is a graph which shows the output signal of the image sensor concerning this embodiment. It is a circuit diagram which shows the other structural example of the image pick-up element which concerns on this embodiment. It is a time chart which shows operation | movement of the image pick-up element shown as another structural example which concerns on this embodiment. It is a graph which shows the example of the output signal of the other image sensor concerning this embodiment. It is a top view which shows the setting pattern of the inflection point of each pixel in the image sensor which concerns on this embodiment. It is a top view which shows the other setting pattern of the inflection point of each pixel. It is a top view which shows the other setting pattern of the inflection point of each pixel. It is a top view which shows the other setting pattern of the inflection point of each pixel. It is a top view which shows the other setting pattern of the inflection point of each pixel. It is a block diagram which shows the structure of the linearization signal correction | amendment part which concerns on this embodiment. It is a graph which shows a mode that the logarithm area | region of the output signal of the image pick-up element which concerns on this embodiment is linearized. It is a graph which shows the inflection point vicinity area | region of the output signal of the image pick-up element based on this embodiment, and its correction | amendment area | region. It is a graph which shows the other example of the inflection point vicinity area of the output signal of the image sensor concerning this embodiment, and its amendment field. It is a conceptual diagram in case the linearization signal correction | amendment part which concerns on this embodiment correct | amends between the same colors. It is a conceptual diagram in case the linearization signal correction | amendment part which concerns on this embodiment correct | amends from another color. It is a conceptual diagram which shows the correction content of the linearization signal correction | amendment part which concerns on this embodiment. It is a top view which shows the example which set two types of inflection points in the image sensor which concerns on this embodiment. It is a top view which shows the example which set four types of inflection points in the image sensor which concerns on this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 2 System control part 3 Lens unit 4 Aperture control part 5 Imaging element 6 Amplifier 7 AD converter 8 Timing generation part 9 Inflection point control part 10 Black reference correction part 11 Linearization signal correction part 12 Evaluation value detection part 13 Image Processing unit 14 AWB control unit 15 Color interpolation unit 16 Color correction unit 17 Gradation conversion unit 18 Color space conversion unit 19 Vertical shift register 20 Horizontal shift register 21 Linear log conversion unit 22 Interpolation value calculation unit 23 Comparison unit 24 Switching unit

Claims (28)

An imaging device having a plurality of pixels that convert incident light into an electrical signal with a plurality of types of conversion characteristics;
A switching point control unit for setting switching points of a plurality of conversion characteristics;
A characteristic converter that converts the output signal of the image sensor into a state uniformly converted by one conversion characteristic;
A signal correction unit that corrects a conversion error in the vicinity of the switching point of the output signal of the pixel of interest using a corresponding region of the output signal of a neighboring pixel having a different switching point;
An imaging apparatus comprising:   2. The switching point control unit divides the plurality of pixels into regular groups, and sets the switching points so that regions near the switching points of the conversion characteristics do not overlap each group. Imaging device.   The imaging apparatus according to claim 1, wherein the switching point control unit sets the switching point so that regions in the vicinity of the switching point of the conversion characteristic do not overlap for each of a plurality of consecutive frames.   The switching point control unit divides the plurality of pixels into regular groups, sets the switching points so that areas near the switching points of the conversion characteristics do not overlap for each group, and sets the switching points for each frame. The imaging apparatus according to claim 3, wherein the switching point is set by being inverted.   5. The imaging apparatus according to claim 2, wherein the switching point control unit sets two types of switching points for each vertical pixel column of the imaging element.   5. The imaging apparatus according to claim 2, wherein the switching point control unit sets four types of the switching points for each vertical pixel column of the imaging element.   5. The imaging apparatus according to claim 2, wherein the switching point control unit sets two types of switching points for each horizontal pixel row of the imaging element. 6.   The imaging device according to claim 2 or 4, wherein the switching point control unit sets four types of the switching points for each horizontal pixel row of the imaging device.   The signal correction unit corrects an output signal value of a target pixel in which the conversion error has occurred to a value corresponding to an interpolation value obtained from an output signal of the other neighboring pixels. Item 9. The imaging device according to any one of Items 8 to 9.   The imaging apparatus according to claim 9, wherein the signal correction unit performs correction when a difference between an output signal value of the target pixel and the interpolation value is outside an allowable error range.   The imaging apparatus according to claim 1, wherein at least one of the plurality of types of conversion characteristics is a logarithmic conversion characteristic.   The imaging apparatus according to any one of claims 1 to 10, wherein at least one of the plurality of types of conversion characteristics is a linear conversion characteristic.   The imaging apparatus according to claim 1, wherein the plurality of types of conversion characteristics are logarithmic conversion characteristics and linear conversion characteristics.   The imaging apparatus according to claim 1, wherein the plurality of types of conversion characteristics are a plurality of linear conversion characteristics having different conversion coefficients. An imaging method using an imaging device having a plurality of pixels that convert incident light into an electrical signal with a plurality of types of conversion characteristics,
A switching point control step of setting switching points of a plurality of conversion characteristics;
A characteristic conversion step of converting the output signal of the image sensor into a state uniformly converted by one conversion characteristic;
A signal correction step of correcting a conversion error in the vicinity of the switching point of the output signal of the pixel of interest using a corresponding region of the output signal of a neighboring pixel having a different switching point;
An imaging method comprising:   16. The switching point control step, wherein the plurality of pixels are divided into regular groups, and the switching points are set so that the regions near the switching points of the conversion characteristics do not overlap each group. Imaging method.   16. The imaging method according to claim 15, wherein, in the switching point control step, the switching points are set so that regions near the switching points of the conversion characteristics do not overlap each other for a plurality of consecutive frames.   In the switching point control step, the plurality of pixels are divided into regular groups, and the switching points are set so that the regions near the switching points of the conversion characteristics do not overlap for each group, and the switching points are set for each frame. The imaging method according to claim 17, wherein the switching point is set by inversion.   19. The imaging method according to claim 16, wherein in the switching point control step, two types of switching points are set for each vertical pixel column of the imaging device.   19. The imaging method according to claim 16, wherein in the switching point control step, four types of the switching points are set for each vertical pixel column of the image sensor.   19. The imaging method according to claim 16, wherein in the switching point control step, two types of the switching points are set for each pixel row next to the imaging device.   19. The imaging method according to claim 16, wherein in the switching point control step, four types of the switching points are set for each horizontal pixel row of the image sensor.   16. The signal correction step performs correction so that an output signal value of a target pixel in which the conversion error has occurred is a value corresponding to an interpolation value obtained from an output signal of the other neighboring pixels. Item 22. The imaging method according to any one of Items 22 above. 24. The imaging method according to claim 23, wherein in the signal correction step, correction is performed when a difference between an output signal value of the target pixel and the interpolation value is outside an allowable error range.   25. The imaging method according to claim 15, wherein at least one of the plurality of types of conversion characteristics is a logarithmic conversion characteristic.   The imaging method according to any one of claims 15 to 24, wherein at least one of the plurality of types of conversion characteristics is a linear conversion characteristic.   The imaging method according to any one of claims 15 to 24, wherein the plurality of types of conversion characteristics are logarithmic conversion characteristics and linear conversion characteristics.   25. The imaging method according to claim 15, wherein the plurality of types of conversion characteristics are a plurality of linear conversion characteristics having different conversion coefficients.

Images