WO2023105916A1 - 固体撮像素子、撮像装置、および、固体撮像素子の制御方法 - Google Patents

固体撮像素子、撮像装置、および、固体撮像素子の制御方法 Download PDF

Info

Publication number
WO2023105916A1
WO2023105916A1 PCT/JP2022/038089 JP2022038089W WO2023105916A1 WO 2023105916 A1 WO2023105916 A1 WO 2023105916A1 JP 2022038089 W JP2022038089 W JP 2022038089W WO 2023105916 A1 WO2023105916 A1 WO 2023105916A1
Authority
WO
WIPO (PCT)
Prior art keywords
pixel
pixels
ratio mode
signal
exposure
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/038089
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
吉満 塩谷
拓也 三上
之康 立澤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Semiconductor Solutions Corp
Original Assignee
Sony Semiconductor Solutions Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Semiconductor Solutions Corp filed Critical Sony Semiconductor Solutions Corp
Priority to US18/715,194 priority Critical patent/US12556834B2/en
Priority to JP2023566111A priority patent/JPWO2023105916A1/ja
Priority to CN202280080050.8A priority patent/CN118339849A/zh
Priority to DE112022005856.0T priority patent/DE112022005856T5/de
Publication of WO2023105916A1 publication Critical patent/WO2023105916A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/50—Control of the SSIS exposure
    • H04N25/57—Control of the dynamic range
    • H04N25/58—Control of the dynamic range involving two or more exposures
    • H04N25/581—Control of the dynamic range involving two or more exposures acquired simultaneously
    • H04N25/583—Control of the dynamic range involving two or more exposures acquired simultaneously with different integration times
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
    • H04N25/46—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by combining or binning pixels
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H04N25/77—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components
    • H04N25/778—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components comprising amplifiers shared between a plurality of pixels, i.e. at least one part of the amplifier must be on the sensor array itself
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements

Definitions

  • This technology relates to solid-state imaging devices. More specifically, the present invention relates to a solid-state imaging device for synthesizing a plurality of images, an imaging device, and a control method for the solid-state imaging device.
  • HDR High Dynamic Range
  • HDR synthesis processing which combines multiple images with different exposure times, has been used in imaging devices and the like.
  • solid-state imaging devices have been proposed that replace the long-second pixels with short-second pixels with short exposure times.
  • the afterimage is suppressed by replacing the pixels of the moving subject with short-time pixels.
  • all the pixels of the subject are replaced with the short-time pixels, so the dynamic range of the subject area cannot be expanded. If the solid-state imaging device performs HDR synthesis processing without replacing pixels of a moving subject with short-time pixels, the dynamic range of the subject can be expanded, but an afterimage may occur.
  • the above-described conventional technique cannot expand the dynamic range while suppressing the afterimage, and it is difficult to improve the image quality of the image data.
  • This technology was created in view of this situation, and aims to improve image quality in solid-state imaging devices that perform synthesis processing that expands the dynamic range.
  • the present technology has been made to solve the above-described problems. based on a driving unit that sets the sensitivity of the third pixel to a value different from that of the fourth pixel by pixel addition when a predetermined pixel addition ratio mode is set, and a predetermined parameter related to the imaging scene.
  • a solid-state imaging device having a mode setting unit for setting either the pixel addition ratio mode or the exposure ratio mode, and a control method thereof. This brings about the effect of improving the image quality.
  • the first aspect further comprises a pixel array section including a predetermined number of pixel blocks in which a plurality of pixels sharing a floating diffusion layer are arranged, wherein the driving section sets the exposure ratio mode.
  • a pixel at a predetermined position within the pixel block may be driven as the first pixel, and a pixel different from the predetermined position within the pixel block may be driven as the second pixel.
  • each of the pixel blocks has nine pixels arranged therein, and when the pixel addition ratio mode is set, the driving section outputs a pixel signal of a central pixel in the pixel block. is generated as the pixel signal of the third pixel, and a signal obtained by adding pixel signals of eight pixels surrounding the central pixel may be generated as the pixel signal of the fourth pixel. This brings about the effect of eliminating the need for gain adjustment.
  • the pixel block may include a deca pixel block in which 10 pixels are arranged and an octa pixel block in which 8 pixels are arranged. This brings about the effect of facilitating the adjustment of the pixel addition number.
  • the driving section when the exposure ratio mode is set, adds the pixel signals of the first pixels and then adds the pixel signals of the second pixels, and
  • the addition ratio mode when the addition ratio mode is set, a signal obtained by adding the pixel signals of a part of the pixel block is generated as the pixel signal of the third pixel, and the pixel signal of the remaining area of the pixel block is generated.
  • the added signal may be generated as the pixel signal of the fourth pixel, and the number of additions of the partial area and the number of additions of the remaining area may be different. This brings about an effect that pixel signals are generated for each of a plurality of pixels with different sensitivities.
  • the first aspect further includes an image processing section that processes the pixel signals, a pair of adjacent pixels in the pixel block share a microlens, and the image processing section A process of obtaining a phase difference from the pixel signal of each pixel may be performed. This brings about the effect of detecting the in-focus position.
  • the deca pixel block and the octa pixel block are divided into the deca pixel block and the octa pixel block by a gain set based on a ratio of charge-to-voltage conversion efficiencies of the deca pixel block and the octa pixel block and a ratio of the number of pixels to be added.
  • An amplifier circuit for amplifying pixel signals in at least one of the blocks may be further provided. This brings about the effect of uniforming the sensitivity of each pixel.
  • the parameter may include at least one of the amount of motion of the subject, the shutter speed, and the frame rate. This brings about the effect of suppressing blurring.
  • the parameters may include ISO (International Organization for Standardization) sensitivity. This brings about the effect of expanding the dynamic range while ensuring the sensitivity.
  • ISO International Organization for Standardization
  • a predetermined exposure ratio mode when a predetermined exposure ratio mode is set, the first pixel and the second pixel are exposed for different exposure times, and a predetermined pixel addition ratio mode is set.
  • either the pixel addition ratio mode or the exposure ratio mode is selected based on a driving unit that sets the sensitivity of the third pixel to a value different from that of the fourth pixel by pixel addition, and a predetermined parameter related to the imaging scene.
  • the imaging apparatus includes a mode setting section for setting and a signal processing section for processing image data read out from the drive section. This brings about the effect of improving the image quality of the image data captured by the imaging device.
  • FIG. 6 is a timing chart showing an example of a driving method in an exposure ratio mode when phase difference AF (AutoFocus) is not performed according to the first embodiment of the present technology; 6 is a timing chart showing an example of a driving method in an exposure ratio mode when performing phase difference AF according to the first embodiment of the present technology; It is a figure showing an example of an addition object and a frame at the time of performing exposure twice in exposure ratio mode in a 1st embodiment of this art.
  • phase difference AF AutoFocus
  • FIG. 7A and 7B are timing charts illustrating an example of a driving method in a pixel addition ratio mode according to the first embodiment of the present technology; It is a figure which shows an example of addition object when phase difference AF is impossible by the pixel addition ratio mode in 1st Embodiment of this technique. It is a figure showing an example of an addition object when phase difference AF is possible in pixel addition ratio mode in a 1st embodiment of this art.
  • FIG. 1 is a block diagram showing a schematic configuration example of a vehicle control system
  • FIG. 4 is an explanatory diagram showing an example of an installation position of an imaging unit
  • First Embodiment Example of Setting Either Exposure Ratio Mode or Pixel Addition Ratio Mode
  • Second Embodiment Example of Setting Either Exposure Ratio Mode or Pixel Addition Ratio Mode Based on ISO Sensitivity
  • Third Embodiment Example of arranging pixel blocks consisting of 9 pixels and setting either exposure ratio mode or pixel addition ratio mode
  • FIG. 1 is a block diagram showing a configuration example of an imaging device 100 according to the first embodiment of the present technology.
  • This imaging device 100 is a device for capturing image data, and includes a solid-state imaging device 200 , a control section 110 and a recording section 120 .
  • a digital camera or an electronic device (smartphone, personal computer, etc.) having an imaging function is assumed.
  • the solid-state imaging device 200 generates image data (that is, frames) by photoelectric conversion, and includes a driving section 210, a pixel array section 220, a reading section 230, a reference signal generating section 240, a signal processing section 250, and a timing control section. 260.
  • the control unit 110 controls the solid-state imaging device 200 to capture frames.
  • the control unit 110 supplies imaging control signals including, for example, a vertical synchronization signal to the solid-state imaging device 200 .
  • the recording unit 120 records frames.
  • the vertical synchronizing signal is a signal indicating the timing of imaging, and a periodic signal with a constant frequency (60 Hz, etc.) is used as the vertical synchronizing signal.
  • a plurality of pixels 221 are arranged in a two-dimensional grid in the pixel array section 220 of the solid-state imaging device 200 .
  • the driving section 210 drives the pixel array section 220 to output pixel signals to the reading section 230 .
  • the reading unit 230 performs AD (Analog to Digital) conversion processing and CDS (Correlated Double Sampling) processing on each pixel signal.
  • the reading unit 230 supplies the frame in which the processed signals are arranged to the signal processing unit 250 .
  • the signal processing unit 250 performs various image processing such as HDR synthesis processing and demosaicing processing on frames.
  • the signal processing unit 250 supplies the processed frame to the recording unit 120 .
  • the signal processing unit 250 also sets the imaging mode of the solid-state imaging device 200 and generates a mode signal MODE indicating the set mode. Details of the imaging mode will be described later.
  • the mode signal MODE is supplied to the drive section 210 and the timing control section 260 .
  • the reference signal generating section 240 generates the reference signal RMP by DA (Digital to Analog) converting the signal from the timing control section 260 .
  • a sawtooth ramp signal for example, is used as the reference signal, and a DAC (Digital to Analog Converter), for example, is used as the reference signal generator 240 .
  • the reference signal generator 240 supplies the generated reference signal RMP to the reader 230 .
  • the timing control section 260 controls the operation timings of the drive section 210 , the reference signal generation section 240 and the reading section 230 in synchronization with the vertical synchronization signal from the control section 110 .
  • the imaging device 100 records frames
  • the frames may be transmitted to the outside of the imaging device 100.
  • an external interface is additionally provided for transmitting frames.
  • the imaging device 100 may display frames.
  • a display section is further provided.
  • FIG. 2 is an example of a plan view of the pixel array section 220 according to the first embodiment of the present technology.
  • Each of the pixels 221 receives and photoelectrically converts visible light of any one of R (Red), G (Green), and B (Blue).
  • Pixels 221 that receive R, G and B light respectively are hereinafter referred to as "R pixel", “G pixel” and “B pixel”.
  • R pixel indicates G pixels arranged in a row including R pixels
  • Gb indicates G pixels arranged in a row including B pixels.
  • the pixel array section 220 includes a predetermined number of pixel blocks in which a plurality of pixels are arranged. It is assumed that the pixels in the pixel block are not shaded. In addition to the pixel blocks, light-shielded light-shielded pixels can also be arranged in the pixel array section.
  • a deca pixel block 300 and an octa pixel block 350 are arranged.
  • Ten G pixels are arranged in the deca pixel block 300 and share a floating diffusion layer (not shown).
  • Octa-pixel block 350 is arranged with 8 R pixels or 8 B pixels, which share a floating diffusion layer.
  • areas surrounded by thick dotted lines indicate deca pixel blocks 300 or octa pixel blocks 350 .
  • deca pixel blocks 300 and octa pixel blocks 350 are alternately arranged in the horizontal and vertical directions, respectively.
  • G pixels are arranged at 10 positions, excluding both ends of the second row, out of 12 positions of 3 rows ⁇ 4 columns.
  • the shape of the deca pixel block 300 can also be expressed as a shape obtained by rotating the letter "H" by 90 degrees.
  • R pixels or B pixels are arranged at eight positions out of 12 positions of 3 rows ⁇ 4 columns, excluding both ends of the first row and the third row.
  • Octa pixel block 350 can also be described as cross-shaped.
  • each of deca-pixel block 300 and octa-pixel block 350 adjacent pairs of pixels in the same row, such as pixels 221 - 1 and 221 - 2 , share microlenses 222 .
  • a subsequent circuit can obtain a phase difference from the pixel signals of these pixels and perform phase difference AF (AutoFocus) for detecting an in-focus position from the phase difference.
  • phase difference AF AutoFocus
  • FIG. 3 is a circuit diagram showing a configuration example of the deca pixel block 300 according to the first embodiment of the present technology.
  • This deca pixel block 300 includes photoelectric conversion elements 311 to 320 , transfer transistors 321 to 330 , reset transistor 341 , floating diffusion layer 342 , amplification transistor 343 and selection transistor 344 .
  • Each of the photoelectric conversion elements 311 to 320 generates charges by photoelectric conversion.
  • the transfer transistor 321 transfers charges from the photoelectric conversion element 311 to the floating diffusion layer 342 in accordance with the transfer signal TRGd1 from the driving section 210 .
  • the transfer transistors 322 to 330 transfer charges from the photoelectric conversion elements 312 to 320 to the floating diffusion layer 342 according to the transfer signals TRGd2 to TRGd10 from the driving section 210.
  • the reset transistor 341 initializes the floating diffusion layer 342 in accordance with the reset signal RST from the driving section 210 .
  • the floating diffusion layer 342 accumulates charges and generates a voltage corresponding to the amount of charges.
  • the amplification transistor 343 amplifies the voltage of the floating diffusion layer 342 .
  • the selection transistor 344 outputs the analog signal amplified by the amplification transistor 343 to the vertical signal line 309 as the pixel signal Vin according to the selection signal SEL from the drive unit 210 .
  • one vertical signal line 309 is wired for each column of pixel blocks.
  • the level of the pixel signal Vin when the floating diffusion layer 342 is initialized is hereinafter referred to as "P-phase level” or “reset level”.
  • the level of the pixel signal Vin when the charge is transferred to the floating diffusion layer 342 is hereinafter referred to as "D-phase level” or “signal level”.
  • FIG. 4 is a circuit diagram showing one configuration example of the octa pixel block 350 according to the first embodiment of the present technology.
  • This octa pixel block 350 includes photoelectric conversion elements 361 to 368 , transfer transistors 371 to 378 , reset transistor 381 , floating diffusion layer 382 , amplification transistor 383 and selection transistor 384 .
  • the circuit configuration of the octa pixel block 350 is the same as the deca pixel block 300 illustrated in FIG. 3, except that there are eight photoelectric conversion elements and eight transfer transistors. That is, eight pixels in octa-pixel block 350 share floating diffusion layer 382 .
  • the deca pixel block 300 and the octa pixel block 350 differ in the number of transfer transistors connected to the floating diffusion layer. Therefore, the number of parasitic capacitances of the transfer transistors connected to the floating diffusion layer is different, and the charge-to-voltage conversion efficiency of the floating diffusion layer is different. Since the deca pixel block 300 has more parasitic capacitance than the octa pixel block 350, the charge-to-voltage conversion efficiency is reduced accordingly.
  • FIG. 5 is a block diagram showing a configuration example of the reading unit 230 according to the first embodiment of the present technology.
  • the reading unit 230 includes multiple load current sources 231 , multiple ADCs (Analog to Digital Converters) 232 , and a transfer control unit 238 .
  • a load current source 231 and an ADC 232 are arranged for each column of pixel blocks.
  • the load current source 231 is connected to the vertical signal line 309 of the corresponding column.
  • the ADC 232 converts the analog pixel signal Vin from the corresponding column into a digital signal.
  • This ADC 232 includes capacitors 233 and 234, a comparator 235, a counter 236, and a latch circuit 237, for example.
  • the reference signal RMP from the reference signal generator 240 and the pixel signal Vin from the corresponding column are input to the comparator 235 via the capacitors 233 and 234 .
  • Comparator 235 compares the signals and provides the comparison result to counter 236 .
  • the counter 236 counts the count value over the period until the comparison result is inverted. This counter 236 supplies a digital signal indicating the count value to the latch circuit 237 .
  • the counter 236 down-counts (or up-counts) the count value when converting the P-phase level (reset level), and up-counts (or down-counts) when converting the D-phase level (signal level). do. This makes it possible to implement CDS processing for obtaining the difference between the P-phase level and the D-phase level. Note that the counter 236 can be configured to perform only one of up-counting and down-counting, and the CDS processing can be performed by a subsequent circuit.
  • the latch circuit 237 holds a digital signal and outputs it as a digital pixel signal Dout under the control of the transfer control section 238 .
  • the transfer control unit 238 controls each of the ADCs 232 to sequentially output the pixel signal Dout.
  • timing control section 260 can control the reference signal generation section 240 to change the slope of the reference signal RMP.
  • ADC 232 is arranged for each column of pixel blocks, the configuration is not limited to this. More than one ADC 232 can be placed per column of pixel blocks. In this case, two or more vertical signal lines 309 are wired for each column.
  • a column amplifier 239 can be added for each column of pixel blocks, and the timing control section 260 can control the analog gain of the column amplifier 239. Details of the analog gain setting method in FIG. 5 or 6 will be described later.
  • FIG. 7 is a diagram for explaining imaging modes in the first embodiment of the present technology.
  • the imaging modes are classified into a normal mode in which HDR synthesis processing is not performed and an HDR mode in which HDR synthesis processing is performed.
  • the HDR mode is classified into an exposure ratio mode and a pixel addition ratio mode.
  • the driving section 210 can add the pixel signals of the pixels of the same color in each pixel block. Since 10 G pixels are arranged in the deca pixel block 300, up to 10 G pixels can be added. Since 8 R or B pixels are arranged in an octa pixel block, up to 8 R or B pixels can be added.
  • the exposure ratio mode is a mode in which multiple exposures with different exposure times are performed when performing HDR synthesis.
  • the driving unit 210 adds pixels having the same exposure time for each pixel block.
  • the drive unit 210 exposes six G pixels in the deca pixel block 300 with the same exposure time, and adds the pixels. In addition, the drive unit 210 exposes the remaining four G pixels in the deca pixel block 300 with a shorter exposure time and adds them together.
  • G: 6-4 in the figure means that 6 pixels in the deca pixel block 300 are added and the remaining 4 pixels with different exposure times are added.
  • the driving section 210 exposes four R pixels or B pixels in the octa pixel block 350 with the same exposure time, and adds the pixels.
  • the driver 210 also exposes the remaining four pixels in the octa pixel block 350 with a shorter exposure time and causes them to be pixel summed.
  • "R/B: 4-4" in the figure means that 4 pixels in the octa pixel block 350 are added and the remaining 4 pixels with different exposure times are added.
  • a frame composed of pixel signals after pixel addition corresponding to the longer exposure time is hereinafter referred to as a "long storage frame”.
  • the frame which consists of the pixel signal after the pixel addition corresponding to the shorter exposure time is hereafter called a "short-storing frame.”
  • the pixel addition ratio mode is a mode in which the sensitivity of a part of the shared block is made higher than that of the rest of the area by pixel addition during HDR synthesis.
  • the driving unit 210 adds pixel signals of a partial area and adds pixel signals of the remaining area for each pixel block.
  • the number of additions for each area is different.
  • the exposure time for each region is the same, the number of additions is different, so the sensitivity for each region is a different value. Therefore, by synthesizing these pixel signals, it is possible to expand the dynamic range.
  • the drive unit 210 exposes all pixels with the same exposure time. Then, for example, the drive unit 210 performs pixel addition of eight G pixels out of ten in the deca pixel block 300, and performs pixel addition of the remaining two G pixels. Thereby, two pixel signals with different addition numbers are generated. "G: 8-2" in the figure means that 8 pixels in the deca pixel block 300 are added and the remaining 2 pixels with the same exposure time are added.
  • the driving unit 210 performs pixel addition of 6 R pixels or B pixels out of 8 pixels in the octa pixel block 350, and performs pixel addition of the remaining 2 pixels. Thereby, two pixel signals with different addition numbers are generated.
  • "R/B: 6-2" in the figure means that 6 pixels in the octa pixel block 350 are added and the remaining 2 pixels with the same exposure time are added.
  • a frame composed of pixel signals after addition of pixels with a larger number of additions is called a "long storage frame” as in the case where the exposure time is long.
  • the frame which consists of the pixel signal after pixel addition of the side with a smaller number of additions is called a "short-term frame” similarly to the case where exposure time is short.
  • the long-accumulation frame and the short-accumulation frame have the same exposure time, but have different sensitivities because the numbers of additions are different. Therefore, by synthesizing these frames, the dynamic range can be expanded.
  • FIG. 8 is a block diagram showing a configuration example of the signal processing unit 250 according to the first embodiment of the present technology.
  • the signal processing section 250 includes a frame buffer 251 , a mode setting section 252 and an image processing section 253 .
  • the frame buffer 251 holds frames from the reading unit 230 .
  • the mode setting unit 252 sets the imaging mode. For example, it is assumed that control unit 110 sets either normal mode or HDR mode according to a user's operation. Mode setting unit 252 determines whether the current imaging mode is normal mode or HDR mode.
  • the mode setting unit 252 sets either the exposure ratio mode or the pixel addition ratio mode based on predetermined parameters related to the imaging scene.
  • the amount of motion of the subject, the shutter speed, the frame rate, etc. are used as parameters related to the shooting scene. If the amount of motion is smaller than a predetermined value, the exposure ratio mode is set; otherwise, the pixel addition ratio mode is set. Also, when the shutter speed is longer than a predetermined value, the exposure ratio mode is set, and otherwise, the pixel addition ratio mode is set. Also, when the frame rate is lower than a predetermined value, the exposure ratio mode is set, and otherwise, the pixel addition ratio mode is set. Note that the configuration is not limited to using only one of motion amount, shutter speed, and frame rate, and mode setting section 252 can also set the mode using a plurality of parameters.
  • the mode setting section 252 generates a mode signal MODE indicating the set mode, and supplies it to the driving section 210 and the timing control section 260 .
  • a mode signal MODE indicating the set mode
  • the mode setting unit 252 can also set either the normal mode or the HDR mode based on the imaging scene or the like.
  • the image processing unit 253 reads frames from the frame buffer 251 and performs various image processing.
  • the image processing unit 253 executes phase difference AF, demosaic processing, and the like as necessary.
  • the image processing unit 253 executes HDR synthesis processing for synthesizing the long accumulation frame and the short accumulation frame.
  • the image processing unit 253 can further perform demosaic processing or the like after the HDR synthesis processing.
  • the image processing unit 253 can also perform phase difference AF in addition to HDR synthesis processing.
  • the image processing unit 253 outputs the processed frame.
  • the drive unit 210 drives the pixel array unit 220 when the exposure ratio mode is set, and exposes a plurality of pixels for different exposure times.
  • the driving unit 210 sets the sensitivity of a part of the pixel block to a value different from that of the rest of the pixel block by pixel addition.
  • the drive unit 210 exposes six pixels in the deca pixel block 300 for a long time to perform pixel addition. Further, the drive unit 210 exposes four pixels in the deca pixel block 300 for a short period of time to add the pixels. Within the octa pixel block 350, 4 of the 8 pixels and the remaining 4 pixels are exposed with different exposure times.
  • the 6 pixels or 4 pixels with the longer exposure time are examples of the first pixels described in the claims, and the 4 pixels with the shorter exposure times are the second pixels described in the claims. is.
  • the drive unit 210 adds eight pixels in the deca pixel block 300 and adds the remaining two pixels. Within the octa pixel block 350, 6 pixels are added and the remaining 2 pixels are added. A pixel obtained by adding 6 pixels or 8 pixels is an example of the third pixel described in the claims. A pixel obtained by adding two pixels is an example of the fourth pixel described in the claims.
  • timing control unit 260 adjusts the gain for the analog pixel signal according to the imaging mode. A gain adjustment method will be described later.
  • the solid-state imaging device 200 adds analog pixel signals, it is not limited to this configuration, and digital pixel signals can also be added. Furthermore, although the solid-state imaging device 200 adjusts the gain for analog pixel signals, it is also possible to adjust the gain for digital pixel signals.
  • FIG. 9 is a timing chart showing an example of an exposure ratio mode driving method when phase difference AF is not performed according to the first embodiment of the present technology.
  • the driving unit 210 sequentially selects and exposes rows of pixel blocks.
  • timing T0 be the timing for starting exposure of a predetermined row.
  • the driving section 210 turns on the reset transistors and the transfer transistors of 10 pixels in the deca pixel block 300 of that row to start exposing those pixels.
  • Timing T2 after TS has elapsed from timing T0 is the shorter exposure end timing.
  • the driving section 210 sets the selection signal SEL to high level, and sets the reset signal RST to high level over the pulse period. Thereby, the floating diffusion layer 342 in the deca pixel block 300 is initialized.
  • the driving section 210 sets the transfer signals TRGd1, TRGd2, TRGd9, and TRGd10 to high level over the pulse period. As a result, charges are transferred from four pixels, and the pixel signals of those pixels are added. At timing T3, the driving section 210 changes the selection signal SEL to low level.
  • Timing T5 after TL has elapsed from timing T0 is the timing of the end of the longer exposure.
  • the driving section 210 sets the selection signal SEL to high level, and sets the reset signal RST to high level over the pulse period. Thereby, the floating diffusion layer 342 is initialized.
  • the driving section 210 sets the transfer signals TRGd3 to TRGd8 to high level over the pulse period. As a result, charges are transferred from six pixels, and the pixel signals of those pixels are added. At timing T6, the driving section 210 changes the selection signal SEL to low level.
  • the reading unit 230 reads the P-phase level (reset level) immediately after timings T1 and T4, reads the D-phase level (signal level) immediately after timings T2 and T5, and obtains the difference between them as the net signal level.
  • the octa pixel block 350 is driven in the same way as the deca pixel block 300 except that 4 pixels are exposed for a long time and the remaining 4 pixels are exposed for a short time.
  • FIG. 10 is a timing chart showing an example of an exposure ratio mode driving method when performing phase difference AF according to the first embodiment of the present technology.
  • the driving section 210 sets the selection signal SEL to high level at timing T11 immediately before the end of exposure, and sets the reset signal RST to high level over the pulse period. Thereby, the floating diffusion layer 342 is initialized.
  • the driving section 210 sets the transfer signals TRGd1 and TRGd9 to high level over the pulse period. This transfers charge from the left side of each of the two pairs of pixels.
  • the driving section 210 sets the transfer signals TRGd1, TRGd2, TRGd9 and TRGd10 to high level over the pulse period. This transfers charges from the two pairs of pixels.
  • the driving section 210 changes the selection signal SEL to low level.
  • the reading unit 230 reads the P-phase level (reset level) immediately after timing T11, and reads the D-phase level (signal level) immediately after each of timings T12 and T13.
  • the first D-phase level corresponds to the D-phase level of the signal obtained by adding the left sides of the two pairs of pixels.
  • the second D-phase level corresponds to the D-phase level of the signal obtained by adding two pairs of pixels.
  • Reading section 230 acquires the left net D-phase level from the difference between the first D-phase level and the P-phase level. Further, reading section 230 obtains the right net D-phase level by subtracting the first D-phase level and the P-phase level from the second D-phase level.
  • the subsequent signal processing unit 250 acquires the phase difference from the D-phase level on the left side and the D-phase level on the right side, and performs phase difference AF. Note that the signal processing section 250 instead of the reading section 230 can obtain the difference between the P-phase level and the D-phase level.
  • the driving method for reading out the P-phase level, the D-phase level, and the D-phase level in this order is hereinafter referred to as "PDD reading".
  • PDD readout is also performed for the remaining six pixels in the deca pixel block 300 .
  • PDD readout is also performed in octa pixel block 350 .
  • the phase difference can be detected in all pixel blocks, it is also possible to detect the phase difference only in a partial area as necessary.
  • FIG. 11 is a diagram showing an example of addition targets and frames when exposure is performed twice in the exposure ratio mode according to the first embodiment of the present technology.
  • a in the same figure is a figure which shows an example of addition object.
  • b in the same figure is a figure which shows an example of the short accumulation frame 501 after addition.
  • c in the same figure is a figure which shows an example of the long storage frame 502 after addition.
  • the four pixels in the first row are labeled Gr1 to Gr4 from the left
  • the two pixels in the second row are labeled Gr5 and Gr6 in order from the left
  • the four pixels in the third row are labeled Gr7 to Gr10 in order from the left. do.
  • two pixels in the first row in the octa pixel block 350 are B1 and B2 in order from the left
  • four pixels in the second row are B3 to B6 in order from the left
  • two pixels in the third row are B7 and B7 in order from the left.
  • R pixels and Gb pixels are R1 to R8 and Gb1 to Gb10.
  • the driving unit 210 exposes Gr1, Gr2, Gr9 and Gr10 with the shorter exposure time.
  • “(S)” in the figure indicates that the exposure time is short.
  • the drive unit 210 exposes the remaining Gr3 to Gr8 with the longer exposure time.
  • “(L)” in the figure indicates that the exposure time is long.
  • B1, B2, B9 and B10 are exposed for short periods of time and the rest are exposed for long periods of time. The same applies to R pixels and Gb pixels.
  • the driving unit 210 adds pixels having the same exposure time.
  • a short frame 501 is generated by pixel addition.
  • R(S) indicates a pixel obtained by adding R1, R2, R7 and R8.
  • Gr(S) indicates a pixel obtained by adding Gr1, Gr2, Gr9 and Gr10.
  • Gb(S) indicates a pixel obtained by adding Gb1, Gb2, Gb9 and Gb10.
  • B(S) indicates a pixel obtained by adding B1, B2, B7 and B8.
  • a long frame 502 is generated by pixel addition.
  • R(L) indicates a pixel obtained by adding R3 to R6.
  • Gr(L) indicates a pixel obtained by adding Gr3 to Gr8.
  • Gb(L) indicates a pixel obtained by adding Gb3 to Gb8.
  • B(L) indicates a pixel obtained by adding B3 to B6.
  • the image processing unit 253 can generate a frame with an expanded dynamic range by synthesizing the short-term frame 501 and the long-term frame 502 with different exposure times. Further, the image processing unit 253 can also perform phase difference AF by the driving unit 210 performing the above-described PDD reading.
  • FIG. 12 is a diagram illustrating an example of addition targets when exposure is performed three times in the exposure ratio mode according to the first embodiment of the present technology.
  • the longest exposure time is TL
  • the shortest exposure time is TS
  • the middle exposure time is TM .
  • "(M)" in the figure indicates that the exposure time is TM .
  • the driver 210 exposes, for example, 4 pixels with the longest TL , 4 pixels with the middle TM, and the remaining 2 pixels with TM . Expose with the shortest TS . Also, in the octa-pixel block 350, the driver 210 may, for example, expose 4 pixels with the longest TL , 2 pixels with the middle TM , and the remaining 2 pixels with the shortest TS .
  • the driver 210 exposes, for example, 4 pixels with the longest TL , 2 pixels with the middle TM , and the remaining 4 pixels, as illustrated in FIG. A pixel can also be exposed with the shortest TS .
  • the exposure time of the octa pixel block 350 of b in the figure is the same as that of a in the figure.
  • FIG. 13 is a diagram illustrating an example of addition targets when exposure is performed four times in the exposure ratio mode according to the first embodiment of the present technology. Let the longest exposure time be TL , the shortest exposure time TS , and the intermediate exposure times TM and Tm . The exposure times for TM and Tm are assumed to be different. "(m)" in the figure indicates that the exposure time is Tm .
  • the driver 210 exposes, for example, 4 pixels with the longest TL , 2 pixels with the middle TM , and 2 pixels with the middle Tm . and the remaining two pixels are exposed with the shortest TS . Also, in the octa-pixel block 350, the driver 210 may, for example, expose 2 pixels with the longest TL , 2 pixels with the middle TM , 2 pixels with the middle Tm , and the remaining Two pixels are exposed with the shortest TS .
  • the solid-state imaging device 200 in the exposure ratio mode, can set various exposure times such as 2 times, 3 times, and 4 times. Note that the number of times of exposure in the exposure ratio mode and the arrangement of objects to be added are not limited to those illustrated in FIGS. 11 to 13 .
  • FIG. 14 is a timing chart showing an example of the pixel addition ratio mode driving method according to the first embodiment of the present technology.
  • the decapixel block 300 two pixels are exposed simultaneously, followed immediately by the remaining eight pixels.
  • the driving section 210 sets the selection signal SEL to high level, and sets the reset signal RST to high level over the pulse period. Thereby, the floating diffusion layer 342 in the deca pixel block 300 is initialized. Then, at timing T32, the driving section 210 sets the transfer signals TRGd1 and TRGd10 to high level over the pulse period. As a result, charges are transferred from two pixels, and the pixel signals of those pixels are added. Subsequently, at timing T33, the drive unit 210 sets the transfer signals TRGd2 to TRGd9 to high level over the pulse period. As a result, charges are transferred from the remaining eight pixels, and the pixel signals of those pixels are added. At timing T34, the driving section 210 changes the selection signal SEL to low level.
  • the reading unit 230 reads the P-phase level (reset level) immediately after timing T31, reads the D-phase level (signal level) immediately after timings T32 and T33, and obtains the difference between them as the net signal level.
  • the driving method of the octa-pixel block 350 is the same as that of the deca-pixel block 300 except that 2 pixels are added and the remaining 6 pixels are added.
  • the pixel signals of two pixels are added and read out as the pixel signals of the short frame. Then, the pixel signals of the remaining six pixels having the same exposure time are added, and these are read out as the pixel signals of the long storage frame.
  • the image processing unit 253 cannot perform phase difference AF in the driving method shown in FIG.
  • FIG. 15 is a diagram illustrating an example of addition objects when phase difference AF is impossible in the pixel addition ratio mode according to the first embodiment of the present technology.
  • a shows an example of an addition object when phase difference AF is impossible in the pixel addition ratio mode
  • b in the same figure shows another example in which the position of the addition object is changed.
  • the solid-state imaging device 200 can add the two pixels at the upper left and the lower right in the pixel block, and add the remaining pixels.
  • deca pixel block 300 Gr1 and Gr10 etc. are added and the remaining 6 pixels are added.
  • octa pixel block 350 B1 and B8 etc. are added and the remaining 6 pixels are added.
  • the image processing unit 253 cannot perform phase difference AF because pixel pairs such as G1 and Gr10 are not adjacent pixel pairs in the horizontal direction. However, the center of gravity of exposure can be aligned between the long-accumulated frame and the short-accumulated frame.
  • FIG. 16 is a diagram illustrating an example of addition targets when phase difference AF is possible in the pixel addition ratio mode according to the first embodiment of the present technology.
  • a shows an example of an addition object when phase difference AF is possible in the pixel addition ratio mode
  • b in the same figure shows another example in which the position of the addition object is changed.
  • the solid-state imaging device 200 can add two pixels in the third row in the pixel block and add the remaining pixels.
  • deca pixel block 300 Gr7 and Gr8 etc. are added and the remaining 6 pixels are added.
  • octa pixel block 350 B7 and B8 etc. are added and the remaining 6 pixels are added.
  • the solid-state imaging device 200 can add the central two pixels in the pixel block and add the remaining pixels.
  • deca pixel block 300 Gr5 and Gr6 etc. are added and the remaining 6 pixels are added.
  • octa pixel block 350 B4 and B5 etc. are added and the remaining 6 pixels are added.
  • pixel pairs such as Gr7 and Gr8 are adjacent pixel pairs in the horizontal direction, so the image processing unit 253 performs phase difference AF by PDD reading. can be done.
  • the center of gravity of exposure shifts between the long-accumulated frame and the short-accumulated frame.
  • FIG. 17 is a diagram for explaining a method of setting the imaging mode according to the first embodiment of the present technology.
  • the solid-state imaging device 200 sets the pixel addition ratio mode, for example, when the motion amount is larger than the threshold, and sets the exposure ratio mode otherwise.
  • any one of 2, 3, and 4 exposure times is set.
  • the dynamic range can be increased.
  • the number of pixels to be added decreases, so the SN (Signal-Noise) ratio deteriorates.
  • the number of exposures is set in consideration of the balance between dynamic range and SN ratio. Further, in the pixel addition ratio mode, it is possible to set whether or not to use phase difference AF.
  • FIG. 18 is a diagram for explaining a method of setting the gain in the exposure ratio mode according to the first embodiment of the present technology. Assume that the number of times of exposure is two. The number of pixel additions for each of the R pixels and B pixels with the longer exposure times is four, and the number of pixel additions for the G pixels with the longer exposure times is six. The number of pixel additions for each of the R and B pixels with the shorter exposure time is four, and the number of pixel additions for the G pixel with the shorter exposure time is four.
  • the charge-to-voltage conversion efficiency is reduced accordingly. For example, if the charge-voltage conversion efficiency is "100" when the floating diffusion layer is not shared by a plurality of pixels, the charge-voltage conversion efficiency corresponding to the R pixel or B pixel when shared is "71.8". . Also, the charge-to-voltage conversion efficiency corresponding to the G pixel is assumed to be "67.0".
  • the gain for the pixel signal of the R pixel or B pixel, which has the shorter exposure time, is also "1.400" according to the same calculation formula as formula 1.
  • FIG. 19 is a diagram showing an example of signal levels before and after amplification in the exposure ratio mode according to the first embodiment of the present technology.
  • a indicates an example of the signal level for each pixel before amplification in the exposure ratio mode.
  • b in the figure shows an example of the signal level for each pixel after amplification in the exposure ratio mode.
  • the signal levels of the R, G, and B pixels with short exposure times are lower than the Gr and Gb pixels with long exposure times. Become. That is, before amplification, a difference in sensitivity occurs between the R and B pixels and the G pixel.
  • the readout unit 230 amplifies the pixel signals of the R, G, and B pixels using the gains obtained by the formulas (1) and (2).
  • the ratio of the exposure time between the long-term frame and the short-term frame is set to 1:1. set.
  • FIG. 20 is a diagram for explaining a method of setting the gain in the pixel addition ratio mode according to the first embodiment of the present technology.
  • the number of pixel additions of each of the R pixels and B pixels corresponding to the long accumulation frame is six, and the number of pixel additions of the G pixels corresponding to the long accumulation frame is eight.
  • the number of pixel additions for each of the R pixels and B pixels corresponding to the short-term frame is two, and the number of pixel additions for the G pixels corresponding to the short-term frame is two.
  • Gains are set so that the sensitivities of the respective pixels are uniform in each of the long-accumulated frame and the short-accumulated frame after amplification. For example, when the gain for the pixel signal of the G pixel corresponding to the long frame is set to "1.000", the gain for the pixel signal of the R pixel or B pixel corresponding to the long frame is obtained by the following equation, for example. be done. (67.0 ⁇ 8)/(71.8 ⁇ 6) ⁇ 1.244 Expression 3
  • the gain for the pixel signal of the R pixel or the B pixel corresponding to the short-term frame is set to "1.000"
  • the gain for the pixel signal of the G pixel corresponding to the short-term frame is obtained by the following equation, for example. be done. (67.0 ⁇ 2)/(71.8 ⁇ 2) ⁇ 1.072 Expression 4 It should be noted that the gains obtained by Equations 3 and 4 are values that have not been converted into decibels.
  • the gain for the pixel signal is set based on the ratio of the charge-to-voltage conversion efficiency and the ratio of the number of pixel additions of the deca pixel block 300 and the octa pixel block 350, respectively.
  • the ADC 232 and the column amplifier 239 amplify the pixel signal with their gains. Note that the ADC 232 or the column amplifier 239 is an example of an amplifier circuit described in claims.
  • FIG. 21 is a diagram showing an example of signal levels before and after amplification in the pixel addition ratio mode according to the first embodiment of the present technology.
  • a indicates an example of the signal level for each pixel before amplification in the pixel addition ratio mode.
  • b in the figure shows an example of the signal level for each pixel after amplification in the pixel addition ratio mode.
  • the amount of light received per unit time is the same for each of the R, Gr, Gb, and B pixels.
  • the signal levels of the R and B pixels are lower than those of the Gr and Gb pixels in the long storage frame with a large number of additions. lower.
  • the signal levels of R and B pixels are higher than those of Gr and Gb pixels.
  • the output levels of R pixels and B pixels are lower than that of G pixels. Also, in a short frame, the output levels of R pixels and B pixels are higher than that of G pixels.
  • the ratio of the output level of the G pixel of the long frame (4 ⁇ V0, etc.) to the output level of the G pixel of the short frame (V0, etc.) is equal to the ratio of the number of additions (4:1, etc.). The same applies to R pixels and B pixels.
  • the readout unit 230 amplifies the pixel signals of the R, G, and B pixels using the gains obtained by Equations 3 and 4. As a result, the sensitivities can be made uniform, as illustrated by b in FIG.
  • FIG. 22 is a flow chart showing an example of the operation of the imaging device 100 according to the first embodiment of the present technology. This operation is started, for example, when the HDR mode is set.
  • the imaging device 100 analyzes the image data (frame), determines the main subject (step S901), and measures the amount of movement (step S902). For example, the amount of motion is measured by a background subtraction method or an inter-frame subtraction method.
  • the imaging apparatus 100 estimates the longest exposure time Tm that does not cause blur based on the amount of motion and various imaging parameters (step S903), and determines whether Tm is equal to or less than Tth (step S903). S904).
  • step S904 If Tm is less than or equal to Tth (step S904: Yes), the imaging apparatus 100 performs imaging in the pixel addition ratio mode (step S905). On the other hand, if Tm is longer than Tth (step S904: No), the imaging apparatus 100 performs imaging in the exposure ratio mode (step S906).
  • the exposure ratio mode When the exposure ratio mode is set in an imaging scene in which the amount of motion of the subject is large (in other words, the exposure time Tm is short), blurring may occur in the long storage frame. Therefore, it is desirable to set the pixel addition ratio mode. On the other hand, in the pixel addition ratio mode, blurring is less likely to occur, but it is more difficult to expand the dynamic range than in the exposure ratio mode. Therefore, if the amount of movement of the subject is small, it is desirable to set the exposure ratio mode. In this way, by switching the mode according to the imaging scene, it is possible to achieve both suppression of blurring and expansion of the dynamic range, thereby improving image quality.
  • step S905 or S906 the imaging apparatus 100 performs HDR synthesis processing (step S907), and ends the operation for imaging.
  • steps S901 to S907 are repeatedly executed in synchronization with the vertical synchronization signal.
  • the imaging apparatus 100 sets either the pixel addition ratio mode or the exposure ratio mode based on the comparison result between Tm and Tth obtained from the amount of motion, but the setting method is not limited to this.
  • the imaging apparatus 100 can also set either the pixel addition ratio mode or the exposure ratio mode based on the comparison result between the amount of motion and a predetermined value without obtaining Tm .
  • FIG. 23 is a diagram showing a setting example of each imaging mode in the first embodiment of the present technology.
  • the number of additions for G pixels is ten, and the number of additions for R and B pixels is eight. Also, phase difference AF is possible, and the frame rate is 60 fps (frame per second).
  • the exposure ratio mode when the number of times of exposure is two, the number of additions for G pixels is 6-4, and the number of additions for R and B pixels is 4-4. Also, phase difference AF is possible, and the output ratio by pixel addition (in other words, the ratio of the number of additions) is 1.5:1 for G pixels.
  • the output ratio of R pixels and B pixels is 1:1.
  • the exposure time ratio is, for example, 1:4, and the frame rate is 30 fps.
  • the exposure ratio mode when the number of exposures is 3, the number of additions for G pixels is 4-4-2, and the number of additions for R and B pixels is 4-2-2. Phase difference AF is possible.
  • the output ratio of G pixels is 2:2:1, and the output ratio of R and B pixels is 2:1:1.
  • the exposure time ratio is, for example, 1:4:16, and the frame rate is 20 fps.
  • the number of additions for G pixels is 8-2, and the number of additions for R and B pixels is 6-2.
  • Phase difference AF may or may not be possible depending on the addition target position.
  • the output ratio of G pixels is 4:1, and the output ratio of R and B pixels is 3:1.
  • the exposure time ratio is 1:1 and the frame rate is 30 fps.
  • the mode setting unit 252 sets either the pixel addition ratio mode or the exposure ratio mode based on parameters such as the amount of motion. It is possible to expand the dynamic range while suppressing blurring. Thereby, the image quality of the image data (frame) can be improved.
  • the mode setting unit 252 sets the mode based on the exposure time Tm obtained from the amount of motion . can also be set.
  • the mode setting unit 252 in the second embodiment differs from that in the first embodiment in that the mode is switched based on the ISO sensitivity.
  • FIG. 24 is a flow chart showing an example of the operation of the imaging device 100 according to the second embodiment of the present technology.
  • the imaging device 100 determines a main subject (step S901) and determines whether the current ISO sensitivity is higher than a predetermined value Sth (step S910). If the ISO sensitivity is higher than Sth (step S910: Yes), the imaging apparatus 100 performs imaging in the pixel addition ratio mode (step S905). On the other hand, if the ISO sensitivity is equal to or lower than Sth (step S910: No), the imaging apparatus 100 performs imaging in the exposure ratio mode (step S906). After step S905 or S906, the imaging apparatus 100 performs HDR synthesis processing (step S907).
  • the shooting scene is often dark.
  • the pixel addition ratio mode in which the number of additions is greater than in the exposure ratio mode (in other words, the sensitivity is increased).
  • the pixel addition ratio mode although the number of additions can be increased, it is difficult to expand the dynamic range compared to the exposure ratio mode. .
  • the configuration is not limited to using only the ISO sensitivity, and the mode setting unit 252 can set the mode using a plurality of parameters including the ISO sensitivity.
  • the mode setting unit 252 sets either the pixel addition ratio mode or the exposure ratio mode based on the ISO sensitivity, so sensitivity is ensured. while increasing the dynamic range. Thereby, image quality can be improved.
  • the deca pixel block 300 and the octa pixel block 350 are arranged in the pixel array section 220, but pixel blocks other than these can be arranged in the pixel array section 220 as well.
  • the pixel array section 220 of the third embodiment differs from that of the first embodiment in that pixel blocks each consisting of 9 pixels are arranged.
  • FIG. 25 is an example of a plan view of the pixel array section 220 according to the third embodiment of the present technology.
  • the pixel array section 220 of the third embodiment differs from the first embodiment in that nona pixel blocks 390 are arranged in place of the deca pixel blocks 300 and octa pixel blocks 350 .
  • nona pixel block 390 9 pixels of the same color are arranged in 3 rows ⁇ 3 columns sharing a floating diffusion layer.
  • nona-pixel blocks 390 made up of Gb pixels and nona-pixel blocks 390 made up of B pixels are alternately arranged.
  • nona pixel blocks 390 made up of Gr pixels and nona pixel blocks 390 made up of R pixels are alternately arranged.
  • the circuit configuration of the nona pixel block 390 is the same as the circuit configurations illustrated in FIGS. 3 and 4, except that the number of pixels is different.
  • FIG. 26 is a diagram showing an example of addition targets in the exposure ratio mode according to the third embodiment of the present technology.
  • a indicates an example of a pixel with a shorter exposure time
  • b in the figure indicates an example of a pixel with a longer exposure time.
  • the imaging device 100 captures frames with the shorter exposure time.
  • the drive unit 210 adds nine pixels in each nona pixel block 390 . As a result, a short frame of Bayer array is generated.
  • the imaging device 100 captures frames with the longer exposure time.
  • the drive unit 210 adds nine pixels in each nona pixel block 390 .
  • a long storage frame of Bayer array is generated. Long-accumulated frames are HDR-combined with short-accumulated frames.
  • the imaging apparatus 100 performs pixel addition in the exposure ratio mode, it is also possible to read out all pixels without pixel addition and perform HDR synthesis processing.
  • FIG. 27 is a diagram illustrating an example of addition targets in the exposure ratio mode according to the third embodiment of the present technology.
  • the driving section 210 adds eight pixels around the center of each nona pixel block 390 in the block. Thereby, a long storage frame is generated.
  • a frame in which one pixel in the central portion of each nona pixel block 390 is arranged is output as a short frame, and HDR-synthesized with the long frame.
  • FIG. 28 is a diagram for explaining a method of setting the gain in the pixel addition ratio mode according to the third embodiment of the present technology.
  • the number of additions for each of the R pixels, G pixels, and B pixels corresponding to the long frame is eight.
  • the number of additions for each of the R pixels, G pixels, and B pixels corresponding to the short frame is one.
  • the pixel blocks in which the R pixels, G pixels, and B pixels are arranged have the same circuit configuration, so the charge-to-voltage conversion efficiency is the same. Therefore, the gain for each pixel signal is set to the same value (eg, 1.000).
  • the R pixel, G pixel, and B pixel have the same charge-to-voltage conversion efficiency and the same number of additions, so the gain for each pixel signal is set to the same value.
  • FIG. 29 is a diagram showing an example of signal levels in the pixel addition ratio mode according to the third embodiment of the present technology.
  • the G pixel and the B pixel are the same, their output levels are the same. Since the sensitivities are the same, the gains for the pixel signals of the R, G, and B pixels may be the same.
  • the ratio of the pixel output level of the long frame (eg, 8 ⁇ V0) and the output level of the pixel of the short frame (eg, V0) is equal to the ratio of the number of additions (eg, 4:1).
  • the R pixel, G pixel, and B pixel have the same sensitivity, and the gain for each pixel signal may be the same.
  • the second embodiment can be applied to the third embodiment.
  • the gains for the pixel signals of the R, G, and B pixels are adjusted in order to match the sensitivities. no longer need to.
  • the technology (the present technology) according to the present disclosure can be applied to various products.
  • the technology according to the present disclosure can be realized as a device mounted on any type of moving body such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility, airplanes, drones, ships, and robots. may
  • FIG. 30 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology according to the present disclosure can be applied.
  • a vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001.
  • the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, a vehicle exterior information detection unit 12030, a vehicle interior information detection unit 12040, and an integrated control unit 12050.
  • a microcomputer 12051, an audio/image output unit 12052, and an in-vehicle network I/F (interface) 12053 are illustrated.
  • the drive system control unit 12010 controls the operation of devices related to the drive system of the vehicle according to various programs.
  • the driving system control unit 12010 includes a driving force generator for generating driving force of the vehicle such as an internal combustion engine or a driving motor, a driving force transmission mechanism for transmitting the driving force to the wheels, and a steering angle of the vehicle. It functions as a control device such as a steering mechanism to adjust and a brake device to generate braking force of the vehicle.
  • the body system control unit 12020 controls the operation of various devices equipped on the vehicle body according to various programs.
  • the body system control unit 12020 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as headlamps, back lamps, brake lamps, winkers or fog lamps.
  • body system control unit 12020 can receive radio waves transmitted from a portable device that substitutes for a key or signals from various switches.
  • the body system control unit 12020 receives the input of these radio waves or signals and controls the door lock device, power window device, lamps, etc. of the vehicle.
  • the vehicle exterior information detection unit 12030 detects information outside the vehicle in which the vehicle control system 12000 is installed.
  • the vehicle exterior information detection unit 12030 is connected with an imaging section 12031 .
  • the vehicle exterior information detection unit 12030 causes the imaging unit 12031 to capture an image of the exterior of the vehicle, and receives the captured image.
  • the vehicle exterior information detection unit 12030 may perform object detection processing or distance detection processing such as people, vehicles, obstacles, signs, or characters on the road surface based on the received image.
  • the imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of received light.
  • the imaging unit 12031 can output the electric signal as an image, and can also output it as distance measurement information.
  • the light received by the imaging unit 12031 may be visible light or non-visible light such as infrared rays.
  • the in-vehicle information detection unit 12040 detects in-vehicle information.
  • the in-vehicle information detection unit 12040 is connected to, for example, a driver state detection section 12041 that detects the state of the driver.
  • the driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 detects the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041. It may be calculated, or it may be determined whether the driver is dozing off.
  • the microcomputer 12051 calculates control target values for the driving force generator, the steering mechanism, or the braking device based on the information inside and outside the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, and controls the drive system control unit.
  • a control command can be output to 12010 .
  • the microcomputer 12051 realizes the functions of ADAS (Advanced Driver Assistance System) including collision avoidance or shock mitigation, follow-up driving based on inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, or vehicle lane deviation warning. Cooperative control can be performed for the purpose of ADAS (Advanced Driver Assistance System) including collision avoidance or shock mitigation, follow-up driving based on inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, or vehicle lane deviation warning. Cooperative control can be performed for the purpose of ADAS (Advanced Driver Assistance System) including collision avoidance or shock mitigation, follow-up driving based on inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, or vehicle
  • the microcomputer 12051 controls the driving force generator, the steering mechanism, the braking device, etc. based on the information about the vehicle surroundings acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, so that the driver's Cooperative control can be performed for the purpose of autonomous driving, etc., in which vehicles autonomously travel without depending on operation.
  • the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the information detection unit 12030 outside the vehicle.
  • the microcomputer 12051 controls the headlamps according to the position of the preceding vehicle or the oncoming vehicle detected by the vehicle exterior information detection unit 12030, and performs cooperative control aimed at anti-glare such as switching from high beam to low beam. It can be carried out.
  • the audio/image output unit 12052 transmits at least one of audio and/or image output signals to an output device capable of visually or audibly notifying the passengers of the vehicle or the outside of the vehicle.
  • an audio speaker 12061, a display unit 12062 and an instrument panel 12063 are illustrated as output devices.
  • the display unit 12062 may include at least one of an on-board display and a head-up display, for example.
  • FIG. 31 is a diagram showing an example of the installation position of the imaging unit 12031.
  • the imaging unit 12031 has imaging units 12101, 12102, 12103, 12104, and 12105.
  • the imaging units 12101, 12102, 12103, 12104, and 12105 are provided at positions such as the front nose of the vehicle 12100, the side mirrors, the rear bumper, the back door, and the upper part of the windshield in the vehicle interior, for example.
  • An image pickup unit 12101 provided in the front nose and an image pickup unit 12105 provided above the windshield in the passenger compartment mainly acquire images in front of the vehicle 12100 .
  • Imaging units 12102 and 12103 provided in the side mirrors mainly acquire side images of the vehicle 12100 .
  • An imaging unit 12104 provided in the rear bumper or back door mainly acquires an image behind the vehicle 12100 .
  • the imaging unit 12105 provided above the windshield in the passenger compartment is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, and the like.
  • FIG. 31 shows an example of the imaging range of the imaging units 12101 to 12104.
  • the imaging range 12111 indicates the imaging range of the imaging unit 12101 provided in the front nose
  • the imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided in the side mirrors, respectively
  • the imaging range 12114 The imaging range of an imaging unit 12104 provided in the rear bumper or back door is shown. For example, by superimposing the image data captured by the imaging units 12101 to 12104, a bird's-eye view image of the vehicle 12100 viewed from above can be obtained.
  • At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information.
  • at least one of the imaging units 12101 to 12104 may be a stereo camera composed of a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection.
  • the microcomputer 12051 determines the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and changes in this distance over time (relative velocity with respect to the vehicle 12100). , it is possible to extract, as the preceding vehicle, the closest three-dimensional object on the course of the vehicle 12100, which runs at a predetermined speed (for example, 0 km/h or more) in substantially the same direction as the vehicle 12100. can. Furthermore, the microcomputer 12051 can set the inter-vehicle distance to be secured in advance in front of the preceding vehicle, and perform automatic brake control (including following stop control) and automatic acceleration control (including following start control). In this way, cooperative control can be performed for the purpose of automatic driving in which the vehicle runs autonomously without relying on the operation of the driver.
  • automatic brake control including following stop control
  • automatic acceleration control including following start control
  • the microcomputer 12051 converts three-dimensional object data related to three-dimensional objects to other three-dimensional objects such as motorcycles, ordinary vehicles, large vehicles, pedestrians, and utility poles. It can be classified and extracted and used for automatic avoidance of obstacles. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into those that are visible to the driver of the vehicle 12100 and those that are difficult to see. Then, the microcomputer 12051 judges the collision risk indicating the degree of danger of collision with each obstacle, and when the collision risk is equal to or higher than the set value and there is a possibility of collision, an audio speaker 12061 and a display unit 12062 are displayed. By outputting an alarm to the driver via the drive system control unit 12010 and performing forced deceleration and avoidance steering via the drive system control unit 12010, driving support for collision avoidance can be performed.
  • At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays.
  • the microcomputer 12051 can recognize a pedestrian by determining whether or not the pedestrian exists in the captured images of the imaging units 12101 to 12104 .
  • recognition of a pedestrian is performed by, for example, a procedure for extracting feature points in images captured by the imaging units 12101 to 12104 as infrared cameras, and performing pattern matching processing on a series of feature points indicating the outline of an object to determine whether or not the pedestrian is a pedestrian.
  • the audio image output unit 12052 outputs a rectangular outline for emphasis to the recognized pedestrian. is superimposed on the display unit 12062 . Also, the audio/image output unit 12052 may control the display unit 12062 to display an icon or the like indicating a pedestrian at a desired position.
  • the technology according to the present disclosure can be applied to, for example, the imaging unit 12031 among the configurations described above.
  • the imaging device 100 in FIG. 1 can be applied to the imaging unit 12031 .
  • the technology according to the present disclosure it is possible to expand the dynamic range and obtain an easier-to-see photographed image, thereby reducing driver fatigue.
  • the present technology can also have the following configuration.
  • (1) When the predetermined exposure ratio mode is set, the first pixel and the second pixel are exposed for different exposure times. a driving unit that sets the sensitivity of the pixel to a value different from that of the fourth pixel; and a mode setting unit that sets either the pixel addition ratio mode or the exposure ratio mode based on a predetermined parameter related to an imaging scene.
  • (2) further comprising a pixel array section including a predetermined number of pixel blocks in which a plurality of pixels sharing the floating diffusion layer are arranged;
  • the driving section drives a pixel at a predetermined position within the pixel block as the first pixel, and drives a pixel different from the predetermined position within the pixel block as the second pixel.
  • the solid-state imaging device which is driven as . (3) nine pixels are arranged in each of the pixel blocks; When the pixel addition ratio mode is set, the driving section generates a pixel signal of the center pixel in the pixel block as a pixel signal of the third pixel, and outputs the pixel signal of each of eight pixels surrounding the center pixel.
  • the solid-state imaging device according to (2) wherein a signal obtained by adding pixel signals is generated as the pixel signal of the fourth pixel.
  • the pixel block includes: a deca pixel block in which 10 pixels are arranged;
  • the driving unit adds the pixel signals of the first pixels and then adds the pixel signals of the second pixels, and when the pixel addition ratio mode is set, the driving unit adds the pixel signals of the first pixels.
  • a signal obtained by adding pixel signals of a part of the pixel block is generated as a pixel signal of the third pixel
  • a signal obtained by adding pixel signals of the remaining area of the pixel block is generated as the pixel signal of the third pixel. generated as pixel signals of 4 pixels.
  • the solid-state imaging device further comprising an image processing unit that processes the pixel signal; a pair of adjacent pixels in the pixel block share a microlens;
  • the solid-state imaging device according to (4) or (5), wherein the image processing section performs a process of acquiring a phase difference from each pixel signal of the pair of pixels.
  • the image processing section performs a process of acquiring a phase difference from each pixel signal of the pair of pixels.
  • the solid-state imaging device according to any one of (4) to (6), further comprising an amplifier circuit that amplifies the pixel signal within.
  • the solid-state imaging device according to any one of (1) to (7), wherein the parameter includes at least one of motion amount of a subject, shutter speed, and frame rate.
  • the parameter includes ISO (International Organization for Standardization) sensitivity.
  • a driving unit that sets the sensitivity of the pixel to a value different from that of the fourth pixel; a mode setting unit that sets either the pixel addition ratio mode or the exposure ratio mode based on a predetermined parameter related to an imaging scene; and an image processing unit that processes image data read from the driving unit.
  • a control method for a solid-state imaging device comprising: a mode setting procedure for setting either the pixel addition ratio mode or the exposure ratio mode based on a predetermined parameter related to an imaging scene.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
PCT/JP2022/038089 2021-12-08 2022-10-12 固体撮像素子、撮像装置、および、固体撮像素子の制御方法 Ceased WO2023105916A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US18/715,194 US12556834B2 (en) 2021-12-08 2022-10-12 Solid-state imaging element, imaging device, and method for controlling solid-state imaging element
JP2023566111A JPWO2023105916A1 (https=) 2021-12-08 2022-10-12
CN202280080050.8A CN118339849A (zh) 2021-12-08 2022-10-12 固态成像元件、成像装置和用于控制固态成像元件的方法
DE112022005856.0T DE112022005856T5 (de) 2021-12-08 2022-10-12 Festkörperbildgebungselement, Bildgebungsvorrichtung und Verfahren zum Steuern eines Festkörperbildgebungselements

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-199280 2021-12-08
JP2021199280 2021-12-08

Publications (1)

Publication Number Publication Date
WO2023105916A1 true WO2023105916A1 (ja) 2023-06-15

Family

ID=86730060

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/038089 Ceased WO2023105916A1 (ja) 2021-12-08 2022-10-12 固体撮像素子、撮像装置、および、固体撮像素子の制御方法

Country Status (5)

Country Link
US (1) US12556834B2 (https=)
JP (1) JPWO2023105916A1 (https=)
CN (1) CN118339849A (https=)
DE (1) DE112022005856T5 (https=)
WO (1) WO2023105916A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2026034214A1 (ja) * 2024-08-09 2026-02-12 ソニーセミコンダクタソリューションズ株式会社 撮像装置

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2025074766A (ja) * 2023-10-30 2025-05-14 キヤノン株式会社 画像処理装置及び方法、撮像装置、プログラム及び記憶媒体

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013100036A1 (ja) * 2011-12-27 2013-07-04 富士フイルム株式会社 カラー撮像素子
WO2018012051A1 (ja) * 2016-07-13 2018-01-18 ソニー株式会社 固体撮像素子、撮像装置、および、固体撮像素子の制御方法
WO2020170565A1 (ja) * 2019-02-19 2020-08-27 ソニーセミコンダクタソリューションズ株式会社 信号処理方法および撮像装置
WO2021215093A1 (ja) * 2020-04-21 2021-10-28 ソニーセミコンダクタソリューションズ株式会社 固体撮像素子、および、撮像装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7815120B2 (en) * 2008-12-12 2010-10-19 Symbol Technologies, Inc. Electronic shutter for a dual-mode imager
TWI576653B (zh) * 2015-07-31 2017-04-01 廣達電腦股份有限公司 曝光控制系統及其方法
US10560629B2 (en) * 2017-05-23 2020-02-11 Google Llc Systems and methods for automatic exposure in high dynamic range video capture systems
JP7374630B2 (ja) 2019-07-09 2023-11-07 キヤノン株式会社 撮像装置及びその駆動方法
US20240292097A1 (en) * 2023-02-28 2024-08-29 Comcast Cable Communications, Llc Systems and methods for camera mode switching

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013100036A1 (ja) * 2011-12-27 2013-07-04 富士フイルム株式会社 カラー撮像素子
WO2018012051A1 (ja) * 2016-07-13 2018-01-18 ソニー株式会社 固体撮像素子、撮像装置、および、固体撮像素子の制御方法
WO2020170565A1 (ja) * 2019-02-19 2020-08-27 ソニーセミコンダクタソリューションズ株式会社 信号処理方法および撮像装置
WO2021215093A1 (ja) * 2020-04-21 2021-10-28 ソニーセミコンダクタソリューションズ株式会社 固体撮像素子、および、撮像装置

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2026034214A1 (ja) * 2024-08-09 2026-02-12 ソニーセミコンダクタソリューションズ株式会社 撮像装置

Also Published As

Publication number Publication date
US20250030959A1 (en) 2025-01-23
JPWO2023105916A1 (https=) 2023-06-15
US12556834B2 (en) 2026-02-17
DE112022005856T5 (de) 2024-10-10
CN118339849A (zh) 2024-07-12

Similar Documents

Publication Publication Date Title
WO2020110679A1 (ja) 固体撮像素子、撮像装置、および、固体撮像素子の制御方法
US11394913B2 (en) Solid-state imaging element, electronic device, and method for controlling correction of luminance in the solid-state imaging element
WO2020105314A1 (ja) 固体撮像素子、および、撮像装置
JP2019029985A (ja) 固体撮像装置、及び電子機器
JP7240324B2 (ja) 固体撮像素子、撮像装置、および、固体撮像素子の制御方法
JP2018011162A (ja) 固体撮像素子、撮像装置、および、固体撮像素子の制御方法
WO2017163890A1 (ja) 固体撮像装置、固体撮像装置の駆動方法、及び、電子機器
WO2020105313A1 (ja) 固体撮像素子、および、撮像装置
JPWO2019017092A1 (ja) アナログデジタル変換器、固体撮像素子、および、アナログデジタル変換器の制御方法
US11451725B2 (en) Solid-state imaging element, imaging apparatus, and method for controlling solid-state imaging element
TWI809037B (zh) 固體攝像元件及攝像裝置
WO2019159499A1 (ja) 固体撮像素子、撮像装置、および、固体撮像素子の制御方法
US12556834B2 (en) Solid-state imaging element, imaging device, and method for controlling solid-state imaging element
WO2018230367A1 (ja) 撮像装置
US11889207B2 (en) Solid-state imaging element, imaging device, and method for controlling solid-state imaging element
WO2022113528A1 (ja) 固体撮像素子、固体撮像素子の制御方法、および、撮像装置
US12621582B2 (en) Solid-state image sensor, imaging apparatus, and control method for solid-state image sensor
US20240406590A1 (en) Solid-state image sensor, imaging apparatus, and control method for solid-state image sensor
US20250203201A1 (en) Imaging apparatus and electronic device
WO2023112480A1 (ja) 固体撮像素子、撮像装置、および、固体撮像素子の制御方法
KR20250163336A (ko) 고체 촬상 소자, 촬상 장치, 및 고체 촬상 소자의 제어 방법
WO2025263114A1 (ja) イメージセンサ、撮像装置、および、イメージセンサの制御方法
WO2021246311A1 (ja) 撮像装置および電子機器

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22903850

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2023566111

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 18715194

Country of ref document: US

Ref document number: 202280080050.8

Country of ref document: CN

122 Ep: pct application non-entry in european phase

Ref document number: 22903850

Country of ref document: EP

Kind code of ref document: A1

WWG Wipo information: grant in national office

Ref document number: 18715194

Country of ref document: US