JP6677504B2 - Imaging device, imaging method, image frame readout control circuit, and signal processing device - Google Patents

Description

  The present invention relates to an image pickup device equipped with a single-chip CMOS that performs image pickup at a frame frequency of 120 Hz under illumination where a power supply frequency is in the 50 Hz range and an intensity change is 100 Hz, an image pickup method thereof, and image frame readout control. The present invention relates to a circuit and a signal processing device.

  In the power supply frequency range of 50 Hz, a lighting device such as a fluorescent lamp exhibits a change in illumination intensity according to 100 Hz which is a pulsation frequency after rectification. Under such illumination intensity change, when imaging is performed at an imaging frame frequency of 60 Hz by the imaging apparatus, flicker occurs because the frequency of the illumination intensity change is not an integral multiple of the imaging frame frequency.

  Therefore, as a countermeasure against such flicker, the electronic shutter period is set to (1/100) seconds (= 10 milliseconds) (see Patent Documents 1 to 3). This is based on the finding that no matter how the phase of the illumination intensity change and the phase of the electronic shutter are shifted, the amount of light incident during 10 milliseconds is kept constant, so that flicker does not occur.

2. Description of the Related Art In recent years, a CMOS image sensor having 33 million pixels intended to be mounted on a Super Hi-Vision system has been known (see Non-Patent Document 1 below).
In addition, a single-panel camera in which a Bayer color filter is attached to pixels of a CMOS image sensor having 33 million pixels is known (see Non-Patent Document 2).
In the development of Super Hi-Vision, since the construction of the above-described flicker countermeasures is urgently needed, if the above-described method can be used for a CMOS type imaging device having a square lattice pixel group equipped with the Bayer color filter, The existing technology can be used efficiently.

JP-A-5-009373 JP-A-6-125495 JP 2000-299822 A

T. Watabe et al., “A 33Mpixel 120fps CMOS Image Sensor Using 12b Column-Parallel Pipelined Cyclic ADCs,” ISSCC Digest of Technical Papers, pp. 388-389, 2012. H. Shimamoto et al., “A Compact 120 Frames / sec UHDTV2 Camera with 35mm PL Mount Lens” SMPTE Motion Imaging Journal, vol.123, no.4, pp.21-28, 2014.

However, when imaging is performed at an imaging frame frequency of 120 Hz, which is a standard for Super Hi-Vision, under illumination with a change in illumination intensity of 100 Hz, flicker of 20 Hz occurs. In this case, since the imaging frame interval is (1/120) seconds = 8.333 milliseconds, when the electronic shutter period is set to 10 milliseconds, the electronic shutter period for the imaging frame interval is set to 6/5, which is larger than 1 Therefore, it is difficult to execute the imaging itself.

  The present invention has been made in view of the above circumstances, and in a case where a color filter formed by arranging a plurality of unit filters in a regular lattice shape is imaged by a single-plate image sensor mounted on a pixel. Another object of the present invention is to provide an imaging apparatus, an imaging method, an image frame readout control circuit, and a signal processing apparatus capable of reducing flicker that occurs when imaging is performed at an imaging frame frequency of 120 Hz when the illumination intensity change is 100 Hz. It is intended for.

The imaging device of the present invention includes:
For each color filter to be mounted, a unitary filter corresponding to each pixel is combined and arranged in a regular lattice, a photoelectric conversion unit that generates charges in accordance with incident light,
A row or column selection circuit for selecting and driving one of the Y row and the X column for the photoelectric conversion unit, and a column or a row for reading a signal for each of the other of the Y row and the X column An image frame readout control unit including a parallel readout circuit unit,
The pixels of the photoelectric conversion unit are configured to be set to either 7680 pixels in the X direction and 4320 pixels in the Y direction, or 3840 pixels in the X direction and 2160 pixels in the Y direction,
The image frame readout control unit uses a non-progressive method, and controls to output a divided image frame for each line number equal to the corresponding line number of a pixel to which one color filter is attached, and The method is characterized in that the divided image frame interval is set to either 8.333 ms or 8.342 ms, and one charge accumulation time of each pixel in the photoelectric conversion unit is set to 10 ms. is there.

  Here, the above and the following "the pixels of the photoelectric conversion unit are configured to be output as either 7680 pixels in the X direction and 4320 pixels in the Y direction, or 3840 pixels in the X direction and 2160 pixels in the Y direction. "Is defined as" the pixels of the photoelectric conversion unit are physically set to any of 7680 pixels in the X direction and 4320 pixels in the Y direction, and 3840 pixels in the X direction and 2160 pixels in the Y direction. " Further, the number of pixels of the photoelectric conversion unit is expanded to either 7680 pixels in the X direction and 4320 pixels in the Y direction or 3840 pixels in the X direction and 2160 pixels in the Y direction by signal processing of the signal processing unit. Or reduced ”.

For the latter, specifically, a case where pixel interpolation processing is performed by using hardware or software to output in the X direction and the Y direction by being expanded or reduced to the respective pixel numbers described above. Is included.
Here, the relationship between the "row or column selection circuit unit" and the "column or row parallel readout circuit unit" is such that when the row selection circuit unit is employed, the column parallel readout circuit unit is employed, and the column selection circuit unit is employed. Is adopted, a row-parallel readout circuit is employed.

  Here, the above-mentioned “for each mounted color filter, unit filters corresponding to each pixel are combined and arranged in a regular grid pattern” means that R, which is mounted on the pixel of the image sensor, A state in which unit filters such as G, B, and W (white) are formed by arranging pixels in a regular grid pattern for each color filter.

  The "non-progressive method" refers to a so-called interlaced scanning method, which is different from the progressive method, which is a method in which scanning is sequentially performed from one direction of the image sensor.

In the specification of the present application, the “image frame” is a line group formed by interlaced scanning for every p lines, that is, an odd frame composed of 2pn + 1 rows, 2pn + 2 rows,... 2pn + p rows, and 2pn + (p + 1) Row, 2pn + (p + 2) rows,... 2pn + 2p rows. Here, p represents 0 or a positive integer.
Here, not only the odd-numbered frames or even-numbered frames but also the interval between the odd-numbered frames and the even-numbered frames is often referred to as an image frame interval.
However, if this is done in the present specification, the interval between odd frames or between even frames will be referred to as an image frame interval because it may be confusing in an essential part of the invention, but the interval between odd frames and even frames will be described. Is referred to as a divided image frame interval for convenience.

  Preferably, the color filter is a Bayer color filter, and the number of lines equal to the number of corresponding lines of the pixel to which the color filter is attached is two.

Further, it is preferable that the image frame read control unit is configured to control so that one charge accumulation time of each pixel with respect to an image frame interval is 6/10.
Further, the photoelectric conversion unit is preferably configured to be shared pixel in a plurality of pixels arranged in a positive direction grid pattern.

Further, the imaging method of the present invention,
For each of the mounted color filters, a unit filter corresponding to each pixel is combined so that a pixel circuit arranged in a regular lattice form generates charges corresponding to light incident on each of the plurality of pixels. To perform photoelectric conversion,
An imaging method by a CMOS imaging device that performs image frame reading in a predetermined order by designating an address of a Y row and an address of an X column for a pixel that performs the photoelectric conversion,
The pixels that perform the photoelectric conversion are controlled to be set to either 7680 pixels in the X direction and 4320 pixels in the Y direction, or 3840 pixels in the X direction and 2160 pixels in the Y direction,
Image frame reading, using a non-progressive method, control to read the divided image frame for each line number equal to the number of corresponding lines of the pixel to which the one color filter is mounted,
And sets the divided image frame interval to one of 8.333 ms or 8.342 ms, to control a charge accumulation time of each pixel in the photoelectric conversion unit so as to set to 10 ms It is a feature.

Further, the image frame read control circuit of the present invention comprises:
An image frame read control signal for a photoelectric conversion unit that generates charges in accordance with incident light, in which unit filters corresponding to respective pixels are combined and arranged in a regular lattice for each color filter to be mounted. An image frame readout control circuit that outputs
A row selection circuit for selecting an address in the Y row and driving pixels included in the Y row; and a column parallel readout circuit for selecting an address in the X column and reading a signal from a pixel included in the X column. ,
The pixels of the photoelectric conversion unit are controlled to be set as either 7680 pixels in the X direction and 4320 pixels in the Y direction, or 3840 pixels in the X direction and 2160 pixels in the Y direction ,
Reading the images frames, using a non-progressive system, one of the controlled so color filter reads the divided image frame every equal number of lines to a corresponding number of lines of pixels to be attached, and, in the divided image frame The interval is set to either 8.333 milliseconds or 8.342 milliseconds, and corresponds to the plurality of pixels so that the accumulation time of each pixel in the photoelectric conversion unit can be set to 10 milliseconds. And at least the accumulation start instruction signal and the accumulation end instruction signal are controlled so as to be output to the photoelectric conversion unit in a predetermined order.

Further, the signal processing device of the present invention,
For each mounted color filter, select a photoelectric conversion unit that generates charges in accordance with incident light and an address of Y row, where unit filters corresponding to each pixel are combined and arranged in a regular grid. A row selection circuit for driving pixels included in the Y row of the photoelectric conversion unit; and a column readout for selecting an address in the X column and reading a signal from a pixel included in the X column of the photoelectric conversion unit. A signal processing device for reading a divided image frame from the photoelectric conversion unit by a divided image frame signal output from the row selection circuit unit or the column readout circuit unit, comprising:
The pixels of the photoelectric conversion unit are controlled to be set as either 7680 pixels in the X direction and 4320 pixels in the Y direction, or 3840 pixels in the X direction and 2160 pixels in the Y direction,
The reading of the divided image frame is controlled using a non-progressive method so as to configure the divided image frame for each number of lines equal to the number of corresponding lines to which one color filter is attached,
The plurality of image frames are set so that the interval between the divided image frames is set to 8.333 ms or 8.342 ms, and the accumulation time of each pixel in the photoelectric conversion unit can be set to 10 ms. At least an accumulation start instruction signal and an accumulation end instruction signal corresponding to a pixel are output from the row selection circuit unit or the column readout circuit unit to the photoelectric conversion unit in a predetermined order toward the divided image frame signal. Control and
In a divided image frame in which a signal is present for each line number equal to the corresponding line number of the pixel to which the one color filter is attached, for each of a plurality of lines that generate pixel signals by pixel interpolation for each line number equal to the corresponding line number An interlaced progressive conversion unit is provided.

In the image pickup apparatus, the image pickup method, the image frame readout control circuit, and the signal processing apparatus according to the present invention, the effective pixels are output in the X direction and the Y direction as either 7680 pixels and 4320 pixels, or 3840 pixels and 2160 pixels. Is configured to be
Odd-numbered frames composed of 2pn + 1 rows, 2pn + 2 rows,... 2pn + p rows are driven by a non-progressive interlace method of a plurality of lines, a frame frequency is set to 120 Hz, and an electronic shutter period is set to 10 ms. Alternatively, the image frame interval between even frames composed of 2pn + (p + 1) rows, 2pn + (p + 2) rows,... 2pn + 2p rows is 16.667 milliseconds, and the divided image frame interval between odd and even frames is 8.333 millimeters. The signal is read out in seconds, and the accumulation time of each pixel is set to 10 milliseconds (n and p are both 0 or a positive integer).

  That is, when imaging is performed at an imaging frame frequency of 120 Hz, which is a standard of Super Hi-Vision, under illumination with a change in illumination intensity of 100 Hz, flicker of 20 Hz occurs. In order to prevent this, when the electronic shutter speed is set to 10 milliseconds, the imaging frame interval (divided image frame interval) is (1/120) seconds = 8.333 milliseconds. Since the electronic shutter period with respect to (image frame interval) is set to 6/5, which is larger than 1, it becomes difficult to perform imaging.

However, in the present invention, even if the electronic shutter speed is set to 10 milliseconds and the imaging frame interval is set to (1/120) seconds = 8.333 milliseconds, the interlaced non-progressive scanning method for a plurality of lines is used as the scanning method. Since the method is adopted, the electronic shutter period with respect to the imaging frame interval (image frame interval (interval that is twice the divided image frame interval: interval between odd-numbered frames or even-numbered frames)) is set to 6/10 of a value smaller than 1. Since it can be set, even when the pixels are arranged in a square lattice shape with a Bayer color filter, flicker that occurs when imaging at 120 Hz under a 100 Hz illumination intensity change in a power supply frequency of 50 Hz is performed. The occurrence can be prevented.

FIG. 2 is an equivalent circuit diagram of a square lattice pixel without pixel sharing according to the first embodiment of the present invention. FIG. 2 is a block diagram illustrating an image pickup apparatus including a pixel array in which a Bayer color filter is mounted on the pixel circuit illustrated in FIG. 1 and an image frame readout control circuit. FIG. 4 is a diagram of a square lattice pixel arrangement with a Bayer color filter. 2 is a time chart of an input signal to a pixel circuit when a signal is read using the first embodiment having the configuration illustrated in FIG. 1. FIG. 6 shows a signal processing method for forming all image signals by pixel interpolation from an odd frame composed of 4n + 1 rows and 4n + 2 rows when signal reading is performed using the first embodiment having the configuration shown in FIG. . FIG. 4 shows a signal processing method for forming all image signals by pixel interpolation from an even frame composed of 4n + 3 rows and 4n + 4 rows when signal reading is performed using the first embodiment having the configuration shown in FIG. . FIG. 2 is a block diagram (flowchart) illustrating a configuration of a signal processing device that performs signal reading using the first embodiment of the configuration illustrated in FIG. 1. 6 is a time chart showing a time-series relationship between an example of a change in illumination intensity at 100 Hz and an accumulation time in an interlace method for every two lines at 120 Hz. FIG. 11 is an equivalent circuit diagram of a square lattice pixel shared by two vertical pixels according to the second embodiment of the present invention. 10 is a time chart of an input signal to a pixel circuit when signal reading is performed using the second embodiment having the configuration shown in FIG. 9. FIG. 11 is an equivalent circuit diagram of a square lattice pixel shared by four pixels of two vertical pixels and two horizontal pixels according to the third embodiment of the present invention. 12 is a time chart of an input signal to a pixel circuit when a signal is read using the third embodiment of the pixel arrangement shown in FIG. 11.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<First embodiment>
First, an equivalent circuit diagram of a pixel circuit using four transistors per pixel used in the CMOS imaging device according to the first embodiment will be described with reference to FIG. The pixel circuit shown in the equivalent circuit diagram is provided corresponding to each pixel of the pixel array of the CMOS type imaging device.

  As shown in FIG. 1, this pixel circuit includes a photodiode (PD: 411), a charge transfer transistor (TX: 412), a floating diffusion capacitance (FD: 413), a reset transistor (RST: 414), and a source follower amplifier transistor. (SF: 415), a selection transistor (SEL: 416), a pixel power supply section (VDD: 417), and a pixel output section (OUT: 418).

Further, in this pixel circuit, a large number of color filters in a Bayer array are mounted in a square lattice pattern in units of four pixels without any gap as shown in FIG. 3, and the pixel array 201 shown in FIG. 2 is configured.
The pixel array 201 forms an imaging device (image sensor) 200 together with the Y-direction scanning unit 202, the X-direction scanning unit 203, the timing generator 204, and the output circuit 205. Note that, in the imaging apparatus 200, the Y-direction scanning unit 202, the X-direction scanning unit 203, the timing generator 204, and the output circuit 205 constitute an image frame readout control circuit according to the present invention.

In the pixel circuit 400 of FIG. 1, the PD 411 stores an amount of negative charges according to the intensity of incident light. The anode of this PD 411 is grounded, and the cathode is connected to the gate of SF 415 via TX 412. The gate of TX412 is connected to the signal line _{L T} from Y direction scanning unit 202, the transfer signal.

SF 415 and SEL 416 are connected in series between VDD 417 and output section 418. The gate of SEL416 is connected to the signal line _{L S} from the Y-direction scanning unit 202, a selection signal is input. RST 414 is connected between VDD 417 and the gate of SF 415. The gate of the RST414 is connected to the signal line _{L R} from the Y direction scanning unit 202, is inputted to the reset signal.
The FD 413 is connected to the gate of the SF 415.

  In order to reset the PD 411, the SEL 416 is turned off and the TX 412 and the RST 414 are turned on. Thereby, the negative charges accumulated in the PD 411 are released to the VDD 417 via the TX 412 and the RST 414, and the reset operation ends.

  From the end of the reset operation of the PD 411, the accumulation of charges by the incident light starts. That is, when the transfer signal and the reset signal are set to the “L” state and the TX 412 and the RST 414 are turned off, an amount of charge corresponding to the intensity of the incident light is stored in the PD 411, and the charge storage time starts.

  On the other hand, the end of the accumulation time is performed as follows. That is, first, the FD 413 is reset by setting the selection signal to “H” level to turn on the SEL 416 and setting the reset signal to “H” level for a predetermined time to turn on the RST 414. Next, by setting the transfer signal to the “H” level state for a predetermined time to turn on the TX 412, the accumulated charge of the PD 411 is moved to the FD 413, and the accumulation time of the PD 411 ends when the TX 412 is turned off.

The timing generator 204 shown in FIG. 2 sends a row selection address signal and a drive control signal to the Y-direction scanning unit 202, and sends a column selection address signal and a read control signal to the X-direction scanning unit 203. The Y-direction scanning unit 202 has functions of a Y-direction scanning circuit and a voltage level shift circuit, and sequentially selects a predetermined plurality of rows of the pixel array 201 according to the input row selection address signal and the drive control signal. and, the signal line L _T in the selected _row, L _R, via the L _S, the transfer signal to each pixel circuit 400 in that row, sends a reset signal and the selection signal.

The X-direction scanning unit 203 has a function of an X-direction scanning circuit and a column circuit, and receives a plurality of Y-direction signal lines L from a plurality of pixel circuits 400 in a predetermined row selected by the Y-direction scanning unit 202. _The current output to _V is converted into a plurality of predetermined signals.
Further, the output circuit 205 outputs a plurality of pixel signals generated by the X-direction scanning unit 203 to the outside.

  FIG. 4 is a time chart illustrating input signals of respective transistors when signal reading is performed using the pixel circuit 400 illustrated in FIG. In the first embodiment (and the second embodiment described below), the image frame rate is 120 Hz, and non-progressive scanning of interlace scanning every two lines is employed.

  In FIG. 4, each graph shows the signal waveforms of SEL 416, RST 414, and TX 412, and the numbers in parentheses after SEL, RST, and TX are indicated on the corresponding lines (rows) in FIG. Pixel. The accumulation time of each corresponding pixel is indicated by a black band. In the first embodiment, n is set from 0 to (4320/4) −1 = 11079.

  In the pixel circuit 400, first, for the pixels on the first line, in order to reset the PD 411, the RST 416 and the TX 412 are simultaneously turned on (RST (1) while the SEL 416 is off (SEL (1) is at the “L” level)). ) And TX (1) are set to “H” level, and then simultaneously turned off (RST (1) and TX (1) are set to “L” level) (see arrow A1 in FIG. 4). As a result, the signal charges of the PD 411 and the FD 413 are released to the VDD 417 via the TX 412 and the RST 414, and the reset process of the PD 411 ends. Immediately after this, the accumulation time of the PD 411 starts.

  After the accumulation time is started, the SEL 416 is turned on (SEL (1) is at the “H” level) (see the arrow B in FIG. 4), and the pixel is selected. When the RST 414 is turned on (RST (1) is at “H” level), the FD 413 is reset. When the RST 414 is off (RST (1) is at “L” level), the charge of the FD 413 is released. (Reset potential) is read out.

  Next, when the TX 412 is turned on (TX (1) is at the “H” level) after the RST 414 is turned off (RST (1) is at the “L” level) during the accumulation time, the signal stored in the PD 411 is accumulated. When the charges move to the FD 413 and the TX 412 is turned off (TX (1) is at “L” level), the potential at this time is read out (SEL (1), RST (1), TX in FIG. 4). (1), time chart of accumulation time (1): see arrow C1). At this time, the accumulation time of the PD 411 ends. Thus, the time from the selection of the pixel to the OFF state of the RST 414 to the OFF state of the TX 412 is one accumulation time of each pixel. The accumulation time is set to, for example, (1/100) seconds (= 10 milliseconds).

  After that, the same operation is sequentially performed on the pixels in the (2) -th row after the (1) -th row (as shown by the black band in FIG. 4, the accumulation time of the PD 411 is started at the timing of the arrow A2). , The accumulation time of the PD 411 ends at the timing of the arrow C2), and then the (5) th row, the (6) th row,..., The (4n + 1) th row, and the (4n + 2) th row odd-numbered frames. Are sequentially performed in the same manner.

  On the other hand, subsequent to the (3) th row, the same signal reading processing as the above processing of the (1) th row is performed (SEL (3), RST (3), TX (3) and accumulation in FIG. 4). (Refer to the time chart of the time (3)), and the signal reading of the entire (3) th row is completed. Thereafter, the signal reading process of the even-numbered frames of the (4) th line, the (7) th line, the (8) th line,..., The (4n + 3) th line, and the (4n + 4) th line are sequentially performed. And so on.

  That is, in the imaging apparatus according to the present embodiment, the reading operation is performed by interlaced scanning every two lines. First, the (1) th row, the (2) th row,... (4n + 2) rows are sequentially selected, signals are read out, signals of odd frames are read, and image signals of odd frames are output. Subsequently, the (3) th row, the (4) th row,..., The (4n + 3) th row, and the (4n + 4) th row are sequentially selected to read out the signal of the even-numbered frame and output the image signal of the even-numbered frame. .

The time interval between the odd frame and the even frame (division image frame interval) is (1/12).
0) Seconds = 8.333 milliseconds is set. The time interval (image frame interval) between odd frames and between even frames is set to (1/60) seconds = 16.667 milliseconds.

  Also, odd-numbered frames of (1), (2),..., (4n + 1), (4n + 2), and (3), (4),. The even-numbered frames on the (4n + 3) -th row and the (4n + 4) -th row have a divided image frame interval of 8.333 milliseconds as described above, and one is configured to read out a signal when one is accumulating charge. Have been. The reason that the accumulation times of the odd frames and the even frames partially overlap each other is that the interval (image frame interval) between the odd frames or the even frames is set while each accumulation time is set to 10 milliseconds. This is for setting to 16.667 milliseconds (60 Hz).

  Here, in FIG. 5, from the odd frames of the (1) th row, the (2) th row,..., The (4n + 1) th row, the (4n + 2) th row, the (3) th row, (4 ),..., (4n + 3) -th row, and (4n + 4) -th row are shown. If G in the third row is expressed as G3, G3 = (G1 + G5) / 2, G in the fourth row is expressed as G4, G4 = (G2 + G6) / 2, and R in the third row is expressed as R3, R3 = (R1 + R5) / 2, and B in the fourth row is denoted as B4, and B4 = (B2 + B6) / 2 is calculated.

  Further, in FIG. 6, from the even frames of the (3) th row, the (4) th row,..., The (4n + 3) th row, the (4n + 4) th row, the (1) th row of the even numbered frame, (2) The pixel interpolation for generating the rows,..., (4n + 1) and (4n + 2) will be described. G in the fifth row is represented as G5, G5 = (G3 + G7) / 2, G in the sixth row is represented as G6, G6 = (G4 + G8) / 2, and R in the fifth row is represented as R5, R5 = (R3 + R7) / 2, and B in the sixth row is represented as B6, which is calculated as B6 = (B4 + B8) / 2.

As shown in FIGS. 5 and 6, pixel interpolation processing of interlace progressive conversion for every two lines is performed to generate all pixel information of odd frames from odd frames and all pixel information of even frames from even frames. .
FIG. 7 shows a block diagram (flowchart) of the signal processing device according to the present embodiment. This signal processing device includes FPN (Fixed Pattern Noise) canceling means S1, gain control means S2, interlace progressive conversion means S3 for each line, demosaic processing means S4, linear matrix processing means S5, and gamma knee processing means S6. ing.

Here, the FPN canceling means S1 has a function of removing FPN (fixed pattern noise), and the gain control means S2 has a function of adjusting each gain of RGB to make it bright. Further, the interlaced progressive conversion means S3 for every two lines has a function of converting an interlaced image signal for every two lines into progressive, and the video signal is formed as a dual green signal of RGBG.
Further, the demosaic processing unit S4 has a function of converting an RGB dual green signal into a full RGB signal.
The linear matrix processing means S5 has a function of performing color correction by performing a 3 × 3 matrix operation process on the RGB input signal.
Further, the gamma / knee processing means S6 has a function of performing gamma processing and knee processing on the signal.
Note that gamma is Y = X ^0.45 , where Y is the output voltage and X is the input voltage.
, And the knee can greatly compress a signal exceeding a predetermined level.

That is, in the signal processing device according to the present embodiment, unlike the related art, interlace progressive conversion processing is performed for every two lines, and demosaic processing is subsequently performed. Thereby, an odd frame and an even frame are read out by interlaced scanning every two lines from a square grid-like pixel array equipped with a Bayer array color filter, and all the pixels of the odd frame and the even frame are interlaced progressively converted every two lines. Information is generated, and color information of all pixels is generated by demosaic processing.
According to the above-described first embodiment, as shown in FIG. 8, the image sensor adopts the interlace method for every two lines under the illumination intensity of 100 Hz (power frequency is in the range of 50 Hz). (Imaging apparatus) In the image pickup apparatus 200, the charge accumulation time of a pixel (photodiode) equipped with a Bayer array color filter is set to 10 milliseconds, the imaging frame frequency is set to 120 Hz, and the flicker is generated while being adapted to Super Hi-Vision. I try to prevent.

  That is, when the electronic shutter speed is set to 10 milliseconds in order to prevent flicker, the imaging frame interval (divided image frame interval) is (1/120) seconds = 8.333 milliseconds. , The electronic shutter period with respect to the imaging frame interval is set to 6/5, which is larger than 1.

  Therefore, in the present embodiment, even if the electronic shutter speed is set to 10 milliseconds and the divided image frame interval is set to (1/120) seconds = 8.333 milliseconds in the pixel equipped with the Bayer array color filter, Since the interlace method for every two lines is adopted, the electronic shutter period for the image frame interval (odd frames or even frames) is set to a value smaller than 1 (in the present embodiment, the interlace method for every two lines is adopted). Therefore, the flicker can be set to 6/10), so that the occurrence of flicker which occurs when imaging at 120 Hz under the illumination intensity change of 100 Hz in the power supply frequency range of 50 Hz can be prevented.

<Second embodiment>
As shown in FIG. 9, the CMOS type imaging device according to the second embodiment has a structure in which two pixels sharing a readout circuit unit of two pixels in the vertical direction are shared.

In the second embodiment, since there are many portions that overlap with the first embodiment, such portions will be briefly described as appropriate. In particular, the device configuration based on FIG. 2 and the Bayer array configuration based on FIG. 3, the pixel interpolation method for every two lines according to FIGS. 5 and 6, and the block diagram (flow chart) of the signal processing device of FIG. , And a detailed description thereof will be omitted.
That is, as shown in FIG. 9, the pixel circuit of the CMOS type imaging device according to the second embodiment is of a two-pixel sharing type, and has two photodiodes (PDs) 911-A and 911-B and two electric charges. Transfer transistors (TX) 912-A and B, floating diffusion capacitance (FD) 913, reset transistor (RST) 914, source follower amplifier (SF) 915, selection transistor (SEL) 916, pixel power supply (VDD) 917, and It comprises a pixel output section (OUT) 918.

  FIG. 10 is a time chart showing input signals of the respective transistors when signal reading is performed using the pixel circuit shown in FIG. 9 and the pixel arrangement of the Bayer array color filter shown in FIG. The number in parentheses is a number indicating on which line the pixel is arranged. For example, SEL (1, 2) and RST (1, 2) indicate input signals of SEL and RST in which two pixels in the first row and the second row share a pixel. TX represented as TX (1) and TX (2) indicate the TX of the pixels in the first row and the second row, respectively. The black bands of the accumulation time (1) and the accumulation time (2) indicate the accumulation time of the PD 911-A of the pixel in the first row and the PD 911-B of the pixel in the second row, respectively.

  In the imaging device according to the second embodiment, first, the pixels in the (1) -th row and the (2) -th row are selected by SEL (1,2), and the pixels are selected by TX (1) and TX (2). A signal is read from the PD 911-A of the pixel in the first row and the PD 911-B of the pixel in the second row (as indicated by the black band in FIG. 10, the accumulation time of the PD 911-A is started at the timing of the arrow A1, (The accumulation time of PD911-A ends at the timing of arrow C1, while the accumulation time of PD911-B starts at the timing of arrow A2, and the accumulation time of PD911-B ends at the timing of arrow C2.) Subsequently, the pixels in the (5) th row and the (6) th row are selected by SEL (5, 6), and the pixels PD911-A of the fifth row are selected in TX (5) and TX (6). A signal is read from PD911-B of the pixel in the sixth row. This process is continuously performed until the signal reading of the 4n + 1-th row and the 4n + 2-th row, and an odd-numbered frame is generated.

  Subsequently, the pixels in the (3) th and (4) th rows are selected by SEL (3,4), and the pixels PD911-A of the third row are selected in TX (3) and TX (4). A signal is read from PD911-B of the pixel in the fourth row. Subsequently, the pixels in the (7) th and (8) th rows are selected by SEL (7, 8), and the pixels PD911-A of the 7th row are selected in TX (7) and TX (8). A signal is read from PD911-B of the pixel in the eighth row. This process is continuously performed until the signal reading of the 4n + 3th row and the signal reading of the 4n + 4th row to generate an even-numbered frame.

  Note that the time interval (divided image frame interval) between the odd-numbered frame and the even-numbered frame is set to (1/120) seconds = 8.333 milliseconds.

  The method of performing interlace progressive conversion for every two lines on odd-numbered frames is the same as in FIG. 5, and the method of performing interlace progressive conversion on every two lines for even-numbered frames is the same as in FIG. The signal processing device used in the device of the second embodiment has the same configuration as that of FIG.

  In the second embodiment, n is set from 0 to (4320/4) −1 = 11079.

  Thus, in the second embodiment, it is possible to prevent the occurrence of flicker when imaging at 120 Hz under a change in illumination intensity of 100 Hz in a power supply frequency range of 50 Hz.

<Third embodiment>
As shown in FIG. 11, the CMOS imaging device according to the third embodiment has a four-pixel sharing structure in which two pixel readout circuit units are shared in the vertical direction and two pixel readout circuit units in the horizontal direction. Have

Note that, in the third embodiment, since there are many portions that overlap with the first embodiment, such portions will be briefly described as appropriate. In particular, since the configuration of the device based on FIG. 2 and the configuration of the Bayer array based on FIG. 3 and the block diagram (flow chart) of the signal processing device of FIG. 7 are substantially the same, detailed description thereof will be omitted.
That is, as shown in FIG. 11, the pixel circuit 1100 includes four photodiodes (PD) 1111-A to D, four charge transfer transistors (TX) 1112-A to D, a floating diffusion capacitance (FD) 1113, It comprises a reset transistor (RST) 1114, a source follower amplifier (amplifying transistor: SF) 1115, a selection transistor (SEL) 1116, a pixel power supply (VDD) 1117, and a pixel output (OUT) 1118.

FIG. 12 is a time chart showing input signals of each transistor when signal reading is performed using the pixel circuit shown in FIG. 11 and the pixel arrangement of the Bayer array color filter shown in FIG. The number in parentheses indicates the number of the line and the arrangement of the pixel. For example, SEL (1A, 1B, 2C, 2D) and RST (1A, 1B, 2C, 2D) are described in the first row, A and B, and in the second row, C and D. 4 shows input signals of SEL and RST in which four pixels are shared. TX represented as TX (1A), TX (1B), TX (2C), TX (2D) is the TX of the A-th and B-th pixels in the first row and the C-th and D-th pixels in the second row, respectively. Is shown. Storage time (1A (period between arrows A1 and C1)), storage time (1B (period between arrows A2 and C2)), storage time (2C (period between arrows A3 and C3)), storage time (2D ( The black bands in the periods of arrows A4 and C4) indicate the accumulation times of the PDs 1111-A to D of the A-th and B-th pixels in the first row and the C-th and D-th pixels in the second row, respectively.

In the imaging device according to the third embodiment, first, the A-th and B-th pixels in the (1) -th row and the C-th and D-th pixels in the (2) -th row are selected by SEL (1A, 1B, 2C, 2D). Then, the signal is read out from the PDs 1111-A and B of the A-th and B-th pixels in the first row by TX (1A) and TX (1B), and then the signal is read out by the TX (2C) and TX (2D). The signal is read from the PDs 1111-C and D of the C-th and D-th pixels in the row.
Subsequently, the A-th and B-th pixels in the (5) th row and the C-th and D-th pixels in the (6) th row are selected by SEL (5A, 5B, 6C, 6D), and TX (5A) and TX ( 5B) The signal is read from the PD1111-A, B of the Ath and Bth pixels in the fifth row, and the PD1111-C of the Cth and Dth pixels in the sixth row in TX (6C) and TX (6D). , D.
This process is continuously performed until the signal reading of the 4n + 1-th row and the 4n + 2-th row, and an odd-numbered frame is generated.

Subsequently, the A-th and B-th pixels in the (3) -th row and the C-th and D-th pixels in the (4) -th row are selected by SEL (3A, 3B, 4C, 4D), and TX (3A) and TX ( 3B), the signals are read from the PDs 1111-A and B of the A-th and B-th pixels in the third row, and the PDs 1111-C of the C-th and D-th pixels in the fourth row in TX (4C) and TX (4D). , D.
Subsequently, the Ath and Bth pixels in the (7) th row and the Cth and Dth pixels in the (8) th row are selected by SEL (7A, 7B, 8C, 8D), and TX (7A) and TX ( 7B) The signal is read from the PDs 1111-A and B of the Ath and Bth pixels in the seventh row, and the PD1111-C of the Cth and Dth pixels in the eighth row in TX (8C) and TX (8D). , D.
This process is continuously performed until the signal reading of the 4n + 3th row and the signal reading of the 4n + 4th row to generate an even-numbered frame.

  Note that the time interval (divided image frame interval) between the odd-numbered frame and the even-numbered frame is set to (1/120) seconds = 8.333 milliseconds.

  The method of performing interlace progressive conversion for every two lines on odd-numbered frames is the same as in FIG. 5, and the method of performing interlace progressive conversion on every two lines for even-numbered frames is the same as in FIG. The signal processing device used in the device of the third embodiment has the same configuration as that of FIG.

  In the third embodiment, n is set from 0 to (4320/4) -1 = 11079.

  Thus, in the third embodiment, it is possible to prevent the occurrence of flicker when imaging at 120 Hz under the illumination intensity change of 100 Hz in the power frequency range of 50 Hz.

Furthermore, the imaging device, the imaging method, and the image frame readout control circuit of the present invention are not limited to those of the above-described embodiment, but may adopt various other aspects. For example, in the above embodiment, among the shared type elements, an example of a two-pixel shared type element shared by two pixels and a four-pixel shared type element shared by four pixels have been described. Other than the above, signal readout can be performed using an element shared by a plurality of pixels.
Also, the method of pixel interpolation is not limited to those shown in FIGS. 5 and 6, and other various methods can be used.

  In the above embodiment, the plurality of pixels constituting the imaging apparatus are physically set to either 7680 pixels in the X direction and 4320 pixels in the Y direction, or 3840 pixels in the X direction and 2160 pixels in the Y direction. However, instead of this, a plurality of pixels are subjected to pixel interpolation processing using hardware or software or the like, and 7680 pixels in the X direction and 4320 pixels in the Y direction, or in the X direction. Even if the number of pixels is expanded (increased) or reduced (decreased) so as to be 3840 pixels or any of 2160 pixels in the Y direction, the same effect as in the above embodiment can be obtained.

  In the above-described embodiment, the image frame interval is set to 1/120 second = 8.333 milliseconds. Alternatively, the image frame interval may be set to 1/120 second × 1001/1000 = 8.342 milliseconds. The effects similar to those of the embodiment can be obtained. Further, in the above embodiment, the frame frequency is set to 120 Hz. However, when the frame frequency is set to 120 × 1000/1001 = 1119.88 Hz, substantially the same effect as that of the above embodiment can be obtained.

  Furthermore, it is possible to mount a global shutter function (global shutter transistor). In this case, it is possible to perform a shutter operation simultaneously for all pixels (actually, odd frame pixels and even frame pixels simultaneously). And all pixels can be read simultaneously. As a result, image distortion can be reduced particularly for a fast-moving subject.

Further, in the above embodiment, a Bayer color filter having a 2 × 2 square lattice is used as a color filter. However, a Bayer color filter having a 2 × 2 square lattice and one of the Bayer color filters is used instead. One G may be replaced by W (white), a two-by-two square lattice may be used, and both Gs of the Bayer color filter may be replaced by W (white). One of the color filters G may be replaced with an NIR (near infrared transmission filter).
In addition, a so-called honeycomb type (different from a square lattice) color filter which reads data in an interlaced manner every two lines may be employed, or a 6 × 6 square lattice-shaped color filter is read out in an interlaced manner every six lines. A color filter called FUJIFILM X-Pro1 (manufactured by FUJIFILM Corporation) may be used, or a color filter called a so-called color sequential method may be used.

Although the configuration is such that the divided image frame is read out every two lines, the configuration may be such that the divided image frame is read out every three or more lines, and the number of pixel lines that are covered when a color filter is mounted on the pixel is determined. The interlace method is used for each of the number of matching lines.
In the above embodiment, a row selection circuit unit (Y-direction scanning unit) for selecting and driving the address of the Y-row, and a column parallel reading circuit unit (X-direction scanning unit) for reading a signal for each X column are used. In place of this, a column selection circuit section (X-direction scanning section) for selecting and driving an address of an X column and a row parallel reading circuit section for reading a signal every Y rows are performed. (Y direction scanning unit) may be used to perform image frame read control.

400, 900, 1100 Pixel circuit 411, 911-A, B, 1111-A-D Photodiode (PD)
412, 912-A, B, 1112-A-D Charge transfer transistor (TX)
413, 913, 1113 Floating diffusion capacitance (FD)
414, 914, 1114 Reset transistor (RST)
415, 915, 1115 Source follower amplifier (SF)
416, 916, 1116 Select transistor (SEL)
417, 917, 1117 Pixel power supply (VDD)
418, 918, 1118 Pixel output unit (OUT)
Reference Signs List 200 Image pickup device 201 Pixel array 202 Y-direction scanning unit 203 X-direction scanning unit 204 Timing generator 205 Output circuit S1 FPN canceling unit S2 Gain control unit S3 Interlace every two lines
Progressive conversion means S4 Demosaic processing means S5 Linear matrix processing means S6 Gamma knee processing means

Claims (7)

For each color filter to be mounted, a unitary filter corresponding to each pixel is combined and arranged in a regular lattice, a photoelectric conversion unit that generates charges in accordance with incident light,
A row or column selection circuit for selecting and driving one of the Y row and the X column for the photoelectric conversion unit, and a column or a row for reading a signal for each of the other of the Y row and the X column An image frame readout control unit including a parallel readout circuit unit,
The pixels of the photoelectric conversion unit are configured to be set to either 7680 pixels in the X direction and 4320 pixels in the Y direction, or 3840 pixels in the X direction and 2160 pixels in the Y direction,
The image frame readout control unit uses a non-progressive method, and controls to output a divided image frame for each line number equal to the corresponding line number of a pixel to which one color filter is attached, and An image pickup apparatus, wherein a divided image frame interval is set to 8.333 ms or 8.342 ms, and one charge accumulation time of each pixel in the photoelectric conversion unit is set to 10 ms. .   2. The image pickup apparatus according to claim 1, wherein the color filter is a Bayer color filter, and the number of lines equal to the number of corresponding lines of pixels to which the color filter is attached is two.   The imaging apparatus according to claim 1, wherein the image frame reading control unit is configured to control so that one charge accumulation time of each pixel with respect to an image frame interval is 6/10. The photoelectric conversion unit, an imaging apparatus according to any one of claims 1 to 3, characterized in that it is configured to be shared pixel in a plurality of pixels arranged in a positive direction grid pattern. For each of the mounted color filters, a unit filter corresponding to each pixel is combined so that a pixel circuit arranged in a regular lattice form generates charges corresponding to light incident on each of the plurality of pixels. To perform photoelectric conversion,
An imaging method by a CMOS imaging device that performs image frame reading in a predetermined order by designating an address of a Y row and an address of an X column for a pixel that performs the photoelectric conversion,
The pixels that perform the photoelectric conversion are controlled to be set to either 7680 pixels in the X direction and 4320 pixels in the Y direction, or 3840 pixels in the X direction and 2160 pixels in the Y direction,
Image frame reading, using a non-progressive method, control to read the divided image frame for each line number equal to the number of corresponding lines of the pixel to which the one color filter is mounted,
And sets the divided image frame interval to one of 8.333 ms or 8.342 ms, to control a charge accumulation time of each pixel in the photoelectric conversion unit so as to set to 10 ms A characteristic imaging method. An image frame read control signal for a photoelectric conversion unit that generates charges in accordance with incident light, in which unit filters corresponding to respective pixels are combined and arranged in a regular lattice for each color filter to be mounted. An image frame readout control circuit that outputs
A row selection circuit for selecting an address in the Y row and driving pixels included in the Y row; and a column parallel readout circuit for selecting an address in the X column and reading a signal from a pixel included in the X column. ,
The pixels of the photoelectric conversion unit are controlled to be set as either 7680 pixels in the X direction and 4320 pixels in the Y direction, or 3840 pixels in the X direction and 2160 pixels in the Y direction ,
Reading the images frames, using a non-progressive system, one of the controlled so color filter reads the divided image frame every equal number of lines to a corresponding number of lines of pixels to be attached, and, in the divided image frame The interval is set to either 8.333 milliseconds or 8.342 milliseconds, and corresponds to the plurality of pixels so that the accumulation time of each pixel in the photoelectric conversion unit can be set to 10 milliseconds. An image frame readout control circuit for controlling at least an accumulation start instruction signal and an accumulation end instruction signal so as to be output to the photoelectric conversion unit in a predetermined order. For each mounted color filter, select a photoelectric conversion unit that generates charges in accordance with incident light and an address of Y row, where unit filters corresponding to each pixel are combined and arranged in a regular grid. A row selection circuit for driving pixels included in the Y row of the photoelectric conversion unit; and a column readout for selecting an address in the X column and reading a signal from a pixel included in the X column of the photoelectric conversion unit. A signal processing device for reading a divided image frame from the photoelectric conversion unit by a divided image frame signal output from the row selection circuit unit or the column readout circuit unit, comprising:
The pixels of the photoelectric conversion unit are controlled to be set as either 7680 pixels in the X direction and 4320 pixels in the Y direction, or 3840 pixels in the X direction and 2160 pixels in the Y direction,
The reading of the divided image frame is controlled using a non-progressive method so as to configure the divided image frame for each number of lines equal to the number of corresponding lines to which one color filter is attached,
The plurality of image frames are set so that the interval between the divided image frames is set to 8.333 ms or 8.342 ms, and the accumulation time of each pixel in the photoelectric conversion unit can be set to 10 ms. At least an accumulation start instruction signal and an accumulation end instruction signal corresponding to a pixel are output from the row selection circuit unit or the column readout circuit unit to the photoelectric conversion unit in a predetermined order toward the divided image frame signal. Control and
In a divided image frame in which a signal is present for each line number equal to the corresponding line number of the pixel to which the one color filter is attached, for each of a plurality of lines that generate pixel signals by pixel interpolation for each line number equal to the corresponding line number A signal processing device comprising an interlace progressive conversion unit.

Images