JP7289635B2 - IMAGING DEVICE AND METHOD OF CONTROLLING IMAGING DEVICE - Google Patents

Description

The present invention relates to an imaging device and an imaging device control method.

In recent years, image sensors are required to have higher image resolution and higher frame rate for moving images. In response to such a demand, in Patent Document 1, an image compression unit is provided in an imaging device to reduce the bandwidth of a transmission path that becomes a bottleneck.

Image compression methods include intra-frame compression in which only reference is made within a frame, and inter-frame compression in which compression is performed by referring to images between frames. In intra-frame compression, the compression rate is determined depending on the object in the frame. On the other hand, inter-frame compression is advantageous when there is little movement, since the compression rate depends greatly on the movement of the subject between frames. For example, in Patent Literature 2, a frame memory is used to temporarily store images other than the frame being captured, and inter-frame compression is achieved by referring to the images.

On the other hand, Patent Document 3 discloses a solid-state imaging device having a 1-bit analog-to-digital converter and a counter for each pixel. The solid-state image sensor performs analog-to-digital conversion on the signal of the light-receiving element for each pixel, so it is possible to eliminate the trade-off between the number of scanning lines and the frame rate of the solid-state image sensor that performs analog-to-digital conversion for each column. is. In this method, the light receiving element is reset every time a certain amount of electric charge is accumulated in the light receiving element, so that the light receiving element is not saturated. The amount of light that can be detected is determined by the upper limit of a counter that counts the number of pulses output when the voltage of the storage capacitor matches the reference voltage.

JP 2014-103543 A JP 2006-295521 A JP 2015-173432 A

However, the imaging device having a compression unit as in Patent Document 1 has a problem that it is difficult to perform complicated compression such as the one described in Patent Document 2 using a frame memory.

SUMMARY OF THE INVENTION In view of such problems, it is an object of the present invention to provide an image pickup apparatus and a control method for the image pickup apparatus that can improve the compression rate while suppressing the scale.

The imaging apparatus of the present invention includes photoelectric conversion means for generating a signal by photoelectric conversion, and in a first frame, counted values are generated based on the signal generated by the photoelectric conversion means, and the first frame In a second frame different from the above , based on the signal generated by the photoelectric conversion means, generating means for generating a count value obtained by inverting the sign of the count value of the previous frame, and and compression means for compressing the count value generated by the generation means.

ADVANTAGE OF THE INVENTION According to this invention, a compression ratio can be improved, suppressing a scale.

4 is a diagram showing a configuration example of a pixel; FIG. It is a figure which shows the structural example of a counter. It is a figure which shows the structural example of a counter. 4 is a timing chart showing driving of pixels; It is a figure which shows the structural example of a solid-state image sensor. It is a figure which shows the structural example of a compression part. 4 is a flowchart showing processing of a compression unit; It is a figure which shows the structural example of an imaging system. It is a figure which shows the structural example of a counter. It is a figure which shows the structural example of a compression part. 4 is a timing chart showing driving of pixels; It is a figure which shows the structural example of a solid-state image sensor. It is a figure which shows the structural example of a solid-state image sensor. 4 is a timing chart showing the operation of the frame memory; FIG. 4 is a diagram showing memory areas; It is a figure which shows the structural example of a compression part.

Preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. The embodiments described below are merely examples, and the present invention is not limited to the configurations described in the embodiments.

(First embodiment)
FIG. 1 is a diagram showing a configuration example of a pixel 100 according to the first embodiment of the invention. The pixel 100 has an avalanche photodiode (hereinafter referred to as APD) 101 , a quench resistor 102 , a waveform shaping circuit 103 and a counter 104 .

The APD 101 is connected through a quench resistor 102 to the node of the reverse bias voltage V _APD . The APD 101 generates charges due to avalanche multiplication when photons are incident thereon, and discharges the generated charges to the node of the reverse bias voltage V _APD via the quench resistor 102 . The APD 101 is photoelectric conversion means, generates a signal by photoelectric conversion, and outputs the generated signal to the waveform shaping circuit 103 .

A waveform shaping circuit 103 is connected to the interconnection point between the APD 101 and the quench resistor 102 . The potential of the interconnection point between the APD 101 and the quench resistor 102 changes due to the generation and discharge of charges in response to the incidence of photons on the APD 101 . The waveform shaping circuit 103 generates a voltage pulse PLS by amplifying and edge detecting changes in potential at the interconnection point between the APD 101 and the quench resistor 102 . Thus, the APD 101, the quench resistor 102, and the waveform shaping circuit 103 function as a 1-bit analog-to-digital converter (light receiving section) by converting the presence or absence of incident photons into voltage pulses.

The counter 104 is a counter that counts the number of voltage pulses PLS generated by the waveform shaping circuit 103, and by outputting the counting result, outputs the pixel value during the exposure period in multiple bits. The counter 104 performs resetting of the counter 104, counting period, and sign-inverting resetting operation of the counter according to the driving signals INV_RST and ZERO_RST input to the pixel 100. FIG.

FIG. 2 is a diagram showing a configuration example of the counter 104. As shown in FIG. The counter 104 includes a flip-flop 200 that holds data, an addition unit 201 that performs counting, a selection unit 202 that determines the type of initial value based on the type of reset signal, and a sign inversion unit 203 that performs sign inversion. have.

The sign inversion unit 203 adds “1” to the logically inverted value of the output value CNT of the flip-flop 200 , thereby outputting a value in which the sign of the output value CNT of the flip-flop 200 is inverted. That is, the sign inverter 203 outputs the two's complement of the output value CNT of the flip-flop 200 .

Selecting section 202 outputs “0” to adding section 201 when signal ZERO_RST is at high level. Selecting section 202 also outputs the output value of sign inverting section 203 to adding section 201 when signal INV_RST is at high level. Selecting section 202 outputs output value CNT of flip-flop 200 to adding section 201 when signals ZERO_RST and INV_RST are at low level.

The adding section 201 adds the voltage pulse PLS to the output value of the selecting section 202 and outputs the addition result to the flip-flop 200 . The flip-flop 200 holds the addition result of the addition section 201 and outputs the held value as the count value CNT. Flip-flop 200 is initialized to an initial value of 0 by an asynchronous reset signal asynchronously to the clock signal.

By setting the signal ZERO_RST to high level, it is possible to count the voltage pulses PLD for a predetermined counting period. Also, by setting the signal INV_RST to high level, it is possible to output the difference between the count value in the current counting period and the count value in the previous counting period.

Counter 104 is a synchronous counter that synchronizes with a clock signal. The clock signal and the asynchronous reset signal are signals common to the entire solid-state imaging device. The counter 104 can calculate the pixel value difference between frames without using a frame memory. Note that the counter 104 may be an asynchronous counter. Since the asynchronous counter can perform counting without using an adder such as a half adder or a full adder, further cost reduction is possible.

FIG. 3 is a diagram showing a configuration example of the asynchronous counter 104. As shown in FIG. The counter 104 has a flip-flop 300 that holds data, a sign inversion controller 301 that outputs a sign inversion control signal for each bit, and a logical product (AND) element 302 . For ease of explanation, the configuration of the 4-bit asynchronous counter 104 will be explained.

When the input signal to the terminal PRST is low level, the flip-flop 300 sets the output terminal Q to high level and the output terminal /Q to low level. Further, flip-flop 300 does not change the levels of output terminals Q and /Q when the input signal to terminal PRST is at high level. When the input signal to the terminal RST is low level, the flip-flop 300 sets the output terminal Q to low level and the output terminal /Q to high level. Further, flip-flop 300 does not change the levels of output terminals Q and /Q when the input signal to terminal RST is at a high level. Further, the flip-flop 300 outputs the positive logic value of the signal input to the input terminal D from the output terminal Q in synchronization with the rising edge of the signal input to the clock terminal, and the signal input to the input terminal D is output from the output terminal /Q.

The sign inversion control section 301 outputs high level to the terminals PRST and RST of the flip-flop 300 when the signal INV_RST is at low level. Further, when the signal INV_RST is at a high level, the sign inversion control section 301 outputs the signal of the output terminal Q of the flip-flop 300 to the terminal PRST, and outputs the logically inverted signal of the output terminal Q of the flip-flop 300 to the AND element 302. output to

The AND element 302 outputs a low level to the terminal RST of the flip-flop 300 when the signal ZERO_RST is at high level, and outputs the output of the sign inversion control section 301 to the terminal RST of the flip-flop 300 when the signal ZERO_RST is at low level. Output a signal.

Next, the operation of the asynchronous counter 104 will be explained. The flip-flop 300 has an initial value of 0 at the output terminal Q and an initial value of 1 at the output terminal /Q. When the flip-flop 300 constituting the 0th bit of the counter 104 detects the rise of the voltage pulse PLS, it outputs the value of the input terminal D from the output terminal Q, and outputs the logically inverted value of the value of the input terminal D to the output terminal /Q. Output from Each of the flip-flops constituting the first and subsequent bits of the counter 104 detects the rise of the signal at the output terminal /Q of the preceding flip-flop, outputs the value of its own input terminal D from the output terminal Q, and outputs the value of its own input terminal D from the output terminal Q. A logically inverted value of the value of the input terminal D is output from the output terminal /Q. An output terminal Q of each flip-flop 300 outputs, for example, a 4-bit count value CNT.

After the predetermined period has passed, AND element 302 performs a reset operation based on the output of control section 505 . When the signal ZERO_RST becomes high level, the value of the output terminal Q of each flip-flop 300 is reset to zero. At this time, in the case of a simple inverted signal, it is expressed in 1's complement and not expressed in 2's complement, so there is a case where the adder cannot be expressed with a simple configuration when used for signal processing. Therefore, in order to use the 2's complement representation, a signal is input from the control unit 505 so that the rising signal is generated only once in the voltage pulse PLS after the signal INV_RST is input. By controlling in this manner, the value obtained by adding 1 to the inverted signal of each bit becomes the initial value, so that the value initialized by the signal INV_RST can be expressed in two's complement.

Further, when the signal INV_RST becomes high level, the sign inversion control section 301 outputs the logically inverted value of the value of the output terminal Q of each flip-flop 300 to the terminals PRST and RST of each flip-flop 300 as a reset value.

FIG. 4 is a timing chart showing how one pixel 100 is driven. An optical image is converted into a digital signal by driving a plurality of pixels 100 in parallel. APD is a potential waveform generated by the APD 101 and the quench resistor 102 and indicates the input potential waveform of the waveform shaping circuit 103 . A voltage pulse PLS is the output voltage of the waveform shaping circuit 103 . Signals CNT_RST and INV_EN are generated by control unit 505 of FIG. Control section 505 generates signals ZERO_RST and INV_RST based on signals CNT_RST and INV_EN.

Specifically, if the signal CNT_RST is high level when the signal INV_EN is low level, the signal ZERO_RST becomes high level. Also, if the signal CNT_RST is high level when the signal INV_EN is high level, the signal INV_RST becomes high level.

A count value CNT represents the output value of the counter 104 . The counter 104 performs the above-described counting and initialization operations with the period during which the signal CNT_RST is at low level as the imaging period. Note that the counter 104 counts when the signal CNT_RST is at low level, but may count according to a separate enable signal.

READ_EN represents the timing of outputting the output value of the counter 104 to the vertical transmission line 508 (FIG. 5), and is issued at the same timing as the signal CNT_RST. Although the signal READ_EN is issued at the same timing as the signal CNT_RST, it is not limited to this. Specifically, the transfer timing to the vertical transmission line 508 may be arbitrarily set by having separate readout signals and reset signals and shifting the timing of their issuance.

Next, a method for driving the pixel 100 will be described. At timing t400, the signal CNT_RST becomes high level while the signal INV_EN is at low level, so the signal ZERO_RST becomes high level. When the signal ZERO_RST becomes high level, the counter 104 clears the count value CNT to 0 in synchronization with the clock signal. Actually, since the count value CNT is reflected in synchronization with the clock signal, a delay occurs in the count value CNT, but the description will be made assuming substantially the same timing.

At timing t401, the signal ZERO_RST becomes low level, exposure starts, and the imaging period starts. Further, when the signal ZERO_RST becomes low level, the signal INV_EN becomes high level, and when the signal CNT_RST becomes high level next time, not the signal ZERO_RST but the signal INV_RST becomes high level.

At timing t402, a photon is incident on the APD 101, so that an avalanche-multiplied charge is generated in the APD 101, and the potential of the APD 101 changes. Since the APD 101 discharges electric charges via the quench resistor 102, it takes time for the APD 101 to reach a constant potential again. A waveform shaping circuit 103 detects a falling edge of the potential of the APD 101 and generates a short voltage pulse PLS. This voltage pulse PLS is input to the counter 104 . The counter 104 counts the voltage pulses PLS, and the count value CNT becomes one.

During timings t401 to t403, the counter 104 continues to increment the count value CNT each time the voltage pulse PLS is generated. At timings t401 to t403, the voltage pulse PLS is generated five times, and the count value CNT is five. The imaging period is from timing t401 to t403.

At timing t403, the count value CNT is 5 and the signals CNT_RST and READ_EN go high. When the signal READ_EN becomes high level, the count value CNT is transmitted to the vertical transmission path 508 and the imaging period is completed. The count value CNT of 5 corresponds to the pixel value of the frame during the imaging period from timing t401 to t403. Frames in the imaging period from timing t401 to t403 are frames for intra-frame compression.

Further, at timing t403, since the signal INV_EN is at a high level, the signal INV_RST is at a high level, and the value of the count value CNT becomes -5 which is the sign-inverted value of 5.

At timing t404, the signals INV_RST, INV_EN, and READ_EN become low level, exposure starts, and the imaging period of the next frame starts. During timings t404 to t405, similarly to timings t401 to t403, counter 104 continues to increment count value CNT each time voltage pulse PLS is generated. Since four voltage pulses PLS are generated between timings t404 and t405, the counter 104 increments the count value CNT of -5 four times and outputs the count value CNT of -1.

At timing t405, while the count value CNT is -1, the signals CNT_RST and READ_EN go high as at timing t403. When the signal READ_EN becomes high level, the count value CNT of -1 is transmitted to the vertical transmission path 508, and the imaging period is completed. The count value CNT of −1 is a value obtained by subtracting the number of voltage pulses PLS generated in the frame of the previous imaging period from the number of voltage pulses PLS generated in the frame of the current imaging period. That is, the count value CNT of -1 is the difference between the number of occurrences of voltage pulses PLS between the current frame and the previous frame. Frames in the imaging period from timing t404 to t405 are frames for inter-frame compression.

At timing t405, since the signal INV_EN is at low level, the signal ZERO_RST becomes high level and the count value CNT becomes zero.

At timing t406, the signals CNT_RST, ZERO_RST, and READ_EN become low level, and the signal INV_EN becomes high level. During timings t406 to t407, counter 104 continues to increment count value CNT each time voltage pulse PLS is generated, similarly to timings t401 to t403. Since four voltage pulses PLS are generated at timings t406 to t407, the counter 104 increments the count value CNT of 0 four times and outputs a count value CNT of 4. FIG. Frames in the imaging period from timing t406 to t407 are frames for intra-frame compression.

At timings t407 to t408, the signals CNT_RST, INV_EN, ZERO_RST and READ_EN are controlled and the count value CNT is transmitted to the vertical transmission path 508, similarly to the timings t403 to t404.

Thereafter, the pixel 100 alternately repeats operations similar to timings t404 to t406 and operations similar to timings t406 to t408. By operating in this manner, the counter 104 counts the number of voltage pulses PLS generated and outputs a count value CNT. The counter 104 alternately outputs the number of voltage pulses PLS generated during the imaging period of the current frame and the difference between the number of voltage pulses PLS generated between the current frame and the previous frame as the count value CNT. This is equivalent to using a frame memory to output the difference value of the same pixel 100 between frames.

FIG. 5 is a diagram showing a configuration example of the solid-state imaging device 500. As shown in FIG. The solid-state imaging device 500 is an imaging device, and includes the plurality of pixels 100 described above, a vertical selection circuit 507, an output switch 501, a horizontal selection circuit 502, an output switch 503, a timing generator (TG) 504, and a control circuit. It has a section 505 and a compression section 506 . A plurality of pixels 100 are arranged in a two-dimensional matrix.

The TG 504 generates timing signals for an imaging period, a transfer period, etc. based on an internal counter, and outputs the timing signals to the vertical selection circuit 507 , the control section 505 and the horizontal selection circuit 502 .

The control unit 505 outputs signals INV_RST, ZERO_RST, etc. to the pixels 100 based on the imaging period. When the signals INV_RST and ZERO_RST are set to the same timing as the readout control signal READ_EN, the control unit 505 controls to output the signals INV_RST and ZERO_RST to each pixel 100 in order.

The vertical selection circuit 507 outputs a high-level READ_EN to the output switch 501 at the end of the imaging period. The output switch 501 transmits the count value CNT output by the pixel 100 to the vertical transmission line 508 when a high level READ_EN is input. A plurality of vertical transmission paths 508 are provided for each column of pixels 100 . The pixels 100 in each column sequentially output the count value CNT to the vertical transmission line 508 via the output switch 501 .

Although the signals INV_RST and ZERO_RST have been described as different signals, they are not limited to this. For example, signals INV_EN and CNT_RST may be input signals to pixel 100 to generate signals INV_RST and ZERO_RST within pixel 100 . Also, the pixel 100 may input at least the signals INV_RST and ZERO_RST in a binary-encoded state and use them as reset selection signals.

A horizontal selection circuit 502 controls an output switch 503 . A plurality of output switches 503 are provided for each column of pixels 100 . The output switch 503 of each column in turn connects the vertical transmission line 508 of each column to the horizontal transmission line 509 .

The compression unit 506 sequentially compresses the count values CNT on the horizontal transmission line 509 and outputs the compressed data to the outside of the solid-state imaging device 500 . The solid-state imaging device 500 compresses the count values CNT of all the pixels 100 for each frame of each imaging period in FIG. 4 and outputs the compressed data.

FIG. 6 is a block diagram showing a configuration example of the compression unit 506. As shown in FIG. The compression unit 506 has a quantization unit 600 , an entropy coding unit 601 , a code amount control unit 602 and a code amount measurement unit 603 .

The quantization unit 600 receives the count value CNT of the pixel 100 in encoding block units, calculates the difference value between the count values CNT of the pixels 100 adjacent in the encoding block in the horizontal direction, and quantizes the difference value. , to generate the quantized data.

The entropy coding unit 601 assigns a code to each quantized data based on the appearance frequency of the input quantized data (symbols) to generate the coded data 604 . The entropy coding scheme is, for example, Golomb coding or Huffman coding, but is not limited thereto.

The code amount measurement unit 603 measures the amount of encoded data generated by the entropy encoding unit 601 for each encoding block. A code amount control unit 602 controls the quantization step of the quantization unit 600 based on the encoded data amount measured by the code amount measurement unit 603 .

FIG. 7 is a flow chart showing the processing of the compression unit 506. As shown in FIG. When the compression process is started, the quantization unit 600 inputs the count value CNT of the pixels 100 in units of encoding blocks.

First, in step S701, the quantization unit 600 calculates a difference value between count values CNT of horizontally adjacent pixels 100 in an encoding block.

Next, in step S702, the quantization unit 600 quantizes the difference value calculated in step S701 using a predetermined value. The predetermined value is a parameter that determines the quantization step, and is dynamically determined by the code amount control section 602, which will be described later. Henceforth, a predetermined value is called QP (Quantization Parameter: Quantization Parameter). Quantization section 600 performs quantization by dividing the difference value by QP and rounding off the fractional part of the division result.

Next, in step S<b>703 , the entropy coding unit 601 assigns codes to the quantized difference values to generate coded data 604 . The code amount measurement unit 603 measures the amount of encoded data generated by the entropy encoding unit 601 for each encoding block.

Next, in step S704, the code amount control unit 602 determines whether or not the encoded data amount for each encoding block is within the target encoded data amount. As the target encoded data amount, an encoded data amount that fits within the output band of the solid-state imaging device 500 is set. If the encoded data amount for each encoding block does not fall within the target encoded data amount, the code amount control unit 602 proceeds to step S705, and the encoded data amount for each encoding block reaches the target encoded data amount. , the process proceeds to step S706.

In step S705, the code amount control unit 602 increases or decreases the QP according to the difference between the encoded data amount for each encoding block and the target encoded data amount, thereby increasing or decreasing the encoded data amount for each encoding block. is within the target encoded data amount. After that, the code amount control unit 602 returns to step S702 and repeats the above processing.

In step S706, the code amount control unit 602 transfers the QP used for quantization of each encoded block, the code allocation information (encoding parameter) of the entropy encoding unit 601, and the compression flag to the encoded data 604. Correspond and output. The compression flag is information indicating intra-frame compression or inter-frame compression. In the frames during the imaging period from timing t401 to t403 in FIG. 4, the compression flag is information indicating intra-frame compression. The compression flag is information indicating inter-frame compression for frames in the imaging period from timing t404 to t405 in FIG. In the frames during the imaging period from timing t406 to t407 in FIG. 4, the compression flag is information indicating intra-frame compression.

As described above, the compression unit 506 can perform intra-frame compression on frames during the imaging period from timings t401 to t403 and frames during the imaging period from timings t406 to t407. Further, the compression unit 506 can perform inter-frame compression without using a DRAM or SRAM used as a frame memory in frames during the imaging period from timing t404 to t405.

In general, inter-frame compression can achieve a high compression rate when there is no inter-frame motion vector, and is well suited for high-frame-rate readout, which tends to impose a strain on the bandwidth of transmission channels. Further, when the number of pixels in one frame to be read is quite large and the frame rate is low, intra-frame compression is well suited. Therefore, the solid-state imaging device 500 may control whether or not to use the functions of the present embodiment based on the readout rate.

FIG. 8 is a diagram showing a configuration example of an imaging system 800. As shown in FIG. The imaging system 800 has an optical system unit 801 , an expansion section 802 , a capture section 803 , a digital signal processing section 804 , and an external recording device 805 in addition to the solid-state imaging device 500 described above.

The optical system unit 801 has an optical lens group including a focusing lens for adjusting focus, a shutter, an aperture, a lens control section, and the like, and forms an optical image on the solid-state imaging device 500 .

The solid-state imaging device 500 is the solid-state imaging device 500 in FIG. 5, photoelectrically converts the image formed by the optical system unit 801, compresses the count value CNT of the pixel 100, and sends the encoded data to the decompression unit 802. Output. Encoded data is associated with a QP, an encoding parameter, and a compression flag.

The decompression unit 802 performs decompression processing on the encoded data based on the DP, the encoding parameter, and the compression flag associated with the encoded data, and outputs decompression result data to the capture unit 803 . When the compression flag is information indicating intra-frame compression, the decompression unit 802 outputs the decompression result of the current frame to an internal storage device such as a DRAM. Further, when the compression flag is information indicating inter-frame compression, the decompression unit 802 reads the decompression result of the previous frame stored in the internal storage device, performs inter-frame reference, and decompresses the current frame. conduct.

The capture unit 803 determines the effective period and type of the pixel signal of each frame output from the decompression unit 802 and outputs the result to the digital signal processing unit 804 . The digital signal processing unit 804 performs digital signal processing such as white balance correction, synchronization processing, and moving image encoding processing on the pixel signal of each frame, and outputs the image to the external recording device 805 . An external recording device 805 records the image output from the digital signal processing unit 804 on a memory card.

As described above, the solid-state imaging device 500 counts the number of voltage pulses PLS generated in each pixel 100 as the count value CNT in frames during the imaging period from timing t401 to t403, and generates encoded data by intra-frame compression. . In addition, the solid-state imaging device 500 counts the difference in the number of voltage pulses PLS generated in each pixel 100 between the current frame and the previous frame as a count value CNT in the frames during the imaging period from timing t404 to t405, and performs inter-frame compression. Generate encoded data.

The solid-state imaging device 500 alternately repeats intra-frame compressed frames and inter-frame compressed frames. Inter-frame compression can be implemented without using frame memory. By performing inter-frame compression, the compression rate is increased and the amount of encoded data can be reduced. In addition, subject-dependent variation in compression rate is reduced. Further, in the inter-frame compression, the count value CNT is close to 0, so the switching current of the CMOS transistor is reduced.

The solid-state imaging device 500 has a 1-bit analog-to-digital converter and a counter 104 . A 1-bit analog-to-digital converter has an APD 101 , a quench resistor 102 and a waveform shaping circuit 103 . The solid-state imaging device 500 is capable of inter-frame compression with a small circuit scale without using a frame memory.

As shown in FIG. 4, the counter 104 generates a count value CNT based on the signal generated by the APD 101 in even frames. In odd-numbered frames, the counter 104 generates a count value CNT obtained by reversing the positive/negative sign of the count value CNT of the previous frame based on the signal generated by the APD 101 . Even-numbered frames and odd-numbered frames are different frames in time series.

Specifically, in even frames, the counter 104 resets the count value CNT of the previous frame to an initial value, and counts against the reset count value based on the signal generated by the APD 101 . Further, the counter 104 has a sign inverting unit 203 that inverts the sign of the count value CNT of the previous frame by adding 1 to the logically inverted value of the count value CNT of the previous frame in odd frames. Note that even frames and odd frames may be interchanged.

(Second embodiment)
In the first embodiment, the sign inverting unit 203 outputs a value obtained by inverting the positive/negative sign of the output value CNT of the flip-flop 200 . As a result, the solid-state imaging device 500 can calculate and compress the difference value of pixel values between frames without using a frame memory, and realize a high compression ratio at a low cost. Further, the sign inverting unit 203 outputs the 2's complement of the output value CNT of the flip-flop 200 as a value obtained by inverting the positive/negative sign of the output value CNT of the flip-flop 200 . This makes it possible to simplify the processing after the decompression unit 802 . However, since the sign inverting unit 203 needs not only to invert the logic value of the output value CNT of the flip-flop 200 but also to add 1, the scale of the counter 104 becomes large and the control of the counter 104 becomes complicated. There is a problem. This problem becomes more conspicuous as the number of pixels 100 increases.

In the second embodiment of the present invention, the sign inverter 900 of FIG. 9 outputs the 1's complement of the output value CNT of the flip-flop 200 . As a result, the imaging system 800 can reduce the circuit scale while simplifying the processing after the decompression unit 802 .

FIG. 9 is a diagram showing a configuration example of the synchronous counter 104 according to the second embodiment. Differences of this embodiment from the first embodiment will be described below. The counter 104 in FIG. 9 is obtained by providing a sign inverting section 900 instead of the sign inverting section 203 in the counter 104 in FIG. The sign inversion unit 900 outputs the logically inverted value of the output value CNT of the flip-flop 200 to the selection unit 202 as a one's complement of the output value CNT of the flip-flop 200 . Compared to the sign inverting unit 203, the sign inverting unit 900 does not have an adding unit that adds 1, so the circuit scale of the synchronous counter 104 can be reduced. The operations of the selector 202, the adder 201 and the flip-flop 200 are the same as those in FIG. Selecting section 202 outputs the output value of sign inverting section 900 to adding section 201 when signal INV_RST is at high level. The adding section 201 adds the voltage pulse PLS to the output value of the selecting section 202 and outputs the addition result to the flip-flop 200 . The flip-flop 200 holds the addition result of the addition section 201 and outputs the held value as the count value CNT.

When the synchronous counter 104 of FIG. 9 is composed of an asynchronous counter similar to that of FIG. 3, the voltage pulse PLS is added for the purpose of adding 1 after the high level signal INV_RST is issued as described in FIG. is no longer required to be controlled at a high level, and control becomes simple. In addition, since it is not necessary to connect the output node of the control unit 505 to the line of the voltage pulse PLS, the congestion of the wiring and the additional electric power for improving the driving capability of the transistor can be alleviated.

FIG. 10 is a diagram showing a configuration example of the compression unit 506 according to this embodiment. The compression unit 506 in FIG. 10 is obtained by adding a 2's complement unit 1000 to the compression unit 506 in FIG. 2's complementing section 1000 outputs the count value CNT of counter 104 to quantization section 600 as it is because the complement selection signal is at low level in the intra-frame compression frames at timings t401 to t403 and t406 to t407. In addition, since the complement selection signal is at a high level in interframe compression frames from timings t404 to t405, the 2's complement unit 1000 adds 1 to the count value CNT of the counter 104 and converts it to 2's complement representation. and outputs the addition result to quantization section 600 . As a result, the input value of quantization section 600 in FIG. 10 becomes the same as the input value of quantization section 600 in FIG. The operations of the quantization unit 600, the entropy coding unit 601, the code amount control unit 602, and the code amount measurement unit 603 are the same as those in FIG. According to the present embodiment, it is possible to reduce the circuit size and improve the controllability for inverting the positive and negative signs when the signal INV_RST is input.

Although the configuration in which the 2's complement unit 1000 in the compression unit 506 adds 1 has been described, the configuration is not limited to this. For example, the solid-state imaging device 500 has the compression unit 506 in FIG. 6, and the decompression unit 802 in FIG. may be one's complement to restore the pixel data of the current frame. In other words, the configuration of the plurality of pixels 100 should be simplified, and the complement representation process should be absorbed in a portion that is bundled at one location on the path.

As described above, the counter 104 and the two's complement unit 1000 generate the count value CNT based on the signal generated by the APD 101 in even frames. In odd frames, the counter 104 and the 2's complementing unit 1000 count the count value obtained by inverting the sign of the count value CNT of the previous frame based on the signal generated by the APD 101. Generate CNTs.

The counter 104 counts against the logically inverted value of the count value CNT of the previous frame in odd frames. The 2's complement unit 1000 adds 1 to the count value CNT counted by the counter 104 in odd frames.

(Third embodiment)
FIG. 11 is a diagram showing a configuration example of a solid-state imaging device 500 according to the third embodiment of the invention. Differences of this embodiment from the first embodiment will be described below. The solid-state imaging device 500 in FIG. 11 differs from the solid-state imaging device 500 in FIG. 5 in that it has two control lines 1101 and 1102 .

The control unit 505 outputs signals EVEN_INV_RST and EVEN_ZERO_RST to the pixels 100 in the even rows via the control line 1101 . Signals EVEN_INV_RST and EVEN_ZERO_RST correspond to signals INV_RST and ZERO_RST of FIG. 1, respectively.

Also, the control unit 505 outputs signals ODD_INV_RST and ODD_ZERO_RST to the pixels 100 in the odd rows via the control line 1102 . Signals ODD_INV_RST and ODD_ZERO_RST correspond to signals INV_RST and ZERO_RST of FIG. 1, respectively.

FIG. 12 is a timing chart showing the control method of the solid-state imaging device 500 according to this embodiment. The solid-state imaging device 500 converts an optical image into a digital signal by driving a plurality of pixels 100 in parallel.

EVEN_APD is a potential waveform generated by the APD 101 and the quench resistor 102 in the even-row pixels 100 and indicates the input potential waveform of the waveform shaping circuit 103 . ODD_APD is a potential waveform generated by the APD 101 and the quench resistor 102 in the odd-numbered pixels 100 and indicates the input potential waveform of the waveform shaping circuit 103 .

The voltage pulse EVEN_PLS is the output voltage of the waveform shaping circuit 103 in the pixels 100 of even rows. The voltage pulse ODD_PLS is the output voltage of the waveform shaping circuit 103 in the pixels 100 in the odd rows.

Signals CNT_RST, EVEN_INV_EN and ODD_INV_EN are generated by control unit 505 of FIG. Control unit 505 generates signals EVEN_ZERO_RST, EVEN_INV_RST, ODD_ZERO_RST and ODD_INV_RST based on signals CNT_RST, EVEN_INV_EN and ODD_INV_EN.

Specifically, when the signal EVEN_INV_EN is low level and the signal CNT_RST is high level, the signal EVEN_ZERO_RST becomes high level. Also, if the signal CNT_RST is high level when the signal EVEN_INV_EN is high level, the signal EVEN_INV_RST becomes high level.

Also, if the signal CNT_RST is high level when the signal ODD_INV_EN is low level, the signal ODD_ZERO_RST becomes high level. Also, if the signal CNT_RST is high level when the signal ODD_INV_EN is high level, the signal ODD_INV_RST becomes high level.

The signal EVEN_ZERO_RST is the signal ZERO_RST that is input to the pixels 100 in the even rows. The signal EVEN_INV_RST is the signal INV_RST that is input to the pixels 100 in even rows.

The signal ODD_ZERO_RST is the signal ZERO_RST that is input to the pixels 100 in the odd rows. The signal ODD_INV_RST is the signal INV_RST that is input to the pixels 100 in the odd rows.

The count value EVEN_CNT represents the output value of the counter 104 in the even row of pixels 100 . The count value ODD_CNT represents the output value of the counter 104 in the odd row of pixels 100 . The counter 104 performs the above-described counting and initialization operations with the period during which the signal CNT_RST is at low level as the imaging period. Note that the counter 104 counts when the signal CNT_RST is at low level, but may count according to a separate enable signal.

READ_EN represents the timing of outputting the output value of the counter 104 to the vertical transmission line 508 (FIG. 11), and is issued at the same timing as the signal CNT_RST. Although the signal READ_EN is issued at the same timing as the signal CNT_RST, it is not limited to this. Specifically, the transfer timing to the vertical transmission line 508 may be arbitrarily set by having separate readout signals and reset signals and shifting the timing of their issuance.

Next, a method for driving the pixel 100 will be described. At timing t400, the signal EVEN_INV_EN is at low level, the signal ODD_INV_EN is at high level, and the signal CNT_RST is at high level, so the signals EVEN_ZERO_RST and ODD_INV_RST are at high level. When the signal EVEN_ZERO_RST goes high, the counters 104 in the even rows clear the count value EVEN_CNT to 0 in synchronization with the clock signal. Further, when the signal ODD_INV_RST becomes high level, the count value ODD_CNT becomes a value obtained by inverting the sign of the immediately preceding count value ODD_CNT. Here, since the imaging is started and the register value of the counter 104 is an undefined value, the count value ODD_CNT is an undefined value. Actually, since the count values EVEN_CNT and ODD_CNT are reflected in synchronization with the clock signal, a delay occurs in the count values EVEN_CNT and ODD_CNT, but the description will be made assuming substantially the same timing.

At timing t401, the signals EVEN_ZERO_RST and ODD_INV_RST become low level, exposure starts, and the imaging period starts. Also, when the signals EVEN_ZERO_RST and ODD_INV_RST go low, the signal EVEN_INV_EN goes high and the signal ODD_INV_EN goes low. Next, when the signal CNT_RST becomes high level, the signal EVEN_INV_RST becomes high level instead of the signal EVEN_ZERO_RST, and the signal ODD_ZERO_RST becomes high level instead of the signal ODD_INV_RST.

At timing t402, a photon is incident on the APD 101, so that an avalanche-multiplied charge is generated in the APD 101, and the potential of the APD 101 changes. Since the APD 101 discharges electric charges via the quench resistor 102, it takes time for the APD 101 to reach a constant potential again. Waveform shaping circuits 103 in even and odd rows detect the falling edge of the potential of APD 101 and generate short-time voltage pulses EVEN_PLS and ODD_PLS, respectively. The voltage pulses EVEN_PLS and ODD_PLS are input to the even and odd row counters 104, respectively. The even-numbered counters 104 count the voltage pulses EVEN_PLS and the count value EVEN_CNT becomes one. The odd-numbered counters 104 count the voltage pulses ODD_PLS, and the count value ODD_CNT becomes an undefined value.

During timings t401 to t403, the even-row and odd-row counters 104 continue to increment the count values EVEN_CNT and ODD_CNT each time the voltage pulses EVEN_PLS and ODD_PLS are generated. At timings t401 to t403, five voltage pulses EVEN_PLS and five voltage pulses ODD_PLS are generated, the count value EVEN_CNT becomes 5, and the count value ODD_CNT becomes an undefined value.

At timing t403, the count value EVEN_CNT is 5 and the signals CNT_RST and READ_EN go high. When the signal READ_EN becomes high level, the count values EVEN_CNT and ODD_CNT are transmitted to the vertical transmission line 508 in order, and the imaging period is completed. The count value EVEN_CNT of 5 corresponds to the pixel value of the frame during the imaging period from timing t401 to t403. For the pixels 100 in even rows, the frames during the imaging period from timings t401 to t403 are frames for intra-frame compression.

Also, at timing t403, since the signal EVEN_INV_EN is at a high level, the signal EVEN_INV_RST is at a high level, and the value of the count value EVEN_CNT becomes -5, which is the sign-inverted value of 5. Also, since the signal ODD_INV_EN is at low level, the signal ODD_ZERO_RST is at high level and the value of the count value EVEN_CNT is zero.

At timing t404, the signals EVEN_INV_RST, ODD_ZERO_RST, EVEN_INV_EN and READ_EN become low level, the signal ODD_INV_EN becomes high level, exposure starts, and the imaging period of the next frame starts. At timings t404 to t405, the even-row and odd-row counters 104 continue to increment the count values EVEN_CNT and ODD_CNT each time the voltage pulses EVEN_PLS and ODD_PLS are generated, similarly to the timings t401 to t403.

Between timings t404 and t405, four voltage pulses EVEN_PLS and four voltage pulses ODD_PLS are generated. The even row counter 104 increments four times to a count value EVEN_CNT of -5 and outputs a count value EVEN_CNT of -1. The odd row counter 104 increments a count value ODD_CNT of 0 four times and outputs a count value ODD_CNT of 4. FIG.

At timing t405, the count value EVEN_CNT is -1 and the count value ODD_CNT is 4, and the signals CNT_RST and READ_EN go high as at timing t403. When the signal READ_EN becomes high level, the count values EVEN_CNT and ODD_CNT are sequentially transmitted to the vertical transmission path 508, and the imaging period is completed.

The count value EVEN_CNT of -1 is a value obtained by subtracting the number of voltage pulses EVEN_PLS generated in the frame in the previous imaging period from the number of voltage pulses EVEN_PLS generated in the frame in the current imaging period. That is, the count value CNT of -1 is the difference between the number of voltage pulses EVEN_PLS generated in the current frame and the previous frame. For the pixels 100 in even rows, the frames during the imaging period from timing t404 to t405 are frames for inter-frame compression.

A count value ODD_CNT of 4 is the number of voltage pulses ODD_PLS generated in the frame during the current imaging period, and corresponds to the pixel values of the pixels 100 in the odd rows. For the pixels 100 in the odd-numbered rows, the frame during the imaging period from timing t404 to t405 is a frame for intra-frame compression.

Also, at timing t405, since the signal EVEN_INV_EN is at low level, the signal EVEN_ZERO_RST becomes high level and the count value EVEN_CNT becomes zero. Also, since the signal ODD_INV_EN is at a high level, the signal ODD_INV_RST is at a high level, and the sign of the count value ODD_CNT is inverted from 4 to -4.

At timing t406, the signals CNT_RST, EVEN_ZERO_RST, ODD_INV_EN, ODD_INV_RST and READ_EN become low level, and the signal EVEN_INV_EN becomes high level.

At timings t406 to t407, the even-row and odd-row counters 104 continue to increment count values EVEN_CNT and ODD_CNT each time voltage pulses EVEN_PLS and ODD_PLS are generated, similarly to timings t401 to t403.

Since four voltage pulses EVEN_PLS are generated at timings t406 to t407, the even-numbered counter 104 increments the count value EVEN_CNT of 0 four times and outputs the count value EVEN_CNT of 4. For the pixels 100 in the even-numbered rows, the frame during the imaging period from timing t406 to t407 is a frame for intra-frame compression.

Since four voltage pulses ODD_PLS are generated at timings t406 to t407, the odd-numbered counter 104 increments the count value EVEN_CNT of -4 four times and outputs the count value EVEN_CNT of 0. For the pixels 100 in the odd-numbered rows, the frames during the imaging period from timing t406 to t407 are frames for inter-frame compression.

At timings t407 to t408, signals CNT_RST, EVEN_INV_EN, ODD_INV_EN, EVEN_ZERO_RST, ODD_INV_RST and READ_EN are controlled in the same manner as at timings t403 to t404. Count values EVEN_CNT and ODD_CNT are transmitted to vertical transmission line 508 .

After that, the pixels 100 in the even-numbered rows and the odd-numbered rows alternately repeat the operations similar to the timings t404 to t406 and the operations similar to the timings t406 to t408. The even-numbered counters 104 alternately output, as count values EVEN_CNT, the number of voltage pulses EVEN_PLS generated during the imaging period of the current frame and the difference between the number of voltage pulses EVEN_PLS generated between the current frame and the previous frame. The odd-numbered counters 104 alternately output the number of voltage pulses ODD_PLS generated during the imaging period of the current frame and the difference between the number of voltage pulses ODD_PLS generated between the current frame and the previous frame as the count value ODD_CNT.

Although the signals ODD_INV_RST and ODD_ZERO_RST and the signals EVEN_INV_RST and EVEN_ZERO_RST have been described as different signals, they are not limited to this. For example, even row pixels 100 may input signals EVEN_INV_EN and CNT_RST and internally generate signals EVEN_INV_RST and EVEN_ZERO_RST. Odd row pixels 100 may also input signals ODD_INV_EN and CNT_RST and generate signals ODD_INV_RST and ODD_ZERO_RST internally. Further, at least the signals EVEN_INV_RST and EVEN_ZERO_RST may be binary-encoded and used as reset selection signals for the pixels 100 in the even-numbered rows. Further, at least the signals ODD_INV_RST and ODD_ZERO_RST that are binary-encoded may be input to the pixels 100 in the odd-numbered rows and used as reset selection signals.

Next, the processing of the compression unit 506 will be described with reference to FIG. The processing of steps S701 to S705 is the same as in the first embodiment.

In step S706, the code amount control unit 602 transfers the QP used for quantization of each encoded block, the code allocation information (encoding parameter) of the entropy encoding unit 601, and the compression flag to the encoded data 604. Correspond and output. The compression flag is information indicating intra-frame compression or inter-frame compression.

In the frames during the imaging period from timing t401 to t403 in FIG. 12, the compression flags of the pixels 100 in the even rows are information indicating intra-frame compression, and the compression flags of the pixels 100 in the odd rows are information indicating inter-frame compression. be.

In the frames during the imaging period from timing t404 to t405 in FIG. 12, the compression flags of the even-numbered pixels 100 are information indicating inter-frame compression, and the compression flags of the odd-numbered pixels 100 are information indicating intra-frame compression. be.

In the frames during the imaging period from timing t406 to t407 in FIG. 12, the compression flags of pixels 100 in even rows are information indicating intra-frame compression, and the compression flags of pixels 100 in odd rows are information indicating inter-frame compression. be.

As described above, the compression unit 506 performs intra-frame compression on the pixels 100 on the even-numbered rows and inter-frame compression on the pixels 100 on the odd-numbered rows in frames during the imaging period from the timings t401 to t403. In addition, the compression unit 506 performs inter-frame compression on the pixels 100 in the even-numbered rows and intra-frame compression on the pixels 100 in the odd-numbered rows in frames during the imaging period from timing t404 to t405. In addition, the compression unit 506 performs intra-frame compression on the pixels 100 on the even-numbered rows and inter-frame compression on the pixels 100 on the odd-numbered rows in frames during the imaging period from timing t406 to t407. Inter-frame compression can be performed without using a frame memory.

The solid-state imaging device 500 alternately performs intra-frame compression and inter-frame compression on the pixels 100 on the even-numbered rows on a frame-by-frame basis, and performs inter-frame compression and intra-frame compression on the pixels 100 on the odd-numbered rows on a frame-by-frame basis. Alternate compressions. Thereby, the amount of data after compression can be smoothed.

Next, the difference of the imaging system 800 of FIG. 8 from the first embodiment will be described. The decompression unit 802 performs decompression processing on the encoded data based on the DP, the encoding parameter, and the compression flag associated with the encoded data, and outputs decompression result data to the capture unit 803 . The expansion unit 802 outputs the expansion result of the current frame to an internal storage device such as a DRAM when the compression flag of the pixel 100 in the even-numbered row or the even-numbered row is information indicating intra-frame compression. Further, when the compression flag of the pixels 100 in even or odd rows is information indicating inter-frame compression, the decompression unit 802 reads out the decompression result of the previous frame stored in the internal storage device, and performs inter-frame compression. Decompress the current frame by referencing it. The decompression unit 802 synthesizes the even-numbered row decompression result and the odd-numbered row decompression result into one image, and outputs the synthesized image to the capture unit 803 .

As described above, according to this embodiment, intra-frame compression and inter-frame compression are alternately performed on even-numbered rows and odd-numbered rows, so that the bandwidth of the transmission path can be set to an average bandwidth between frames. .

As shown in FIG. 12, the even-numbered counter 104 generates a count value EVEN_CNT of the current frame based on the signal generated by the even-numbered APD 101 in the even-numbered frame. The even-numbered counter 104 counts the previous frame count based on the signal generated by the even-numbered APD 101 and the current frame count based on the signal generated by the even-numbered APD 101 in the odd-numbered frame. Generate a count value EVEN_CNT that indicates the numerical difference.

The odd-numbered counter 104 counts the count value of the previous frame based on the signal generated by the odd-numbered APD 101 in the even-numbered frame and the current frame count based on the signal generated by the odd-numbered APD 101 . A second count value ODD_CNT is generated that indicates the difference from the numerical value. Further, the odd-numbered counter 104 generates a count value ODD_CNT of the current frame based on the signal generated by the odd-numbered APD 101 in the odd-numbered frame. Note that odd-numbered rows and even-numbered rows may be interchanged. Here, all pixels 100 are divided into spatially distinct frames of pixels 100 in even rows and pixels 100 in odd rows. In that case, the even rows of pixels 100 correspond to the first frame and the odd rows of pixels 100 correspond to the second frame.

(Fourth embodiment)
FIG. 13 is a diagram showing a configuration example of a solid-state imaging device 500 according to the fourth embodiment of the invention. The solid-state imaging device 500 uses the frame memory 1301 to smooth the transmission band between frames. Differences of this embodiment from the third embodiment will be described below. A solid-state imaging device 500 in FIG. 13 is obtained by providing a frame memory 1301 and a compression unit 1302 in place of the compression unit 506 in the solid-state imaging device 500 in FIG. The frame memory 1301 stores the count value CNT of the horizontal transmission line 509 and outputs the stored count value CNT to the compression section 1302 . The compression unit 1302 receives the count value CNT of the previous frame from the frame memory 1301, receives the count value CNT of the current frame from the horizontal transmission line 509, and compresses them.

The control unit 505 does not output the signal EVEN_INV_RST to the pixels 100 in the even rows via the control line 1101, but outputs the high-level pulse signal EVEN_ZERO_RST between each frame. 2 outputs “0” to the addition unit 201 when the signal EVEN_ZERO_RST is at high level, and outputs the output value CNT of the flip-flop 200 when the signal EVEN_ZERO_RST is at low level. Output to the addition unit 201 . The even-numbered counter 104 outputs the number of occurrences of the voltage pulse EVEN_PLS as the count value EVEN_CNT at timings t401 to t403 in all frames.

In addition, the control unit 505 does not output the signal ODD_INV_RST to the pixels 100 in the odd rows via the control line 1201, but outputs the high-level pulse signal ODD_ZERO_RST between frames. 2 outputs “0” to the addition unit 201 when the signal ODD_ZERO_RST is at high level, and outputs the output value of the flip-flop 200 when the signal ODD_ZERO_RST is at low level. CNT is output to addition section 201 . The odd-numbered counter 104 outputs the number of occurrences of the voltage pulse ODD_PLS as the count value ODD_CNT at timings t404 to t405 in all frames.

FIG. 14 is a diagram showing a configuration example of the compression unit 1302 of FIG. 13. As shown in FIG. A compression unit 1302 in FIG. 14 is obtained by adding a subtractor 1400 and a selection unit 1401 to the compression unit 506 in FIG. Subtractor 1400 subtracts the count value EVEN_CNT or ODD_CNT of the previous frame input from frame memory 1301 from the count value EVEN_CNT or ODD_CNT of the current frame input from horizontal transmission line 509 , and outputs the subtraction result to selection section 1401 . That is, the subtractor 1400 outputs the difference between the count value of the current frame and the count value of the previous frame as an inter-frame difference signal.

The output value of the subtractor 1400 corresponds to the count value EVEN_CNT of -1 in the frame between timings t404 and t405 and the count value ODD_CNT of 0 in the frame between timings t406 and t407 in FIG. The value of the horizontal transmission line 509 corresponds to the count value EVEN_CNT of 5 in the frame from timing t401 to t403, the count value ODD_CNT of 4 in the frame from timing t404 to t405, and the count value EVEN_CNT of 4 in the frame from timing t406 to t407. do.

The selection section 1401 outputs the output value of the subtractor 1400 or the output value of the horizontal transmission path 509 to the quantization section 600 according to the selection signal of the control section 505 . As shown in FIG. 12, selection section 1401 outputs the output value of subtractor 1400 to quantization section 600 in the case of an odd-numbered row in an even-numbered frame, and outputs the output value of horizontal transmission line 509 in the case of an even-numbered row. The output value is output to quantization section 600 . Further, as shown in FIG. 12, in an odd-numbered frame, selector 1401 outputs the output value of subtractor 1400 to quantization section 600 in the case of even-numbered rows, The output value of 509 is output to the quantization section 600 .

FIG. 15 is a timing chart showing a control method of the solid-state imaging device 500. FIG. In FIG. 15, the delay time is ignored for simplification of explanation. The horizontal transmission path 509 indicates whether the count value transmitted from the horizontal transmission path 509 is an even-numbered row count value or an odd-numbered row count value. FrameCnt is the frame number of the current imaging period, and is incremented when the imaging period and readout period are completed. LineCnt is the line number of the count value transmitted from the horizontal transmission line 509 . RAM_AREA indicates a storage area of the frame memory 1301 corresponding to the frame number FrameCnt and line number LineCnt.

FIG. 16 is a diagram showing the storage area RAM_AREA of the frame memory 1301. As shown in FIG. The storage area RAM_AREA is an area of the frame memory 1301 accessed based on the frame number FrameCnt and line number LineCnt.

When the frame number FrameCnt is 0, and when the line number LineCnt is 0 and 2, the count value of the horizontal transmission line 509 is written to 0 and 1 in the storage area RAM_AREA, respectively. When the frame number FrameCnt is 0, and when the line numbers LineCnt are 1 and 3, the count values are read from 1 and 2 of the storage area RAM_AREA, respectively.

Similarly, when the frame number FrameCnt is 1 to 5, the count value of the horizontal transmission line 509 is written in the storage area RAM_AREA corresponding to the frame number FrameCnt and the line number LineCnt, or the count value is read from the storage area RAM_AREA.

When the control of the storage areas RAM_AREA of frame numbers FrameCnt 0 to 5 is completed, the next frame returns to the frame number FrameCnt 0, and the storage areas RAM_AREA of frame numbers FrameCnt 0 to 5 are controlled again.

By performing processing in this manner, the capacity of the frame memory 1301 does not have to correspond to the number of rows of the solid-state imaging device 500, so cost can be reduced. The capacity required by the control of the frame memory 1301 is the number of horizontal pixels×(the number of vertical rows/2+2).

In FIG. 15, RAM_CZ is a control signal for the frame memory 1301, and enables commands based on RAM_WZ, which will be described later, when it is at low level. RAM_WZ is a write control signal for the frame memory 1301, writes the count value of RAM_D to the storage area RAM_AREA when it is low level, and reads the count value from the storage area RAM_AREA to RAM_Q when it is high level. X in this timing chart indicates Don'tCare and indicates that any input is acceptable.

At timing t1500, the imaging period of the 0th frame is completed, and the count values of the 0th row are sequentially transferred via the vertical transmission path 508 and the horizontal transmission path 509. FIG. At this time, the frame number FrameCnt is the 0th frame and the line number LineCnt is the 0th line. The count value of the horizontal transmission path 509 in the 0th row of the current frame is written to 0 in the predetermined storage area RAM_AREA and is output to the compression unit 1302 at the same time.

At timing t1501, transfer from the horizontal transmission line 509 of the 0th row is completed, and transfer from the horizontal transmission line 509 of the 1st row is started. +1 is added to the line number LineCnt, and the count value of the first line of the previous frame stored in 1 of the predetermined storage area RAM_AREA is read out. The read count value of the first row of the previous frame is output to the compression unit 1302 together with the count value of the first row of the current frame. In compression section 1302 , subtractor 1400 subtracts the count value of the first line of the previous frame from the count value of the first line of the current frame, and outputs the subtraction result to quantization section 600 via selection section 1401 .

At timing t1502, as at timing t1500, the count value of the second even row is transferred via the horizontal transmission path 509. FIG. The count value of the second row of the current frame is written to 1 in the storage area RAM_AREA and is output to the compression unit 1302 at the same time. In compression section 1302 , selection section 1401 outputs the count value of the second row of the current frame to quantization section 600 .

At timing t1503, the count value of the third row, which is an odd number, is transferred via the horizontal transmission path 509, as at timing t1501. Readout processing of the count value in the third row of the previous frame stored in the predetermined storage area RAM_AREA 2 is performed. The read count value of the third row of the previous frame is output to the compression unit 1302 together with the count value of the third row of the current frame. In compression section 1302 , subtractor 1400 subtracts the count value in the third row of the previous frame from the count value in the third row of the current frame, and outputs the subtraction result to quantization section 600 via selection section 1401 .

At timing t1504, the output of the third row count value is completed, and the imaging period of the first frame starts.

At timing t1505, the imaging period of the first frame ends, and the count values of the 0th row of the first frame start to be sequentially transferred via the vertical transmission path 508 and the horizontal transmission path 509. FIG. At this time, the frame number FrameCnt is 1 and the line number LineCnt is 0. The count value of the 0th row of the previous frame is read out from 0 of the predetermined storage area RAM_AREA. The read count value of the 0th row of the previous frame and the read count value of the 0th row of the current frame are output to the compression unit 1302 . In the compression unit 1302 , the subtractor 1400 subtracts the count value of the 0th row of the previous frame from the count value of the 0th row of the current frame, and outputs the subtraction result to the quantization unit 600 via the selection unit 1401 .

At timing t1506, the transfer from the 0th row horizontal transmission line 509 is completed, and the transfer of the 1st row count value from the horizontal transmission line 509 is started. +1 is added to the line number LineCnt. The count value of the first row of the current frame of the horizontal transmission line 509 is written to 1 in the predetermined storage area RAM_AREA and is output to the compression section 1302 at the same time. In compression section 1302 , selection section 1401 outputs the count value of the first row of the current frame to quantization section 600 .

At timing t1507, the count value of the second row of the first frame is transferred via the horizontal transmission line 509, as at timing t1505. The count value of the second row of the previous frame is read out from 1 of the predetermined storage area RAM_AREA. The read count value of the second row of the previous frame and the read count value of the second row of the current frame are output to the compression unit 1302 . In compression section 1302 , subtractor 1400 subtracts the count value of the second line of the previous frame from the count value of the second line of the current frame, and outputs the subtraction result to quantization section 600 via selection section 1401 .

At timing t1508, the count value of the third row is transferred from the horizontal transmission line 509 as at timing t1506. The count value of the third row of the current frame of the horizontal transmission path 509 is written to 1 in the predetermined storage area RAM_AREA and is output to the compression section 1302 at the same time. In compression section 1302 , selection section 1401 outputs the count value of the third row of the current frame to quantization section 600 . At timing t1509, the processing of the first frame ends.

As described above, the solid-state imaging device 500 repeatedly controls the write and read operations to the frame memory 1301 of even-numbered rows and odd-numbered rows corresponding to even-numbered frames and odd-numbered frames. As a result, in even-numbered frames, only the odd-numbered rows refer to the odd-numbered rows of the previous frame, and the difference between the count values of the current frame and the previous frame can be obtained. In the odd frame, only the even rows refer to the even rows of the previous frame to obtain the difference between the count values of the current frame and the previous frame. The compression section 1302 can smooth the transmission path band between frames by compression processing.

A frame memory 1301 stores the count value of the previous frame counted based on the signal generated by the APD 101 . The subtractor 1400 calculates the count value of the previous frame based on the signal generated by the APD 101 and the current count value based on the signal generated by the APD 101 in the odd rows of the even frames and the even rows of the odd frames. Generate a count value that indicates the difference from the frame count value.

In this embodiment, the inter-frame reference operation for even-numbered pixels and odd-numbered pixels based on even-numbered frames and odd-numbered frames has been described, but the control is not limited to this. For example, it can be used for inter-frame referencing of pixels in horizontal even and odd columns based on even and odd frames.

In addition, in the technique of arranging two pixels under one microlens by dividing the exit pupil into two, the divided left pixel and right pixel are used as the even-numbered pixels and the odd-numbered pixels. A similar effect can be obtained even if the pixels are controlled as columns.

That is, the pixel types are classified into two types, and each type of pixel is controlled to be written to and read from the frame memory 1301 based on even-numbered frames and odd-numbered frames, respectively, and compression processing is performed. Smoothing of the transmission line band becomes possible.

The solid-state imaging device 500 of the first to fourth embodiments can be applied to smartphones, tablets, industrial cameras, medical cameras, vehicle-mounted cameras, etc., in addition to digital cameras and video cameras. Also, although an example in which the counter 104 counts the number of photons incident on the APD 101 has been described, the present invention is not limited to this. The counter 104 may perform analog-to-digital conversion by counting the time until the magnitude relationship between the photoelectric conversion signal generated by the photodiode and the ramp signal that changes with time is reversed.

It should be noted that the above-described embodiments are merely examples of specific implementations of the present invention, and the technical scope of the present invention should not be construed to be limited by these. That is, the present invention can be embodied in various forms without departing from its technical concept or main features.

101 APD, 102 quench resistor, 103 waveform shaping circuit, 104 counter, 506 compression section

Claims (32)

photoelectric conversion means for generating a signal by photoelectric conversion;
In the first frame, a count value is generated based on the signal generated by the photoelectric conversion means, and in the second frame different from the first frame , the signal generated by the photoelectric conversion means is generated. a generation means for generating a count value counted against a count value obtained by inverting the sign of the count value of the previous frame;
and compression means for compressing the count value generated by the generation means. The generating means resets the count value to an initial value in the first frame, and counts the reset count value based on the signal generated by the photoelectric conversion means. The imaging device according to claim 1 . The generation means resets the count value of the previous frame to an initial value in an even frame, counts the reset count value based on the signal generated by the photoelectric conversion means, and in an odd frame, 3. The count value according to claim 1, wherein a count value obtained by inverting the sign of the count value of the previous frame is generated based on the signal generated by the photoelectric conversion means. imaging device. The generating means resets the count value of the previous frame to an initial value in the odd-numbered frame, counts the reset count value based on the signal generated by the photoelectric conversion means, and counts the reset count value in the even-numbered frame, 3. The count value according to claim 1, wherein a count value obtained by inverting the sign of the count value of the previous frame is generated based on the signal generated by the photoelectric conversion means. imaging device. 5. The method according to any one of claims 1 to 4, wherein, in the second frame, the generating means counts the count value obtained by inverting the sign of the count value of the first frame. The imaging device described. The generating means has inverting means for inverting the sign of the count value of the first frame by adding 1 to the logically inverted value of the count value of the first frame in the second frame. 6. The imaging device according to claim 5, characterized in that: The generating means is
in the second frame, counting means for counting against a logically inverted value of the count value in the first frame;
5. The imaging apparatus according to claim 1, further comprising adding means for adding 1 to the count value of the second frame counted by said counting means. The imaging device has a plurality of pixels,
each of the plurality of pixels has the photoelectric conversion means and the generation means;
7. The imaging apparatus according to claim 5, wherein said compressing means sequentially compresses the count values generated by said generating means among said plurality of pixels. 5. The method according to any one of claims 1 to 4, wherein, in the second frame, the generating means converts the count value of the first frame into 2's complement representation after counting in 1's complement representation. The imaging device according to any one of items 1 and 2. photoelectric conversion means for generating a signal by photoelectric conversion;
In the first frame, a count value is generated based on the signal generated by the photoelectric conversion means, and in the second frame different from the first frame , the signal generated by the photoelectric conversion means is generated. generating means for generating a count value obtained by logically inverting the count value of the previous frame;
and compression means for compressing the count value generated by the generation means. 11. The imaging apparatus according to any one of claims 1 to 10, wherein the first frame and the second frame are different frames in time series. 11. The imaging apparatus according to claim 1, wherein said first frame and said second frame are spatially different frames made up of different photoelectric conversion means. a first photoelectric conversion means for generating a signal by photoelectric conversion;
a second photoelectric conversion means for generating a signal by photoelectric conversion;
In the first frame, first generation means for generating a first count value counted based on the signal generated by the first photoelectric conversion means;
In the first frame, the count value of the previous frame counted based on the signal generated by the second photoelectric conversion means and the count value counted based on the signal generated by the second photoelectric conversion means a second generating means for generating a second count value indicating a difference from the count value of the first frame;
and compression means for compressing the first count value generated by the first generation means and the second count value generated by the second generation means. In the second frame, the first generation means generates a count value of the previous frame counted based on the signal generated by the first photoelectric conversion means and the signal generated by the first photoelectric conversion means. generating a first count value indicating a difference from the count value of the second frame counted based on
14. The method according to claim 13, wherein said second generation means generates a second count value counted based on the signal generated by said second photoelectric conversion means in said second frame. The imaging device described. The first generation means generates a first count value of the current frame counted based on the signal generated by the first photoelectric conversion means in even frames, and generates the first count value in odd frames. a first frame indicating a difference between a count value of a previous frame counted based on the signal generated by the photoelectric conversion means and a count value of the current frame counted based on the signal generated by the first photoelectric conversion means; produces a count of
The second generating means, in an even-numbered frame, is based on the count value of the previous frame counted based on the signal generated by the second photoelectric conversion means and the signal generated by the second photoelectric conversion means. a second count value indicating a difference from the count value of the current frame counted in the odd-numbered frame; 15. The imaging apparatus according to claim 13, wherein the count value of . The first generation means resets the first count value to an initial value in the first frame, and generates the reset first count value based on the signal generated by the first photoelectric conversion means. generating a first count value counted against the count value;
In the first frame, the second generation means generates a count value obtained by inverting the sign of the second count value of the previous frame based on the signal generated by the second photoelectric conversion means. 16. The imaging apparatus according to any one of claims 13 to 15, generating a second count value counted by In the second frame, the first generation means generates a count value obtained by inverting the sign of the first count value of the previous frame based on the signal generated by the first photoelectric conversion means. generating a counted first count;
The second generation means resets the second count value to an initial value in the second frame, and generates the reset second count value based on the signal generated by the second photoelectric conversion means. 17. The imaging device of claim 16, wherein the imaging device generates a second count value counted against the count value. The first generation means resets the first count value of the previous frame to an initial value in an even-numbered frame, and generates the reset first count value based on the signal generated by the first photoelectric conversion means. Counting is performed with respect to the count value, and in odd frames, counting is performed with respect to the count value obtained by inverting the sign of the first count value of the previous frame based on the signal generated by the first photoelectric conversion means. generating a first count value with
In an even frame, the second generating means counts the count value obtained by inverting the sign of the second count value of the previous frame based on the signal generated by the second photoelectric conversion means. resetting the second count value of the previous frame to an initial value in an odd-numbered frame; and based on the signal generated by the second photoelectric conversion means, the reset second count value 18. The imaging device according to claim 16, wherein counting is performed with respect to a count value of two. 19. The second generating means according to any one of claims 16 to 18, wherein, in the first frame, the count value obtained by inverting the sign of the second count value of the previous frame is counted. 1. The imaging device according to item 1. In the first frame, the second generating means adds 1 to the logically inverted value of the second count value of the previous frame, thereby inverting the positive/negative sign of the second count value of the previous frame. 20. An imaging device according to claim 19, comprising means. The second generation means is
counting means for counting in the first frame against a logically inverted value of a second count value in the previous frame;
19. The imaging apparatus according to any one of claims 16 to 18, further comprising adding means for adding 1 to the count value counted by said counting means. The imaging device has a first pixel and a second pixel,
The first pixel has the first photoelectric conversion means and the first generation means,
the second pixel has the second photoelectric conversion means and the second generation means;
21. The compression means sequentially compresses the first count value generated by the first generation means and the second count value generated by the second generation means. The imaging device according to . The imaging device has a first pixel and a second pixel,
The first pixel has the first photoelectric conversion means and the first generation means,
the second pixel has the second photoelectric conversion means and the counting means;
22. The imaging apparatus according to claim 21, wherein said adding means adds 1 to the count value counted by said counting means. The imaging device has a memory for storing a count value of a previous frame counted based on the signal generated by the second photoelectric conversion means,
The second generating means generates, in the first frame, the count value of the previous frame counted based on the signal generated by the second photoelectric conversion means stored in the memory, and the second generating a second count value indicating a difference from the count value of the first frame counted based on the signal generated by the photoelectric conversion means of 10. The image pickup device according to claim 1. The imaging device has a plurality of photoelectric conversion means arranged in a matrix,
The first photoelectric conversion means are photoelectric conversion means for even rows,
25. The imaging apparatus according to claim 13, wherein the second photoelectric conversion means are photoelectric conversion means for odd rows. The imaging device has a plurality of photoelectric conversion means arranged in a matrix,
The first photoelectric conversion means are photoelectric conversion means for even columns,
25. The imaging apparatus according to any one of claims 13 to 24, wherein the second photoelectric conversion means are photoelectric conversion means of odd columns. 25. The imaging apparatus according to any one of claims 13 to 24, wherein said first photoelectric conversion means and said second photoelectric conversion means are arranged under one microlens. the photoelectric conversion means is an avalanche photodiode;
a waveform shaping means for outputting a voltage pulse based on the signal generated by the avalanche photodiode;
13. The imaging apparatus according to claim 1, wherein said generating means counts the number of said voltage pulses output from said waveform shaping means. the first photoelectric conversion means and the second photoelectric conversion means are avalanche photodiodes;
a waveform shaping means for outputting a voltage pulse based on the signal generated by the avalanche photodiode;
28. The imaging according to any one of claims 13 to 27, wherein said first generation means and said second generation means count the number of said voltage pulses output from said waveform shaping means. Device. a photoelectric conversion step of generating a signal by photoelectric conversion with a photoelectric conversion means;
The generation means generates a count value counted based on the signal generated by the photoelectric conversion means in the first frame, and generates the count value generated by the photoelectric conversion means in the second frame different from the first frame. a generation step of generating a count value counted with respect to a count value obtained by inverting the sign of the count value of the previous frame based on the signal obtained;
and a compression step of compressing the count value generated in the generation step by compression means. a photoelectric conversion step of generating a signal by photoelectric conversion with a photoelectric conversion means;
The generation means generates a count value counted based on the signal generated by the photoelectric conversion means in the first frame, and generates the count value generated by the photoelectric conversion means in the second frame different from the first frame. a generation step of generating a count value obtained by logically inverting the count value of the previous frame based on the received signal;
and a compression step of compressing the count value generated in the generation step by compression means. a first photoelectric conversion step of generating a signal by photoelectric conversion by the first photoelectric conversion means;
a second photoelectric conversion step of generating a signal by photoelectric conversion by the second photoelectric conversion means;
a first generation step of generating a first count value counted based on the signal generated by the first photoelectric conversion means in a first frame by the first generation means;
By the second generation means, in the first frame, the count value of the previous frame counted based on the signal generated by the second photoelectric conversion means and the signal generated by the second photoelectric conversion means a second generation step of generating a second count value indicating a difference from the count value of the first frame counted based on
and a compression step of compressing the first count value generated in the first generation step and the second count value generated in the second generation step by compression means. control method.

Images