CN103957365A - CMOS image sensor structure for realizing predictive coding image compression - Google Patents

Abstract

The invention relates to the integrated circuit design field of microelectronics and the digital image coding compression field. According to the technical scheme, a CMOS image sensor structure for realizing predictive coding image compression is used for achieving the purposes that on the basis of no reduction of image sensing quality, the area and power consumption introduced in due to a predictive coding module which is additionally used are reduced, predictive coding is completed while an image is acquired, and disorder caused by an operational amplifier in the coding process is eliminated. The CMOS image sensor structure is characterized in that firstly, a pixel value read from a pixel array is transmitted to a correlated double-sampling circuit to be subjected to correlated double-sampling so that fixed-pattern noise can be eliminated, then, a predictive coding circuit carries out predicted value obtaining operation under control of a sequential circuit, a column-level subtracter is used for obtaining the difference, namely a residual value, between the predicted value and an original pixel value after the predicted value is obtained, and finally, the residual value is subjected to analog-digital conversion to obtain final output codes. The CMOS image sensor structure for realizing predictive coding image compression is mainly applied to integrated circuit design.

Description

CMOS Image Sensor Architecture for Predictive Coding Image Compression

technical field

The invention relates to the field of integrated circuit design of microelectronics and the field of digital image coding and compression, in particular to a CMOS image sensor structure for realizing predictive coding.

technical background

The traditional CMOS image sensor-based video signal acquisition and processing process needs to quantify all pixel values and transmit them outside the chip for compression processing. The compressed data can be stored or transmitted. It is not difficult to find that the readout end of the CMOS image sensor reads all the data, and discards a large amount of redundant data in the subsequent compression process. The entire image compression process has done a lot of useless work, making the processing efficiency low. . If the compressed data can be obtained directly at the readout end of the CMOS image sensor, redundant data can be eliminated at the source, especially as the scale of the pixel array continues to increase, the amount of data to be processed continues to soar, such On-chip compression steps are more meaningful, and the continuous development of CMOS technology provides the possibility for on-chip compatible image compression.

Since image compression functions are mostly integrated on mobile devices, there are many restrictions on chip area and power consumption. This makes not all compression algorithms suitable for on-chip compression. Classic image compression algorithms include block matrix transformation, wavelet transformation, and predictive coding. The first two methods require complex operations such as vector multiplication and accumulation, and often require DSP To realize it, its hardware consumption is huge, and its advantages in image compression will be offset by its excessive power consumption and area. Relatively speaking, predictive coding is much easier to implement. It only needs to use the pixels around the current pixel to predict its value, and subtract the predicted value from the current pixel value to generate a residual value to complete predictive coding. In addition, image compression methods such as 2D-DCT and wavelet transform are all lossy image compression. In some fields, such as medical treatment, aerospace, etc., such lossy images are unacceptable. As a lossless image compression method, predictive coding has Incomparable advantages. Most of the existing focal plane compression methods are block-matrix conversion methods, which require a large number of switched capacitor circuits and huge capacitor arrays, which will not only amplify the low precision of analog circuits, but also make it difficult to meet the area requirements.

Contents of the invention

In order to overcome the deficiencies of the existing technology, a new CMOS image sensor structure that can realize predictive coding image compression is proposed. Compared with the traditional processing flow, the area and power consumption introduced by the additional predictive coding module (such as DSP or ASIC circuit for predictive coding transformation) are reduced without reducing the quality of image sensing. The predictive encoding is completed while the image is acquired, and the offset caused by the operational amplifier is eliminated during the encoding process. For this reason, the technical solution adopted by the present invention is a CMOS image sensor structure for realizing predictive coding image compression, specifically, the pixel values read out from the pixel array are first transmitted to a correlated double sample (CDS) circuit Correlated double sampling is carried out in order to eliminate the fixed pattern noise. Next, the predictive encoding circuit will perform the operation of obtaining the predicted value under the control of the sequential circuit. After obtaining the predicted value, the column-level subtractor will be used to obtain the predicted value and the original pixel value. The difference between is the residual value, and finally the residual value is subjected to analog-to-digital conversion (AD) to obtain the final output code;

The predictive encoding circuit is carried out in units of 2×2 pixel blocks. Every two pixels and two columns share a predictive encoding circuit. The 2×2 pixel blocks are transmitted to the predictive encoding circuit in sequence from top to bottom. The predictive encoding circuit It is composed of an integral circuit and a storage circuit. The integral circuit is used to calculate the average value, that is, the predicted value; the storage circuit is used to store the pixel values in the 2×2 pixel block in units of columns; the column-level subtractor is used to The pixel value is subtracted from the predicted value to find the residual value.

The integral circuit consists of two sampling capacitors C _S connected to two pixel columns, an operational amplifier, a non-inverting input control switch of the sampling capacitor C _S , an inverting input control switch of the sampling capacitor C _S , an operational amplifier feedback capacitor C _C , and an operational amplifier feedback capacitor _CC control switch, op amp feedback switch, switch between reference point and op amp feedback capacitor _CC .

The storage circuit is composed of two sampling capacitors respectively connected to the two pixel columns, an operational amplifier, an inverting input control switch of the sampling capacitor, a feedback capacitance control switch of the operational amplifier, and a feedback switch of the operational amplifier.

Technical characteristics and effects of the present invention:

The invention provides a CMOS image sensor structure capable of realizing predictive coding image compression. Integrating predictive coding into the inherent processing flow of CMOS image sensors completes lossless image compression, which has incomparable advantages over lossy image compression such as block matrix conversion. The analog circuit is used to complete the predictive coding work that can only be completed by DSP or complex ASIC circuits in the past, and has the inherent advantages of low power consumption and low area of the analog circuit. Predictive coding is carried out in 2×2 pixel blocks, and the predicted value can be obtained by averaging four pixel values. The algorithm is simple and easy to implement in hardware, and the errors caused by predictive coding will not be accumulated. While seeking the predicted value, the work of eliminating the offset of the operational amplifier is also completed. The output data can be encoded in any way, which improves the efficiency of the image sensor and is suitable for wireless sensing, video surveillance, biomedical and other fields.

Description of drawings

Figure 1 is a schematic diagram of the overall structure of the proposed CMOS image sensor;

Fig. 2 is the structural representation of predictive encoding circuit module;

Fig. 3 is a schematic diagram of the circuit principle of the predictive encoding module;

Fig. 4 is a schematic diagram of a sequential circuit of a predictive encoding module.

Detailed ways

In order to reduce the circuit complexity of the chip and improve the processing efficiency, the predictive coding operation is completed in the analog domain. The invention provides a CMOS image sensor structure capable of realizing predictive coding image compression. It consists of a pixel array, readout and correlated double sampling circuit, predictive encoding circuit, analog to digital converter (ADC) and timing control circuit, see Figure 1. The pixel values read from the pixel array are first sent to the correlated double sampling (CDS) circuit for correlated double sampling to eliminate fixed pattern noise, and then the predictive coding circuit will calculate the predicted value under the control of the sequential circuit After obtaining the predicted value, the column-level subtractor will be used to obtain the difference between the predicted value and the original pixel value, that is, the residual value. Finally, the residual value is subjected to analog-to-digital conversion (AD) to obtain the final output code. The main inventive part is the predictive coding circuit.

The overall architecture adopts the column-level processing form, and the predictive encoding is performed in units of 2×2 pixel blocks, so the predictive encoding circuit shares one for every two columns, and the 2×2 pixel blocks are sequentially transmitted to the predictive encoding in order from top to bottom. In the circuit, see Figure 2 for the module diagram and Figure 3 for the schematic diagram of the circuit. It is composed of an integral circuit and a storage circuit. The integral circuit is used to calculate the average value, that is, the predicted value. Specifically, the 2 The four pixel values in the ×2 pixel block are accumulated and the operation of dividing by four is completed by using the capacitance ratio. It is composed of capacitors CS, CC and the op amp A1. The op amp is shared. Two columns of pixels share an op amp for charge transfer. Operation, where the capacitance of CC is four times that of CS to complete the averaging operation divided by four; the storage circuit is used to store the pixel values in the 2×2 pixel block in units of columns, so each column uses one alone, which consists of C1, C2, Cf, A2, C1', C2', Cf', A3', the two columns have the same circuit structure. The capacitance of each capacitor is: CS=1/4CC=0.5pf, C1=C2=Cf=C1’=C2’=Cf’=0.5pf. ; The column-level subtractor is used to subtract the pixel value and the predicted value to obtain the residual value.

The working sequence diagram is shown in Figure 4. The entire predictive coding process is divided into three working states: reset phase, integral phase and readout phase. The following takes a 2×2 pixel block as an example to illustrate the specific working principle of the circuit. As shown in Figure 3, the pixel values after correlated double sampling are sequentially read out to the storage circuit and the predicted value generation circuit in units of columns. The pixels are read out simultaneously.

In the reset phase, Kr, K1, Kr', K1' are set high, the voltage across CS becomes VCS=Vref+Voff1-Vcds[i] (i=1 or 2, indicating the number of rows), and the charge on Cc is QCc =4C*Voff1, where Vref is the reference voltage of the operational amplifier, Voff1 is the offset voltage of A1, and Vcds1 is the pixel value after correlated double sampling. After the reset operation of the lower storage circuit, the voltage across Cf becomes VCf=Voff2, VCf’=Voff3, where Voff2 and Voff3 are the offset voltages of the two operational amplifiers A2 and A3.

In the averaging stage, K2 is set high, and the voltage across Cs becomes Voff1, so that the charge on Cs is transferred to Cc. At this time, QCc=4C*Voff1+C*(Vpix[1,1]+Vpix[1, 2]), the two pixel values in the first row, first column and first row, second column are in brackets. Such an operation will be repeated for another cycle, so that the four pixel values are all accumulated and divided by four. At this time QCc=4C*Voff1+C*(Vpix[1,1]+Vpix[1,2]+Vpix[2,1]+Vpix[2,2]), since the voltage across Cc is Vom and Voff1, So (Voff1+Vref-Vom)*4C=4C*Voff1+C*(Vpix[1,1]+Vpix[1,2]+Vpix[2,1]+Vpix[2,2]), you can get Vom =Vref-1/4C*(Vpix[1,1]+Vpix[1,2]+Vpix[2,1]+Vpix[2,2]). In this way, the average value of the pixel block can be obtained through the prediction circuit, and the offset voltage of the operational amplifier is also eliminated. While calculating the average value above, the pixel values are also stored in their respective storage circuits by column. C1, C1' and C2, C2' respectively store the pixel values of the first row and the second row. The specific operation is to use Two-phase non-overlapping clocks control the gating of S1 and S2 to read pixel values. The voltage across C1 is VC1=VCDS[1,2]-Voff2, the voltage across C2 is VC2=VCDS[2,2]-Voff2, the voltage across C1' is VC1=VCDS[1,1]-Voff3, C2 'The voltage at both ends is VC2=VCDS[2,1]-Voff3.

When the signal reading phase arrives, the prediction circuit has completed the averaging operation, so Vom maintains a stable output. The storage circuit completes the transfer of charges on C1, C1', C2 and C2' under the control of K1', K2', S1' and S2'. After reading out the two pixel values of the first row, QCf=-C*Voff2+C*VCDS[1,1], QCf’=-C*Voff2+C*VCDS[1,2].

So Vo1=Vref-Vpix[1,1], Vo2=Vref-Vpix[1,2]. Next, Vom, Vo1 and Vom, Vo2 will be sent to their respective column-level subtractors for difference operation, and the residual value is calculated as

$\mathrm{<span\; class="notranslate">\; VR</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; Vpix</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 4</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; Vpix</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span>$

Where VR[i,j] represents the residual value of the pixel value of row i, column j, The average of the four pixel values in the pixel block, which is the predicted value.

So far, the residual value of the pixel has been calculated, and then the residual value will be sent to the column-level ADC for quantization operation, and the quantized digital code value can be stored or sent to off-chip for entropy coding.

In order to make the purpose, technical solution and advantages of the present invention clearer, an example of specific numerical values will be given below in conjunction with the accompanying drawings for further detailed description.

Since the prediction circuit works in units of 2×2 pixel blocks, the overall architecture is based on such pixel units. Therefore, only specific numerical values are given for a 2×2 pixel block for illustration.

Now assume that the values of the four pixels in the pixel block after readout and correlated double sampling are V[1,1]=1.3, V[1,2]=1.4, V[2,1]=1.6, V[2 ,2]=1.7, then the pixel values V[1,1] and V[1,2] of the first row will be firstly sent to the prediction circuit for processing. They will be sent to their respective column-level integration circuits and sampling circuits. After one clock cycle, the voltage across CC in the integration circuit is (V[1,1]+V[1,2])/4, and the two C1 in each column-level storage circuit stores V[1,1] and V[1,2] respectively. When the second clock cycle arrives, V[2,1] and V[2,2] will be sent to the prediction circuit in the same way. After one clock cycle, the voltage value of CC in the integrating circuit is (V[1 ,1]+V[1,2]+V[2,1]+V[2,2])/4, this is the average value of the voltage in the pixel block, which is the predicted value, and the two column-level storage circuits C2 in will store V[2,1] and V[2,2] respectively. At this time, VCC=1.5 in the integration circuit, VC1=1.3, VC2=1.6 in the storage circuit of the first column, and VC1=1.4, VC2=1.7 in the storage circuit of the second column. So far, the work of the prediction circuit has been completed, and the calculation of the residual value will be carried out next.

The four pixel values and predicted values are sent to the respective column-level analog subtractors for subtraction operation, and the four residual values are respectively 0.2, 0.1, 0.1 and 0.2. Finally, the column-level ADC performs quantization operation on the residual value by column to obtain the final predictive coding value.

It can be seen that the present invention can integrate the predictive coding operation into the sensing process of the CMOS image sensor, which greatly improves the image compression processing efficiency.

Claims (3)

1. A CMOS image sensor structure for realizing predictive coding image compression, characterized in that the CMOS image sensor structure is specifically that the pixel values read from the pixel array are first transmitted to a correlated double sample (correlated doublesample, CDS) circuit Correlated double sampling is carried out in order to eliminate the fixed pattern noise. Next, the predictive encoding circuit will perform the operation of obtaining the predicted value under the control of the sequential circuit. After obtaining the predicted value, the column-level subtractor will be used to obtain the predicted value and the original pixel value. The difference between is the residual value, and finally the residual value is subjected to analog-to-digital conversion (AD) to obtain the final output code; The predictive encoding circuit is carried out in units of 2×2 pixel blocks. Every two pixels and two columns share a predictive encoding circuit. The 2×2 pixel blocks are transmitted to the predictive encoding circuit in sequence from top to bottom. The predictive encoding circuit It is composed of an integral circuit and a storage circuit. The integral circuit is used to calculate the average value, that is, the predicted value; the storage circuit is used to store the pixel values in the 2×2 pixel block in units of columns; the column-level subtractor is used to The pixel value is subtracted from the predicted value to find the residual value. 2. The CMOS image sensor structure for realizing predictive coding image compression as claimed in claim 1 is characterized in that the integrating circuit consists of two sampling capacitors _CS , an operational amplifier, and a sampling capacitor _CS connected to two pixel columns respectively. The non-inverting input control switch, the sampling capacitor C _S inverting input control switch, the op amp feedback capacitor C _C , the op amp feedback capacitor C _C control switch, the op amp feedback switch, and the switch between the reference point and the op amp feedback capacitor C _C. 3. the CMOS image sensor structure that is used to realize predictive coding image compression as claimed in claim 1 is characterized in that, storage circuit is connected to two sampling capacitors, operational amplifiers, sampling capacitor inverting input control of two pixel columns respectively switch, op amp feedback capacitor control switch, op amp feedback switch.